cuda.core: add LaunchConfig.programmatic_stream_serialization for programmatic dependent launch - #2456
cuda.core: add LaunchConfig.programmatic_stream_serialization for programmatic dependent launch#2456lijinf2 wants to merge 5 commits into
Conversation
…alization Allow users to set CU_LAUNCH_ATTRIBUTE_PROGRAMMATIC_STREAM_SERIALIZATION through LaunchConfig, matching the is_cooperative attribute pattern (NVIDIA#1334).
Add an end-to-end Hopper+ test that launches primary and secondary kernels on the same stream with programmatic_stream_serialization, and asserts overlap only when the PDL attribute is enabled (NVIDIA#1334).
Drop unused secondary sync/sleep from the overlap test, clarify the primary clock window comment, and print a short success line for CI.
github-actions
Bot
added
the
cuda.core
|
/ok to test d3b3b25 |
|
lijinf2
changed the title
| is_cooperative : bool, optional | ||
| Whether this config can be used to launch a cooperative kernel. | ||
| programmatic_stream_serialization : bool, optional | ||
| Whether to allow programmatic stream serialization (PDL). When True, |
There was a problem hiding this comment.
PDL is programmatic dependent launch, "programmatic stream serialization" is a sub-concept inside PDL?
There was a problem hiding this comment.
I think so. PDL looks to include both programmatic stream serialization and programmatic event.
CUDA programming guide PDL webpage shows an example of cudaLaunchAttributeProgrammaticStreamSerialization.
The page also describes cudaLaunchAttributeProgrammaticEvent but no code example.
| attr.value.cooperative = 1 | ||
| attrs.append(attr) | ||
|
|
||
| if config.programmatic_stream_serialization: |
There was a problem hiding this comment.
Non blocking Q: I see from the comment to this function "once all modules are cythonized, this function can be dropped in favor of the cdef method above". Are all modules cythonized?
There was a problem hiding this comment.
I have no answer. this cpdef function might still be needed for tests / non-Cython callers.
There was a problem hiding this comment.
Just checked codebase. This function is used in tests/test_launcher.py::test_to_native_launch_config_no_cluster, where there is a python caller "from cuda.core._launch_config import _to_native_launch_config"
| # PDL overlap is opportunistic; retry a few times on a quiet GPU. | ||
| saw_overlap = False | ||
| for _ in range(5): | ||
| if _run(secondary_cfg) == 1: | ||
| saw_overlap = True | ||
| break |
There was a problem hiding this comment.
Frankly, I have huge doubt over testing a non-deterministic behavior and asserting it in the test. We shouldn't be introducing flakiness in the test ourselves.
However kernel overlapping is opportunistic in nature. I vibe coded with my agent for a few rounds and I couldn't find a way to write a kernel and turn this into a deterministic test. My agent suggested that we should actually inspect the edge data from cuda graph to validate its dependency. But we don't have PDL in graph for now. (right?)
IMO, it's probably also acceptable to not expect saw_overlap here to always pass. Instead, we can use pytest.xfail when we didn't observe an overlap run. In that case, we validate the API via inspecting over a period of CI runs, not just through a single flaky failure.
There was a problem hiding this comment.
Yes, graph support is planed in next PR.
Done. Revised to xfail.
|
/ok to test 717fe7f |
2 participants
Description
Part of #1334
Expose
LaunchConfig.programmatic_stream_serializationto setCU_LAUNCH_ATTRIBUTE_PROGRAMMATIC_STREAM_SERIALIZATION, and add mapping plusHopper+ primary/secondary overlap tests.
Reference: https://docs.nvidia.com/cuda/cuda-programming-guide/04-special-topics/programmatic-dependent-launch.html
Release notes / Graph showcase will be in next PR.
Checklist