prevent AOTI library collisions between delegates (#21287)#21287
Open
shoumikhin wants to merge 1 commit into
Open
prevent AOTI library collisions between delegates (#21287)#21287shoumikhin wants to merge 1 commit into
shoumikhin wants to merge 1 commit into
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21287
Note: Links to docs will display an error until the docs builds have been completed. ⏳ No Failures, 40 PendingAs of commit b64eabe with merge base 8134bb2 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
Bot
added
the
CLA Signed
Contributor
|
@shoumikhin has exported this pull request. If you are a Meta employee, you can view the originating Diff in D113322459. |
This PR needs a
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes a correctness issue in the CUDA backend’s AOTInductor temporary shared-library handling by ensuring each delegate instance in a process writes to a distinct temporary .so path, preventing overwrites and subsequent symbol-lookup crashes when multiple identical partitions share the same so_blob_key.
Changes:
- Add a per-process
std::atomic<uint64_t>counter to disambiguate temporary.sofilenames for delegates that share the same content key. - Update the temp library naming scheme from
{key}{pid}.soto{key}{pid}_{counter}.so, while keeping the existing cleanup behavior on delegate destruction.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
shoumikhin
added a commit
to shoumikhin/executorch
that referenced
this pull request
Jul 23, 2026
Summary: The CUDA backend writes each compiled AOTInductor library to a temporary file, named from its content key plus the process id, before loading it. Two identical CUDA partitions can share the same content key, so both delegates computed the same path; loading the second library overwrote the first while it was still in use, and symbol lookup could then crash. Append a per-process atomic counter to the temporary file name so every delegate in a process gets a distinct path. The file is still removed when the delegate is destroyed, as before; only the name changes. Differential Revision: D113322459
shoumikhin
added a commit
to shoumikhin/executorch
that referenced
this pull request
Jul 23, 2026
Summary: The CUDA backend writes each compiled AOTInductor library to a temporary file, named from its content key plus the process id, before loading it. Two identical CUDA partitions can share the same content key, so both delegates computed the same path; loading the second library overwrote the first while it was still in use, and symbol lookup could then crash. Append a per-process atomic counter to the temporary file name so every delegate in a process gets a distinct path. The file is still removed when the delegate is destroyed, as before; only the name changes. Differential Revision: D113322459
meta-codesync
Bot
changed the title
shoumikhin
added a commit
to shoumikhin/executorch
that referenced
this pull request
Jul 23, 2026
Summary: The CUDA backend writes each compiled AOTInductor library to a temporary file, named from its content key plus the process id, before loading it. Two identical CUDA partitions can share the same content key, so both delegates computed the same path; loading the second library overwrote the first while it was still in use, and symbol lookup could then crash. Append a per-process atomic counter to the temporary file name so every delegate in a process gets a distinct path. The file is still removed when the delegate is destroyed, as before; only the name changes. Differential Revision: D113322459
shoumikhin
force-pushed
the
export-D113322459
branch
from
July 23, 2026 20:06
22c44c4 to
0e42901
Compare
shoumikhin
added a commit
to shoumikhin/executorch
that referenced
this pull request
Jul 23, 2026
Summary: The CUDA backend writes each compiled AOTInductor library to a temporary file, named from its content key plus the process id, before loading it. Two identical CUDA partitions can share the same content key, so both delegates computed the same path; loading the second library overwrote the first while it was still in use, and symbol lookup could then crash. Append a per-process atomic counter to the temporary file name so every delegate in a process gets a distinct path. The file is still removed when the delegate is destroyed, as before; only the name changes. Differential Revision: D113322459
Summary: The CUDA backend writes each compiled AOTInductor library to a temporary file, named from its content key plus the process id, before loading it. Two identical CUDA partitions can share the same content key, so both delegates computed the same path; loading the second library overwrote the first while it was still in use, and symbol lookup could then crash. Append a per-process atomic counter to the temporary file name so every delegate in a process gets a distinct path. The file is still removed when the delegate is destroyed, as before; only the name changes. Differential Revision: D113322459
shoumikhin
force-pushed
the
export-D113322459
branch
from
July 23, 2026 20:10
0e42901 to
b64eabe
Compare
shoumikhin
added a commit
to shoumikhin/executorch
that referenced
this pull request
Jul 23, 2026
Summary: The CUDA backend writes each compiled AOTInductor library to a temporary file, named from its content key plus the process id, before loading it. Two identical CUDA partitions can share the same content key, so both delegates computed the same path; loading the second library overwrote the first while it was still in use, and symbol lookup could then crash. Append a per-process atomic counter to the temporary file name so every delegate in a process gets a distinct path. The file is still removed when the delegate is destroyed, as before; only the name changes. Differential Revision: D113322459
Comment on lines
+339
to
+342
| filesystem::path so_path = temp_dir / | ||
| (so_blob_key + to_string(get_process_id()) + "_" + | ||
| to_string(so_file_counter.fetch_add(1, std::memory_order_relaxed)) + | ||
| ".so"); |
shoumikhin
added a commit
to shoumikhin/executorch
that referenced
this pull request
Jul 23, 2026
Summary: The CUDA backend writes each compiled AOTInductor library to a temporary file, named from its content key plus the process id, before loading it. Two identical CUDA partitions can share the same content key, so both delegates computed the same path; loading the second library overwrote the first while it was still in use, and symbol lookup could then crash. Append a per-process atomic counter to the temporary file name so every delegate in a process gets a distinct path. The file is still removed when the delegate is destroyed, as before; only the name changes. Differential Revision: D113322459
shoumikhin
added a commit
to shoumikhin/executorch
that referenced
this pull request
Jul 23, 2026
Summary: The CUDA backend writes each compiled AOTInductor library to a temporary file, named from its content key plus the process id, before loading it. Two identical CUDA partitions can share the same content key, so both delegates computed the same path; loading the second library overwrote the first while it was still in use, and symbol lookup could then crash. Append a per-process atomic counter to the temporary file name so every delegate in a process gets a distinct path. The file is still removed when the delegate is destroyed, as before; only the name changes. Differential Revision: D113322459
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
The CUDA backend writes each compiled AOTInductor library to a temporary file,
named from its content key plus the process id, before loading it. Two identical
CUDA partitions can share the same content key, so both delegates computed the
same path; loading the second library overwrote the first while it was still in
use, and symbol lookup could then crash.
Append a per-process atomic counter to the temporary file name so every delegate
in a process gets a distinct path. The file is still removed when the delegate is
destroyed, as before; only the name changes.
Differential Revision: D113322459