Repository navigation
Keep relocation site names unique for values sharing an objectid - #1117
Merged
Merged
Conversation
Relocation slots and relocatable boxes are named after the `objectid`
of the value they refer to, and an equal name was taken to mean an equal
value. Distinct values can share an `objectid`, though: a type only
contributes its 32-bit hash, which does not tell `W{Vector}` from
`W{Vector{Any}}`, so singletons, immutables and types built from such a
pair collide. A function referencing both failed to compile with
"refers to conflicting values", or "is already in use" for boxes.
Track which value each name was handed out for, and give a value whose
name is taken by another one an ordinal suffix. Globals are now visited
in module order rather than that of the `gv_to_value` dictionary, so
that the suffixes are the same in every session. A box is reused when
its value is referenced through several globals, as slots already were.
Fixes #1116.
The documentation promised that a relocation name always refers to the same value, so that a loader could put any number of objects in one symbol namespace and let their weak record definitions coalesce. Names cannot guarantee that across objects: jobs can share a namespace, and the names are derived from hashes. Two objects that disagree on a name then silently read each other's word. Describe what does hold: names are unique within an object, and a loader sharing a namespace has to merge the manifests it loads there with `add_relocation!`, which throws when slots disagree. Relocatable boxes cannot be checked that way, so objects containing them must not share a namespace. Records stay weak for the loaders that need it; only Julia's JIT before 1.14 forces objects into one `JITDylib`.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1117 +/- ##
=======================================
Coverage 93.04% 93.05%
=======================================
Files 32 32
Lines 8213 8222 +9
=======================================
+ Hits 7642 7651 +9
Misses 571 571 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relocation slots and relocatable boxes are named
<namespace>_<gv base>_<objectid>, and an equal name was taken to mean an equal value. Distinct values can share anobjectid, though: a type only contributes its 32-bit hash, and that hash treats a free type variable as its upper bound, soW{Vector}andW{Vector{Any}}hash the same. Singletons, immutables and types built from such a pair collide, and a function referencing both fails to compile:On device jobs the same happens for boxed isbits constants such as
B{Vector}(1)andB{Vector{Any}}(1), which fail with "Interior relocation global ... is already in use". Reactant runs into this when two calls in one function are inferred as returningVectorandVector{Any}.The first commit makes names unique within a module by construction: the collector tracks which value each name was handed out for, and a value whose name is taken by another one gets an ordinal suffix. Globals are visited in module order instead of the order of the
gv_to_valuedictionary, so the suffixes are the same in every session. A box is now reused when its value is referenced through several globals, as slots already were.I considered mixing a structural type hash into the names instead, which keeps a name a pure function of its value. It needs a reimplementation of Julia's type hashing, and still fails for immutables that hold a colliding type in a field.
The second commit corrects the documented contract for loaders. The docs promised that a name refers to the same value in every object, so that objects sharing a symbol namespace could let their weak record definitions coalesce. Names cannot guarantee that (jobs can share a namespace, and names come from hashes), and ORC coalesces weak definitions without comparing them: two objects that disagree on a name silently read each other's word. A loader that shares a namespace now has to merge the manifests it loads there with
add_relocation!, which throws when slots disagree. Records stayweak_odr, because Julia's JIT before 1.14 only offers a singleJITDylib; once no loader needs that they can become strong definitions, which turns a clash into a link error.Tested on Julia 1.11, 1.12 and nightly. The new collector test fails on main with the error above.
Fixes #1116