ELF: Take the word size from the machine, not the container

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Problem

ELF.extract_arch takes the machine name from e_machine and the width from
reader.elfclass, which agree for every ordinary object. The x32 ABI is where
they do not: an ELFCLASS32 container holding EM_X86_64 code, so the class
gives the pointer width and the machine gives the instruction set. Resolved by
the class, arch_from_id returns 32-bit X86 and the instruction stream is
decoded as x86. On tests/x86_64/x32_relocatable.o:

arch: <Arch X86 (LE)>
  0x00400000  55       push ebp
  0x00400003  8d0437   lea eax, [edi + esi]

where the same bytes are push rbp and lea eax, [rdi + rsi]. On a real x32
object the divergence is not cosmetic — the effect on block and function
recovery is in the validation record.

Root cause

The width comes from the container:

return archinfo.arch_from_id(arch_str, "le" if reader.little_endian else "be", reader.elfclass)

reader.elfclass is the pointer width, which for x32 is genuinely 32. The
instruction set is a property of e_machine, and nothing consults it.

Fix

Take the width from the machine for EM_X86_64. That fixes the decode and
leaves the pointer width modelled as 64-bit where the ABI has 32, which is a
real approximation with a real cost: relocation slots are then written eight
bytes wide into four-byte slots, which loses roughly one object in forty to a
failed load. That cost is measured in the validation record. A faithful x32
architecture — x86-64 instructions with 32-bit pointers — would fix both and is
a larger piece of work than this repair.

Testing

tests/test_arch_detect.py::TestArchDetect::test_elf_x32 asserts
isinstance(ld.main_object.arch, archinfo.ArchAMD64) on the fixture in
angr/binaries#198 and fails that assertion on master. Across 17 x32 ELFs,
unwind-table function starts covered by no recovered block go from 69,664 to 2;
the population, the whole-suite results and the relocation-width measurement
are in the validation record.

Validation: #791 (comment)

sync: angr/binaries#198

session: sharpen

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Validation record for head 7090300b108d58a0820dc5aa5ffa10d8fad3f5d3 against baseline 0e77ade3c39a3cee05f65051e57955675e1ac21b.

• Reproducer, on the baseline: an ELFCLASS32 object whose e_machine is
  EM_X86_64 loads as X86 at 32 bits. On this head it loads as AMD64.
  Confirmed on both the 1.2 KB fixture and a real 11.5 MB x32 executable

• Effect on decoding, measured on that 11.5 MB object with the sweep's own
  CFGFast parameters (normalize=True, data_references=False, resolve_indirect_jumps=True, force_complete_scan=False):

  loaded as	nodes	functions	FDE starts covered
  X86 (baseline)	42,979	25,504	3,566 / 69,747
  AMD64 (this head)	272,696	74,613	69,747 / 69,747

  The 66,181 missed on the baseline is what the sweep recorded for this object

• Effect across the population, not one object. The same measurement over 17
  x32 ELFs (the object above plus 16 others, 13 KB to 11.5 MB, sampled at random
  from the objects the sweep flagged), comparing baseline against this head:
  69,664 FDE starts missed on the baseline, 2 on this head — 99.997%. Arch
  resolution goes X86/32 to AMD64/64 on all 17. The 2 residual misses are
  single FDEs in two objects (sha256 f30c0aba…, 30d40d38…) and are ordinary
  undiscovered-function tail, not an x32 artifact. Node and function counts on
  this head match a forced ArchAMD64 override exactly, so the change is
  behaviourally identical to selecting the architecture by hand

• Regression: tests/test_arch_detect.py::TestArchDetect::test_elf_x32. On the
  baseline it fails its isinstance(arch, ArchAMD64) assertion — the reproducer
  above, not an unrelated error

• Focused: python -m pytest -q tests/test_arch_detect.py — 2 passed

• Whole suite, head against baseline: tests/ gives 1 failed, 239 passed, 9
  skipped on both sides, and the one failure is test_clemory.py::test_cclemory,
  which fails identically on the baseline and does not touch this path

• Lint: ruff check and ruff format clean. pylint cle/backends/elf/elf.py
  8.86 against a baseline of 8.85; tests/test_arch_detect.py 7.39 against 7.37

• Typecheck: pyright cle/backends/elf/elf.py 46 against a baseline of 46;
  the test file 0 against 0

• The width is passed as the string "64" because arch_from_id(ident, endness, bits="") documents that parameter as a string and its body tests
  "64" in bits. Passing the integer works at runtime but adds a pyright
  diagnostic against the declared type

• Where this was found: a block-level sweep over material that is not public makes x32 the largest missing-code signal it recorded — 1,622 objects and 295,628 unwind-table function starts covered by no block, about half of that category. The 11.5 MB object measured above is its largest

• Scope: cle only. The fixture is Add an x32 relocatable object binaries#198

Known limitation, stated plainly. This models an x32 object's pointers as
64-bit, which they are not. It is an approximation chosen over the current
behaviour, where the instruction stream does not decode at all. A faithful x32
architecture — x86-64 instructions with 32-bit pointers — would need a new
archinfo class, and is a larger change than this one. If you would rather have
that, this PR is the measurement to size it against rather than the fix to keep.

Addendum 2026-08-26 — what the 64-bit approximation costs, measured.

The limitation stated at the end of this comment is sharper than "pointers are modelled too wide", and the detail matters for deciding whether to take this change as it stands.

Resolving the architecture to AMD64 also sets the word size to 64, so relocations are applied with struct_fmt='<Q' and write eight bytes into x32's four-byte relocation slots. The relocation classes are right — an x32 object's types really are R_X86_64_*, so selecting the amd64 relocation classes is correct — but the slot width is not. The file says so itself: on the object below, .rela.dyn has sh_entsize 12, i.e. Elf32_Rela, so the table is 32-bit-shaped while the types are x86-64.

Three outcomes, in order of how often they occur:

• Harmless, where the upper four bytes of the slot are already zero. This is the common case and is why the recovery numbers above hold.

• A corrupted neighbouring word, where they are not.

• A failed load, where the slot lies within eight bytes of the end of a backer. On an x32 shared object, sha256 6e16eee5…, applying R_X86_64_RELATIVE at dest_addr=0x187850 raises

  struct.error: pack_into requires a buffer of at least 127480 bytes for packing
  8 bytes at offset 127472 (actual buffer size is 127477)
  ...
  cle/backends/relocation.py line 151, in relocate
    self.owner.memory.pack_word(self.dest_addr, self.value)
  KeyError: 1603664
  

  The same object loads on master, so this is a regression introduced by this change rather than something it exposes. It was the only load failure among the forty largest x32 objects sampled, so the rate is roughly one in forty and depends on segment layout rather than on anything about the code.

A minimal object built with clang --target=x86_64-unknown-linux-gnux32 -shared -fPIC -nostdlib does not crash — it loads on both revisions — but the defect is still visible on it: this head writes 8 bytes where master writes 4, at the same destination address. So a fixture can demonstrate the width, though not the crash, which needs the layout coincidence.

None of this changes the recovery result above: 69,664 missed FDE-covered bytes to 2 across 17 objects, reproduced. It does change the trade being offered. A faithful x32 architecture — x86-64 instructions with 32-bit pointers — would fix the decoding and the relocation width together, and on this evidence that is the better change even though it is larger. If you would rather have that, this branch is the measurement to size it against; if you would rather take this now, the one-in-forty load failure is the cost.

Re-keyed 2026-09-04, after a rebase onto cle master 0e77ade3c39a3cee05f65051e57955675e1ac21b. The branch moved from 0b2056d3a8cfee414a3824ee175e7b4958a10b15 to 7090300b108d58a0820dc5aa5ffa10d8fad3f5d3 because angr master bumped its sibling pins from 9.3.4.dev0 to 9.3.5.dev0 on 2026-09-02, and ci / Build installs the branch's own pyproject.toml with uv pip install --no-sources, so a fresh run on the old head died on a .dev version no index publishes. check-stale-pins.py refuses the old head and exits 0 on the new one. This was a plain replay: no conflict, no hand resolution, no fixup. git range-diff 929991db..0b2056d3 0e77ade3..7090300b reports every commit =, and the branch's own added and removed lines are byte-identical across the move, 17 lines on each side. Master touched none of the files this change touches in the range rebased over. So every figure above describes the same patch on a new base. The opening line named fc9980672a4aeeee06f2aa80c90723fa2d4e9953 before this edit, which is one rebase further back than the head this pass moved: the branch had already been replayed once without the record being re-keyed. Measured the same way over the whole chain, the added and removed lines at fc998067 are byte-identical to those at the new head, 17 on each side, so the record and the branch have never described different patches.

Hosted CI at 7090300b108d58a0820dc5aa5ffa10d8fad3f5d3, read 2026-09-04T18:00Z: 20 checks, all green -- 18 check runs plus the docs/readthedocs.org:cle and pre-commit.ci - pr status contexts, in run 33891448669, attempt 2.

Correction, 2026-09-04. An earlier version of the paragraph above read this same head at 16:24Z as 15 success, 4 failure, 1 cancelled, with the failures on cle.errors.CLEFileNotFoundError for tests/aarch64/langdetect_go.macho and tests/aarch64/relocatable_object.macho -- fixtures cle master began needing on 2026-09-03 in b6ff025b (#808) and 3812052d (#787) -- and said the branch was waiting on a rebase of angr/binaries#198. angr/binaries#198 now sits on binaries master 003e82a2bfa641530924055695b36cec8af483ab and carries both fixtures, and the whole run was re-run at this unchanged cle head. Nothing on this branch moved.

[angr-bot]

Corpus decompilation diffs can be found at angr/dec-snapshots@master...angr/cle_791

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

This came out of assembling the angr/vibr preview snapshot (every green open PR merged together, then each component's full test suite run against the result with an angr/binaries checkout). Both PRs are green on their own.

Combined with #734, this PR fails tests/test_elfcore.py::test_prstatus_abi_mismatch. #734's test builds a Linux x32 core (EM_X86_64 with ELFCLASS32) on the premise that cle picks X86 for it. This PR's bits = "64" if arch_str == "EM_X86_64" else reader.elfclass in cle/backends/elf/elf.py resolves the same core to AMD64, so ELFCore.__parse_auxv (cle/backends/elf/elfcore.py:409) unpacks 8-byte words from the core's 4-byte auxv note:

struct.error: unpack_from requires a buffer of at least 160 bytes ... (actual buffer size is 152)

Either PR alone passes; merged, x32 cores raise instead of loading. The auxv (and the rest of the note parsing) probably needs the container's word size even when the machine is 64-bit, rather than the arch detection reverting.

Until this changes, this PR is excluded from the preview (exclusions.txt in zardus/angr-agentic, skill angr-maintain-vibr). Its own new test test_elf_x32 also needs binaries #198 to be exercised.

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Architecture resolution and the resulting disassembly of .text in
tests/x86_64/x32_relocatable.o (the fixture in angr/binaries#198), before and
after this change. The command is the same on both sides:

import cle
ld = cle.Loader("tests/x86_64/x32_relocatable.o",
                main_opts={"backend": "elf"}, auto_load_libs=False)
m = ld.main_object
sec = m.sections_map[".text"]
data = ld.memory.load(sec.vaddr, sec.memsize)
for i in m.arch.capstone.disasm(data, sec.vaddr):
    print(hex(i.address), i.bytes.hex(), i.mnemonic, i.op_str)

Before — the ELFCLASS32 container resolves to 32-bit X86, and the same
bytes are read as an x86 instruction stream:

angr/cle master (a4fb800)

arch:  <Arch X86 (LE)>
bits:  32   memory_endness: Iend_LE
.text bytes: 5589e58d04375dc30f1f8400000000005589e58d047f5dc3
disassembly of .text with X86:
  0x00400000  55                   push ebp
  0x00400001  89e5                 mov ebp, esp
  0x00400003  8d0437               lea eax, [edi + esi]
  0x00400006  5d                   pop ebp
  0x00400007  c3                   ret 
  0x00400008  0f1f840000000000     nop dword ptr [eax + eax]
  0x00400010  55                   push ebp
  0x00400011  89e5                 mov ebp, esp
  0x00400013  8d047f               lea eax, [edi + edi*2]
  0x00400016  5d                   pop ebp
  0x00400017  c3                   ret 

After — EM_X86_64 decides the instruction set, and the stream is read as
what the toolchain emitted:

with this change

arch:  <Arch AMD64 (LE)>
bits:  64   memory_endness: Iend_LE
.text bytes: 5589e58d04375dc30f1f8400000000005589e58d047f5dc3
disassembly of .text with AMD64:
  0x00400000  55                   push rbp
  0x00400001  89e5                 mov ebp, esp
  0x00400003  8d0437               lea eax, [rdi + rsi]
  0x00400006  5d                   pop rbp
  0x00400007  c3                   ret 
  0x00400008  0f1f840000000000     nop dword ptr [rax + rax]
  0x00400010  55                   push rbp
  0x00400011  89e5                 mov ebp, esp
  0x00400013  8d047f               lea eax, [rdi + rdi*2]
  0x00400016  5d                   pop rbp
  0x00400017  c3                   ret 

This fixture is small enough that both decoders produce output; what differs is
which registers the operands name, and the register file the rest of the stack
then models. The population effect is in the validation record.

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Resolved: the pull request this could not be rolled up with

This pull request has been excluded from every mono rollup because applied
together with #734 it broke tests/test_elfcore.py::test_prstatus_abi_mismatch.
That turned out to be a genuine defect rather than an assertion pinning the old
arch, and it has been fixed at its root in #734.

Resolving an x32 core to AMD64 is right for the instruction set and wrong for
ELFCore.__parse_auxv, which sized an Elf_auxv_t entry with self.arch.bytes.
An auxv entry is two words of the container's width, so the 152-byte
NT_AUXV of tests/x86_64/elfcore_linux_x32.core -- 19 Elf32_auxv_t entries
-- ran off the end of the note at eight bytes per word and failed the load
outright:

struct.error: unpack_from requires a buffer of at least 160 bytes for unpacking
8 bytes at offset 152 (actual buffer size is 152)

The entries that did fit decoded to nonsense: the first pair read
0xffd4900000000021 instead of AT_SYSINFO_EHDR = 0xffd49000. So this is the
same class/machine distinction this pull request draws, in the one other place
in the ELF backend that had assumed the two agree -- not a second cost of the
64-bit pointer-width approximation the description already records.

cle 143abf3 on #734 takes the width from self._reader.elfclass. With it, the
auxv this branch's arch change would have mangled comes out byte-identical to
what a 32-bit read produces, AT_HWCAP = 0xbfebfbff and
AT_PHENT = sizeof(Elf32_Phdr) = 0x20 among it, and test_prstatus_abi_mismatch
passes with its assertions unchanged.

Verification

master + this branch + #734 at 143abf3, cle's whole suite:

1 failed, 250 passed, 9 skipped

The one failure is this branch's own test_elf_x32, which needs
tests/x86_64/x32_relocatable.o from the still-open angr/binaries PR 198 and
fails identically with this branch alone against binaries master. Nothing about
the interaction.

session: sharpen

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

An attempt to measure how much of a corpus's failure surface this removes, and
what that measurement can and cannot say.

Sample. 12,000 objects drawn uniformly at random, from a seeded permutation,
out of a 624,920-object internal corpus of compiler- and vendor-produced
binaries; 11,989 were retrievable and probed.

Method. Each object is loaded with the catalogue's declared load recipe and
CFGFast is taken as far as the first failure. A 1,717-object subset — the
whole ArchNotFound class plus 500 objects that already reach CFG — is probed
against master (eac0e5540516b9199dd6a91933e80dc774ea3eac) and against this
branch's head (fc9980672a4aeeee06f2aa80c90723fa2d4e9953) in one environment.

What the sample can say. Nothing about the benefit: the sample contains no
x32 objects at all. No object in the 11,989 declares the x32 ABI, and none of
the 1,212 ArchNotFound failures is an EM_X86_64 machine in an ELFCLASS32
container. So the case this change fixes does not occur here, and a "0 objects
cleared" figure for it would be a statement about the corpus, not about the
change.

What it can say. That nothing else moves. On this head the 1,212
ArchNotFound objects land exactly where they land on master, and 500 objects
that already reached CFG are unchanged — 0 of 500 differ. Forcing the word size
to 64 for EM_X86_64 does not disturb any other architecture in the sample,
which is the risk a reviewer would reasonably want bounded, since the edit sits
in the path every ELF takes.

session: sharpen

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Rebased onto 929991db after #795 and #766 landed; the conflict was in _arch_from_reader and the resolution keeps both paths, since the MIPS branch returns before bits is computed. 254 passed, 9 skipped on master and 255 passed, 9 skipped here, the extra being test_elf_x32. Its fixture sibling angr/binaries#198 was four commits behind and is rebased too — CI fetches the sync sibling at its own head, so without that this branch would have failed test_ppc64_abiv1_untyped_function_import on a fixture it never touched.

Worth raising, because #795 merged into this same function and reasons the opposite way. It handles MIPS n32/O64 — a 64-bit instruction stream in an ELFCLASS32 container — and says the word size must stay 32, because Elf32_Rel, Elf32_Sym and 4-byte GOT slots are all still 32-bit; only the ISA is 64-bit. x32 is that same shape, and this change resolves it as bits = 64, which gives ArchAMD64 and a 64-bit pointer width for an ABI whose pointers are 4 bytes.

The difference is availability, not principle: #795 had ArchMIPSN32 to return, and archinfo has no equivalent for x32 — it registers .*x32 as 32-bit ArchX86, where the instruction stream does not decode at all. So this is the better of the two options that exist today, and the consistent fix is an ArchX32 in archinfo with 32-bit pointers and the AMD64 instruction set, after which this line becomes a return like the MIPS one. Happy to open that if you would rather have it before this merges.

session: sharpen