ELF: repair only the header bytes that are loaded

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Problem

An ELF whose section header table pyelftools cannot walk, and whose lowest mapped segment does not start at file offset 0, loads with four fields of its own ELF header written over the image. On the new tests/i386/bios.bin.truncated.elf six bytes of the loaded image differ from the file:

address    in memory        in the file
0xf9c30    74640000         06080000
0xf9c3e    280003000200     0000c5090000

0x6474 is that file's section header table offset, and 40, 3, 2 are e_shentsize, e_shnum and e_shstrndx beside it. They land at 0xf9c30 and 0xf9c3e because the restore addresses them from min_addr, and that image's only PT_LOAD starts at file offset 0x70.

Root cause

ELF.__init__ probes the section table, and when the walk raises it zeroes e_shoff, e_shentsize, e_shnum and e_shstrndx in a PatchedStream so pyelftools rereads the file from the program headers alone. Those zeroes reach memory with everything else the segments map, so the original bytes are stored back at the end of the constructor:

for offset, patch in patch_undo:
    self.memory.store(AT.from_lva(self.min_addr + offset, self).to_rva(), patch)

offset is a file offset into the ELF header -- 0x20 and 0x2e for a 32-bit ELF, 0x28 and 0x3a for a 64-bit one. min_addr + offset is that offset's address only when the lowest mapped segment sits at file offset 0. Nothing in ELF requires that, and nothing requires the header to be mapped at all.

Fix

offset_to_addr is what translates a file offset, so the restore asks it and skips a byte whose offset maps nowhere. Byte by byte, because the run can straddle a segment boundary.

This also stops an object with no PT_LOAD at all from raising out of the constructor: on master such a file reaches Clemory.store on an empty memory and dies with a KeyError naming the patched offset, and it now loads with nothing mapped and nothing restored. No tracked object has that shape, so nothing here tests it.

CGC subclasses ELF, so CGC binaries reach this loop too, and #726 repairs the same defect in the header that backend substitutes. NRFIN_00059 from the DARPA CGC corpus is a public reproducer for the pair -- lungetech/cgc-cfe-submission-corpus at 6e28340, file NRFIN_00059/2988916517-NRFIN_00059-5c96c811....rcb, whose first PT_LOAD starts at file offset 0x40. Master raises the TypeError #726 fixes and never loads it; this change alone leaves that same TypeError; #726 alone loads it with nine bytes overwritten by this defect; with both, none. No tracked CGC fixture has that shape, and the case below needs neither #726 nor a download.

Testing

tests/test_patched_stream.py::test_malformed_sections_unmapped_header loads the new fixture. ::test_malformed_sections, which already loaded tests/i386/oxfoo1m3, gains the same assertion for the case where the header is mapped. Both compare the whole loaded PT_LOAD against the file's own bytes rather than the four fields, so they fail on a wrong-address write anywhere in the segment. The new test fails on the merge base and passes with the change; deleting the restore loop entirely makes the oxfoo1m3 test fail with zeroes where the header bytes belong, so the loop is still load-bearing.

The fixture is new because nothing already tracked reaches this path: of the 861 tracked \x7fELF objects in angr/binaries at fc07821, one takes the fallback -- tests/i386/oxfoo1m3 -- and it is unaffected, its only PT_LOAD being at file offset 0. None of the 98 CGC fixtures takes it either. Merge angr/binaries#226 first. Validation: #828 (comment)

sync: angr/binaries#226

🤖 Generated with Claude Code

session: sharpen

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Full loaded image of the only PT_LOAD in tests/i386/bios.bin.truncated.elf, compared against the file bytes it was mapped from, before and after this change. The window is the region the restore writes to; the totals below it cover the whole segment.

Before — four ELF header fields are written over the image, because the restore addresses them from min_addr and this image's PT_LOAD starts at file offset 0x70:

angr/cle master 0e77ade

segment: file offset 0x70 -> vaddr 0xf9c10, filesize 0x63f0
offset_to_addr(0) = None   (the ELF header is mapped nowhere)

addr      loaded image                      file
000f9c20  27070000a7070000                 27070000a7070000
000f9c28  da070000f4070000                 da070000f4070000
000f9c30  7464000068090000                 0608000068090000                   <-- differs
000f9c38  a9090000b3092800                 a9090000b3090000                   <-- differs
000f9c40  03000200e9090000                 c5090000e9090000                   <-- differs
000f9c48  470c0000e00c0000                 470c0000e00c0000

bytes of the mapped PT_LOAD differing from the file: 6 of 25584

After — the restore asks offset_to_addr and writes nothing, because none of those file offsets is mapped:

with this change

segment: file offset 0x70 -> vaddr 0xf9c10, filesize 0x63f0
offset_to_addr(0) = None   (the ELF header is mapped nowhere)

addr      loaded image                      file
000f9c20  27070000a7070000                 27070000a7070000
000f9c28  da070000f4070000                 da070000f4070000
000f9c30  0608000068090000                 0608000068090000
000f9c38  a9090000b3090000                 a9090000b3090000
000f9c40  c5090000e9090000                 c5090000e9090000
000f9c48  470c0000e00c0000                 470c0000e00c0000

bytes of the mapped PT_LOAD differing from the file: 0 of 25584

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Validation record for head 6242c2f14220f061c0a2e1917882c67fd2b38779 against baseline 0e77ade3c39a3cee05f65051e57955675e1ac21b.

• Regression: python -m pytest tests/test_patched_stream.py -q — fails on baseline (1 failed, 2 passed: test_malformed_sections_unmapped_header, the loaded PT_LOAD differing from the file at segment index 32), passes on head (3 passed)
• Load-bearing check: with the restore loop deleted entirely, test_malformed_sections fails with zeroes at 0x8048020 and 0x804802e where tests/i386/oxfoo1m3's header bytes belong, so the loop is not simply switched off
• Focused: python -m pytest tests -q — baseline 261 passed, 9 skipped; head 262 passed, 9 skipped. Both arms in fresh interpreters under the gate environment
• Full suite: gate tree, 10 suites, 9 green — workspace, test-inputs, test-packages, mono, archinfo, pypcode, pyvex 65 passed, cle 262 passed / 9 skipped, angr-rust 642 passed / 0 failed. angr 2727 passed, 68 skipped, 2 xfailed, 3 failed, 4 errors — all 7 in tests/llm and tests/mcp, none outside those two directories: 2 collection errors on No module named 'pydantic', 3 failures on No module named 'pydantic_ai', 2 collection errors on No module named 'fastmcp'
• Lint/type: run-ci-diff-checks.py --base 0e77ade3c39a — 2 changed files; pylint cle/backends/elf/elf.py 10.00 -> 10.00 and tests/test_patched_stream.py 10.00 -> 10.00; pyright errors 46 -> 46 and 0 -> 0
• Hooks: black==26.5.1 and ruff==0.16.5 clean on both files, pyupgrade==3.21.2 rewrites nothing
• Fixture census: 861 tracked \x7fELF objects at angr/binaries fc07821, 1 takes the fallback (tests/i386/oxfoo1m3, unaffected, its only PT_LOAD at file offset 0); 98 tracked CGC fixtures, 0 take it
• Conflicts: git merge-tree --write-tree against the heads of CGC: Repair only the header bytes that are loaded #726, Relocate R_*_RELATIVE against the load bias, not the mapped base #773, ELF: do not relocate an image the linker has already finished with #801 and ELF: keep the section header table when one section's sh_link is dangling #802 — no conflict with any

Caveats, one line each:

• Not the complete gate. pre-commit and feature-instances were not requested (they need tooling and the network); pysoot and angr-management cannot run in the gate tree and are reported COMPONENTS ABSENT. Corpus sweeps were running on the shared trees, so entering the workspace shell would have reinstalled the editables they were using.
• The gate's worktree-cleanliness suite failed on files in the angr-agentic workspace edited by concurrent work during the run, none of them touched by this change.
• The angr suite residue above is missing optional dependencies in the gate tree, not a result of this change; a loader change cannot cause a missing-module import error.
• The gate's isolation check reported one changed shared object: a cffi artifact another agent's test run wrote inside its own feature instance, not in the gate tree and not one of the pinned native libraries.
• An ELF with no PT_LOAD that takes the fallback raises KeyError out of ELF.__init__ on master (the key is the patched offset) and loads with this change. No tracked object has that shape, so it is untested here.
• The CGC pairing in the description was measured on NRFIN_00059 from lungetech/cgc-cfe-submission-corpus at 6e28340, downloaded anonymously (HTTP 200, private: false, x-ratelimit-limit: 60); the tarball hashes to 6dc98398... and the member to 5c96c811.... No tracked CGC fixture has that shape, and the bios.bin.truncated.elf case needs neither it nor CGC: Repair only the header bytes that are loaded #726.

[angr-bot]

Corpus decompilation diffs can be found at angr/dec-snapshots@master...angr/cle_828