Conversation
Loader._static_backend picks a backend by asking each registered default backend's is_compatible whether it can load a stream. A probe that raised instead of returning False took the whole load down with it, so one backend's inability to parse bytes that are not its format decided the outcome for every other backend behind it in the probe order. Five of the twenty-one default probes raise on ordinary malformed input. ELF and ELFCore build a whole ELFFile to read e_type, so a truncated ELF raises out of pyelftools. Apk and Jar call ZipFile, so a local file header with no central directory behind it raises BadZipFile. UefiFirmware asks the stream for a file descriptor, so a stream with no fileno raises AttributeError and an empty file raises ValueError out of mmap. Guarding each of those in turn leaves the next backend exposed. The probes hand arbitrary bytes to pyelftools, zipfile and uefi_firmware, and the set of exceptions they raise is not bounded: ELF and ELFCore raise both ELFParseError and ELFError, and uefi_firmware's AttributeError is documented nowhere. The loop is the one place the property can be stated once. A probe that raises is now logged with its backend named and then treated as declining. The stream is rewound because a probe that raises leaves it wherever it stopped, and two probes read before they seek: PE.is_compatible and Minidump.is_compatible.
|
THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS Validation record for head Working tree clean. The diff is The regression is load-bearingThe fix was committed first, then reverted against the commit, so both arms run the same Complete local gateThe gate ran on
A green gate that skipped a suite is green over less than it appears to be. Absent Lint and type, against the merge base
Probe survey21 of the 22 registered backends are Measured at the raisers: ELF leaves the stream at 63 bytes in on A third reproducer, on cle's pinned dependencies
The failing member is the Hosted CI on this head
That is pyvex renaming its FFI parser cache into place at
A prediction of every check green was recorded before the push, naming two ways it could |
|
THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS Full loader output for Before — backend detection dies inside a probe, so the caller gets pyelftools' and mmap's cle masterAfter — the probe that cannot answer is logged with its backend named, the loop carries with this changeNeither file becomes loadable; both were unloadable and stay unloadable. What changes is that |
|
Corpus decompilation diffs can be found at angr/dec-snapshots@master...angr/cle_829 |
|
THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS The Re-running the Windows job on the identical head passed. angr/pyvex#569 fixes the |
2 participants
THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS
Problem
Loader._static_backendpicks a backend by asking each registered default backend'sis_compatiblewhether it can load the stream. A probe that raises instead of returningFalse takes the whole load with it, so one backend's inability to parse bytes that are not
its format decides the answer for every backend behind it.
Two examples on cle master, needing nothing but a stock install and
tests/x86_64/fauxwarefrom
angr/binaries:Neither file is loadable, but neither error is cle's. The first comes out of pyelftools via
ELF.is_compatible, the second out ofmmapviaUefiFirmware.is_compatible, and in bothcases every later backend,
Blobincluded, is never asked.Root cause
21 of the 22 registered backends are
is_defaultand get probed. Five are not totalfunctions of their input:
ELF,ELFCoreelf.py:289,elfcore.py:75ELFError,ELFParseErrorELFFileto reade_typeApk,Jarapk.py:243,jar.py:66BadZipFileUefiFirmwareuefi_firmware.py:56AttributeError,ValueErrorfileno; an empty fileELFis probed second, so a truncated ELF header ends the load before anything butCARTFilehas been asked.Fix
Hardening those five in turn leaves the next backend exposed and does not stay fixed. The
probes hand arbitrary bytes to
pyelftools,zipfileanduefi_firmware, and the set ofexceptions they raise is not bounded:
ELFandELFCoreraise bothELFParseErrorandELFError, anduefi_firmware'sAttributeErroris documented nowhere. The loop is the oneplace the property can be stated once.
Exception; that backend declines and the loop carries on. No probe in cleraises deliberately, so nothing intentional is swallowed.
traceback: a probe that cannot answer for bytes that are not its format is the expected
case, not a fault.
BaseException, soKeyboardInterruptandSystemExitend a load.CLECompatibilityError: Unable to find a loader backend ... Perhaps try the 'blob' loader?loader.py:735already logs a swallowed exception in exactly this form:log.warning("Dynamic load failed: %r", e).The rewind matters because a probe that raises leaves the stream wherever it stopped,
measured at 63, 32 and 512 bytes in for the raisers above, and two probes read before they
seek:
PE.is_compatible(pe.py:208) andMinidump.is_compatible(
minidump/__init__.py:92). Rewinding hands them the stream at the start.Overlap with
cle#720, which is ours and open. #720 hardens theUefiFirmwareprobe,so it also fixes the empty-file example above and the ar static archives behind it, and it
fixes a second defect this change cannot reach: that probe maps the whole container when
handed a slice of one. Where the two meet they agree on what
Loaderreturns, though #720also asserts
UefiFirmware.is_compatibleitself returns False, which this change does notmake true. #720's code does not touch
ELF,ELFCore,ApkorJar; this change coversthem too. The two need no ordering against each other.
Testing
tests/test_backend_probe.pyloads the unmodifiedtests/x86_64/fauxwarewithCARTFile.is_compatiblepatched to consume 100 bytes and raise.CARTFileis probed firstand
fauxwareis claimed byELFsecond, so the assertions are that ELF still loads withthe same entry as a clean load, that the failure is reported with the backend named, and that
the next probe finds the stream at position 0. All three fail on the merge base and pass
here.
Validation: #829 (comment)
session: sharpen