Refuse packing through a read-only Clemory view

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Problem

Loader.gen_ro_memview() hands out a ClemoryReadOnlyView, and it is not read-only. store and __setitem__ refuse, but pack and pack_word write straight into the parent Clemory. Loading binaries/tests/x86_64/fauxware, calling gen_ro_memview(), and then trying all three writes at the entry point:

loader.memory before: 31ed4989d15e4889
store: NotImplementedError: ClemoryReadOnlyView does not support storing
pack: returned, no exception
pack_word: returned, no exception
loader.memory after:  9090909090909090

The write lands in loader memory, so the entry point is the rewritten one. CFGFast builds one of these views in _pre_analysis and discards it in _post_analysis, and Block.bytes, Clinic._convert_vex_fast and both the VEX and p-code lifters read through it while it exists.

No code in cle, angr or angr-management packs through the view today, so this is an unguarded public API rather than a live analysis bug. The validation record names the revisions, and an instrumented run counted zero pack and pack_word calls with a view receiver during CFGFast and Decompiler over eight fixtures on six architectures.

Root cause

ClemoryReadOnlyView inherits ClemoryBase.pack, which takes the backer out of self.backers(addr) and calls struct.pack_into on it. ClemoryReadOnlyView._flatten_backers appends the parent Clemory's own bytearray objects rather than copies -- that aliasing is what the class is for, since it exists to make reads fast -- so the pack_into writes into the parent. ClemoryBase.pack_word ends in return self.pack(...), so it inherits the same hole rather than having one of its own.

Fix

Override pack on ClemoryReadOnlyView to raise NotImplementedError, the way store and __setitem__ already do. One method covers both spellings, because pack_word goes through self.pack.

Deliberately not done: holding memoryview(backer).toreadonly() in _flatten_backers instead. That would close hand-mutation of a backer as well and cost no copy, but it changes the type backers() yields, and angr rejects that type. The VEX lifter failed on every fixture I ran, while cle's own suite stayed green.

Testing

New test_clemory_read_only_view_refuses_writes in tests/test_clemory.py loads binaries/tests/x86_64/fauxware, calls gen_ro_memview(), and asserts for store, pack and pack_word both that the call is refused and that loader.memory at the entry point is byte-identical afterwards. It fails on master, where the two pack calls succeed and rewrite the entry point. CFGFast and Decompiler over eight tracked fixtures give identical function and decompilation counts on both revisions.

Validation: #826 (comment)

session: sharpen

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Validation record for head eb9ed4349646463ef4836e80ad6ebc347d5eb485 against baseline 0e77ade3c39a3cee05f65051e57955675e1ac21b.

• Regression: python -m pytest tests/test_clemory.py -k read_only_view — 1 passed on head; with git checkout 0e77ade3 -- cle/memory.py it is 1 failed at assert refused, because pack returns and rewrites the entry point
• Full suite: python -m pytest tests/ — 261 passed / 9 skipped / 0 failed on baseline, 262 passed / 9 skipped / 0 failed on head
• Lint/type: angr's merge-base-relative jobs, reproduced locally — pylint cle/memory.py 10.00 -> 10.00 and tests/test_clemory.py 5.12 -> 6.00; pyright errors cle/memory.py 0 -> 0 and tests/test_clemory.py 9 -> 9
• Hooks: pre-commit run --files cle/memory.py tests/test_clemory.py — exit 0, no file rewritten
• Push guards: test-input and stale-pin checks both exit 0 at the head
• Reachability: ClemoryBase.pack and pack_word instrumented to count calls whose receiver is a ClemoryReadOnlyView, over CFGFast plus up to three Decompiler runs on 8 tracked fixtures across 6 architectures (x86_64, i386, armel, mipsel, ppc, aarch64), with angr at 854269112a5ce5b1b356cf6d67bff9abaf986b55 — 0 such calls on either revision, and identical function counts on both (40, 41, 35, 38, 34, 23, 103, 39). The instrument was shown able to report a non-zero count: packing through the view by hand reports 2 pack and 1 pack_word on the baseline
• Enumeration: ClemoryReadOnlyView, gen_ro_memview and memory_ro_view appear at 13 lines in 5 files in angr 87411a719c96ddc3f3954590b1833b1789e19697 — one in CFGFast that builds the view, four guards and bindings in block.py, one binding in clinic.py, and seven parameter annotations and isinstance checks in the VEX and p-code lifters. None of the 13 calls a write method. The same three names appear at 0 lines in angr-management aa843e5c10645ba361620768d59642a460805e80, which has 340 tracked .py files and 114 lines matching loader
• Receivers: Loader.dynamic_load on binaries/tests/x86_64/libc.so.6 makes 1282 pack_word calls with the view already built, every one on a Clemory
• Rejected alternative, measured: holding memoryview(backer).toreadonly() in _flatten_backers also blocks the write and passes the cle suite 261/9/0, and breaks CFGFast on all 8 of the fixtures above with TypeError: Unsupported backer type <class 'memoryview'> from angr's VEXLifter._load_bytes

Caveats: the fixture is binaries/tests/x86_64/fauxware, already tracked in angr/binaries at 003e82a2, so no sibling pull request is needed. The reachability and enumeration results are scoped to cle, angr and angr-management at the revisions named above and say nothing about third-party callers.

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Writing through Loader.memory_ro_view on binaries/tests/x86_64/fauxware, before and after this change. Run from a cle checkout with binaries beside it:

import cle

ld = cle.Loader("../binaries/tests/x86_64/fauxware", auto_load_libs=False)
ld.gen_ro_memview()
view = ld.memory_ro_view
entry = ld.main_object.entry

print("loader.memory before:", ld.memory.load(entry, 8).hex())
for label, call in [
    ("store", lambda: view.store(entry, b"\x90" * 8)),
    ("pack", lambda: view.pack(entry, "8s", b"\x90" * 8)),
    ("pack_word", lambda: view.pack_word(entry, 0x9090909090909090)),
]:
    try:
        call()
        print(f"{label}: returned, no exception")
    except NotImplementedError as e:
        print(f"{label}: NotImplementedError: {e}")
print("loader.memory after: ", ld.memory.load(entry, 8).hex())

Before — store is refused, the two pack calls are not, and the entry point in loader memory is overwritten:

cle master 0e77ade

loader.memory before: 31ed4989d15e4889
store: NotImplementedError: ClemoryReadOnlyView does not support storing
pack: returned, no exception
pack_word: returned, no exception
loader.memory after:  9090909090909090

After — all three are refused and loader memory is unchanged:

with this change

loader.memory before: 31ed4989d15e4889
store: NotImplementedError: ClemoryReadOnlyView does not support storing
pack: NotImplementedError: ClemoryReadOnlyView does not support packing
pack_word: NotImplementedError: ClemoryReadOnlyView does not support packing
loader.memory after:  31ed4989d15e4889

[angr-bot]

Corpus decompilation diffs can be found at angr/dec-snapshots@master...angr/cle_826