ELF: Select the ARM BE8 architecture when EF_ARM_BE8 is set

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Parked as a draft; do not review or merge in this shape. The checks above
are green only because [tool.uv.sources] pins archinfo to the be8/arch
branch of archinfo 374, which has been closed. The branch still resolves, so CI
still passes against a change that is not going to land; against archinfo
master this diff raises TypeError on every ARM load, not only on BE8 objects,
because it passes instruction_endness= on all ARM paths. What has to happen
before this is ready is in Fix.

Problem

ELF.extract_arch reads the ARM float-ABI bits out of e_flags but not
EF_ARM_BE8, so a BE8 object — big-endian data with little-endian
instructions, which ARMv6 introduced — is indistinguishable from a BE32 one.
The fixture pair in binaries 202 makes the difference visible: be8_loop has
e_flags=0x05800400 with EF_ARM_BE8 set, be32_loop has 0x05000400 with
it clear, and their .text sections are exact four-byte reversals of each
other. A BE8 object is loaded as fully big-endian, VEX fetches every
instruction byte-swapped, and across 48 BE8 ARM ELFs the baseline covers 4.892%
of the bytes marked as ARM code by $a mapping symbols.

Root cause

e_flags is consulted for the ABI bits and not for EF_ARM_BE8, and
arch_from_id has no way to express the split in the first place:
VexArchInfo carries one endness for both instruction fetch and data access.

Fix

Read the flag alongside the ABI bits and build the architecture with
little-endian instruction endness; a BE8 object carrying neither ABI bit
returns ArchARM directly rather than falling through to arch_from_id. Data
accesses are then tagged little-endian too, which is wrong, and separating the
two needs a libVEX change.

That is the design the parking note rejects. Ghidra already ships
ARM:LEBE:32:v7LEInstruction and ARM:LEBE:32:v8LEInstruction, declared
endian="big" instructionEndian="little", and its ARM opinion constrains on
the BE8 flag directly, so the split can be had today through the p-code path
without an archinfo change at all. Before this becomes reviewable, the layer
question has to be settled that way, and it has to account for cle 610, a
third-party attempt at ARM BE8 in this same file that has been open far longer.

Testing

tests/test_arch_detect.py covers both fixtures; on master the new case fails
with assert <Endness.BE> == <Endness.LE> for arch.instruction_endness. The
suite figures and the 48-object recovery measurement in the validation record
were taken with the closed archinfo branch on the path, so they describe a
dependency that no longer exists; they are kept as the size of the problem, not
as a claim about this branch. No output comparison is posted while this is
parked.

Validation: #793 (comment)

sync: angr/binaries#202

session: sharpen

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Validation record for head e38a4dc8, replacing the record for the withdrawn approach. The
description above still describes that withdrawn approach and is stale; it will be rewritten before
this leaves draft.

Why every check was red

The branch passed instruction_endness= on every ARM path, and that keyword exists only on
archinfo's be8/arch branch, which the branch pinned through [tool.uv.sources]. cle's ci / jobs
take their siblings from the shared image rather than from that pin, so they got archinfo master and
raised TypeError: ArchARM.__init__() got an unexpected keyword argument 'instruction_endness' on
44 of the 70 ARM ELF objects in angr/binaries - every object with either float-ABI bit set, so
armhf/fauxware (e_flags=0x05000402) and armel/test_arrays (0x05000202) included, against 0 on
the merge base. All eleven red checks were that one cause: the ten ci / Test shards
(elf.py:356/358 in the job logs), ci / Lint (three E1123 Unexpected keyword argument,
10.00 -> 9.86) and ci / Typecheck (three No parameter named "instruction_endness").

archinfo#374, which would have added the keyword, is closed: the ask there was to use pypcode and
ArchPcode instead.

What changed

e38a4dc8 takes exactly that route, so cle needs no archinfo change:

• the EF_ARM_BE8 branch returns archinfo.ArchPcode("ARM:LEBE:32:v7LEInstruction"), Ghidra's
  endian="big" instructionEndian="little" language, which Ghidra's own ARM ELF opinion selects on
  the same flag. pypcode is an optional cle dependency, so without it the branch falls through to
  master's answer with a log line and the test skips.
• every other ARM path is restored verbatim to master, and pyproject.toml is back to
  branch = "master".
• ALL_RELOCATIONS is keyed on arch.name, so the language id joins it beside the existing
  sparc:BE:32:default entry. Without that a dynamically linked BE8 object would silently lose every
  relocation.

Net diff is +48 -0 across elf.py, relocation/__init__.py and tests/test_arch_detect.py.

Evidence

Differential over every EM_ARM ELF in angr/binaries (68) plus the two BE8 fixtures = 70 objects,
loaded on merge base d2ecea06 and on e38a4dc8, comparing (class, name, memory_endness, instruction_endness): exactly one differs, armeb/be8_loop. The other 69 - 31 ARMEL, 25 ARMHF,
14 ARMCortexM - are byte-identical.

On the BE8 fixture, 0x2010c now decodes as cmp r0, #0x1; movlt r0, #0x0; master reads the same
bytes as smlatteq r0, r3, r0, r5. be8_loop (0x05800400) and be32_loop (0x05000400) differ
only in bit 23 and their .text are exact four-byte reversals.

Locally, with cle imported from the branch worktree: 13 of the named failing ARM tests in angr pass
(test_cfg_switches_armel, test_kepler_server_armhf, test_block_instruction_addresses_armhf,
test_func_in_added_segment_by_patcherex_arm, test_thumb_mode, test_armel_final_missing_block,
test_armel_cfgswitches_gcc, test_cfg_elf_no_section_headers, test_manysum_armel,
test_callable_c_manyfloatsum_armhf, test_fauxware_armhf, test_decompiling_amp_challenge03_arm,
test_decompiling_armhf_float_int_conversion), and three of them still fail with the exact CI
TypeError against the unfixed head. cle's own suite: 220 passed against 215 on the merge base, with
an identical pre-existing failure set. angrop is not checked out in this workspace and was not run.

Merge-base-relative lint and typecheck, reproducing the hosted jobs' comparison:

cle/backends/elf/elf.py                  10.00 -> 10.00   badness 0.2720283855706682 -> 0.2697947214076246
cle/backends/elf/relocation/__init__.py  10.00 -> 10.00   badness 0.0 -> 0.0
tests/test_arch_detect.py                10.00 -> 10.00   badness 0.0 -> 0.0

The Test (Pyodide) leg is the one environment with no pypcode, and it reported
tests/test_arch_detect.py as s.s..: test_elf_arm_be8 skipped exactly as intended while the
little-endian controls passed. That is the optional-dependency fallback confirmed on a real runner.

Why this stays in draft

is_arm_arch() is a prefix test on the architecture name, so it answers True for ARM:LEBE:..., and
CFGFast then reads switch_mode_on_nodecode, which a p-code architecture does not carry -
Project(be8_loop).analyses.CFGFast() raises AttributeError. archinfo#376 proposed narrowing the
predicate and was rejected, on the grounds that is_arm_arch() is meant to answer True for p-code ARM
languages too, so the repair belongs in angr and nobody owns it yet. No CI job covers this: no BE8
fixture reaches angr's suite, so these checks can go green while it is still true.

Two more things a reviewer should weigh. MetaELF._load_plt keys ATTEMPT 1 on the exact names
("ARM", "ARMEL", "ARMHF", ...), so a dynamically linked BE8 object no longer gets PLT stubs named
from its jmprel symbols; no fixture exercises it, because both BE8 fixtures are static. And the
p-code route loses the ARMEL/ARMHF float-ABI distinction for BE8 objects and moves them off
VEX/capstone onto the p-code engine, a trade that has not been measured.

The three platform legs' test_uefi_image_is_not_windows failure was unrelated to this diff:
angr/binaries#202 was behind binaries master and so lacked tests/riscv64/uefi/HighMemDxe.efi, which
a test on cle master loads. Master has since been merged into that fixture branch (46d75fe).

session: sharpen

[angr-bot]

Corpus decompilation diffs can be found at angr/dec-snapshots@master...angr/cle_793

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

Moving this to draft. It should not be reviewed or merged in its current shape,
and the checks above overstate how ready it is.

The green run is green because this pull request pins archinfo to the
be8/arch branch of angr/archinfo 374, and that pull request has been closed.
The branch still resolves, so CI still passes, but it is testing against a
change that is not going to land. Against archinfo master this diff raises
TypeError on every ARM load, not only on BE8 objects, because it passes
instruction_endness= on all ARM paths.

The closure was correct, and the investigation that followed it says so: Ghidra
ships ARM:LEBE:32:v7LEInstruction and ARM:LEBE:32:v8LEInstruction, declared
endian="big" instructionEndian="little", and its ARM opinion constrains on the
BE8 flag directly. That route works today without any archinfo change — a load
through it returns big-endian data from little-endian instruction decoding, which
is what BE8 means. VEX genuinely cannot represent the split, since it carries one
endianness for both fetch and data.

So the remaining question is not whether BE8 can be supported but which layer
should do it, and the answer likely costs this branch its current design. Parking
it rather than reworking it now, because there is also angr/cle 610, a
third-party attempt at ARM BE8 in this same file that has been open far longer;
whatever lands here should account for that rather than compete with it.

No action is requested. This note exists so the dead pin is not mistaken for a
working dependency by whoever looks next.

session: mega-corpus

[zardus]

THIS MESSAGE WAS GENERATED BY AN AUTOMATED PROCESS

The angr half of the block is open as angr/angr#7041. CFGArchOptions now resolves a
default ARM option set for any architecture is_arm_arch() accepts that its OPTIONS
table does not name, and the two other reads behind that predicate — arch.thumb_prologs
and Block.capstone — get the guards their own siblings in the same analysis already
carry. Nothing there narrows is_arm_arch(), which ltfish said on archinfo 376 is meant
to answer True for the p-code ARM languages as well as the VEX ones.

With it, CFGFast(normalize=True) completes on this branch's own be8_loop fixture
loaded as ArchPcode("ARM:LEBE:32:v7LEInstruction"), where angr master raises
AttributeError: 'CFGArchOptions' object has no attribute 'switch_mode_on_nodecode'.
All 22 p-code ARM languages complete, against 0 of 22 before.

Two things in the description above are stale against head e38a4dc8, and both should be
rewritten before this leaves draft. The parking note says the checks are green only
because [tool.uv.sources] pins archinfo to the closed be8/arch branch; the net diff
is 3 files and touches no pyproject.toml at all. And Fix still describes the withdrawn
instruction_endness= approach, while the code returns
archinfo.ArchPcode("ARM:LEBE:32:v7LEInstruction") — the pypcode route twizmwazin asked
for when he closed archinfo 374.