[alexreinking]

Moving a bit field at a varying position y to a constant position c,

((x >> y) & m) << c        // or with % 2^w in place of & m

currently lowers on ARM to a variable shift right (ushl by -y), a shift left by the constant c, and a mask. Neon's ushl/sshl (vshl on arm32) shift right for negative amounts, so the two shifts can be one:

(x << (c - y)) & (m << c)  // c - y as a signed shift amount

This holds for either signedness of x whenever 0 <= y < bits and 0 <= c < bits: every bit of the field lands in the same place, and bits shifted off the top are dropped by the shifted mask. CodeGen_ARM now does this rewrite for vector shift_lefts when constant_integer_bounds proves y in range. A constant y is left alone because LLVM already combines constant shifts. SVE is excluded because the [us]shl patterns aren't available there.

It isn't a target-independent simplifier rule because the fused form is worse on x86, where variable shifts with signed amounts expand to two shifts and a select.

The pattern comes from decoding bit-plane quantized weights, e.g. ggml's q5_0/q5_1, where the fifth bit of each 4-bit code is packed 8 to a byte: bit i % 8 of a repeated qh byte moves to bit 4. Per 16 lanes, aarch64 before:

ushl.16b v3, v4, v2
shl.16b  v3, v3, #4
and.16b  v3, v3, v0

and after:

ushl.16b v4, v5, v3
and.16b  v4, v4, v0

Benchmarks

Apple M-series, q5 kernels written in Halide (from the ggml app on #9561), LLVM 22. Ratio = ggml time / Halide time (higher is better), median of 3 rounds, all with #9574 applied (since superseded by #9606, now on main):

kernel	before	after
q5_0 · q8_0 vec_dot, K=4096	0.76	0.79
q5_1 · q8_1 vec_dot, K=4096	0.81	0.89
q5_0 gemv 4096×4096, 1 thread	0.80	0.93
q5_1 gemv 4096×4096, 1 thread	0.88	0.97
q5_0 gemv 4096×14336, 1 thread	0.81	0.99
q5_0 gemv 4096×4096, 10 threads	0.91	1.06

The q5_0 dot product loop drops from 58 to 56 instructions.

Tests

Adds correctness_arm_bit_field_shift:

• compares the vectorized path (rewritten) against the scalar path (not rewritten) for all eight integer types, both mask forms, field widths 1 and 3, several destinations c, and shift amounts with known bounds and with unknown bounds (which must not be rewritten);
• cross-compiles the q5 pattern for arm-64-linux and checks the assembly has a ushl and no shl. With the rewrite disabled, this check fails.

Validation:

• ctest -R "correctness_(arm_bit_field_shift|simd_op_check_arm|bitwise_ops|shift_by_unsigned_negated|intrinsics|arm_dot_product_by_element)$": passed
• ctest -L correctness -R "simd|vector|shift|bit|mod|math|intrin|narrow|widen|mul|div|saturat|round|cast|arm" -E cross_compilation: 101 tests, all passed or skipped (GPU, HVX, PowerPC, RISC-V not built). correctness_cross_compilation fails locally only because Hexagon isn't enabled in this build.

Checklist

• Tests added or updated (not required for docs, CI config, or typo fixes)
• Documentation updated (if public API changed)
• Python bindings updated (if public API changed)
• Benchmarks are included here if the change is intended to affect performance.
• Commits include AI attribution where applicable (see Code of Conduct)

[codecov]

Codecov Report

❌ Patch coverage is 98.00000% with 1 line in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (alexreinking/arm-repeat-shuffle@8f87766). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
src/IRMatch.h	95.65%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@                        Coverage Diff                         @@
##             alexreinking/arm-repeat-shuffle    #9575   +/-   ##
==================================================================
  Coverage                                   ?   73.13%           
==================================================================
  Files                                      ?      263           
  Lines                                      ?    82586           
  Branches                                   ?    20195           
==================================================================
  Hits                                       ?    60397           
  Misses                                     ?    15894           
  Partials                                   ?     6295           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.