Skip to content

Fix prefetches of struct-typed buffers and under loop-dependent lets - #9612

Merged
alexreinking merged 1 commit into
mainfrom
alexreinking/prefetch-fixes
Nov 5, 2026
Merged

alexreinking merged 1 commit into
mainfrom
alexreinking/prefetch-fixes

Conversation

@alexreinking

Copy link
Copy Markdown
Member

This PR fixes two bugs in prefetch lowering. Both showed up when prefetching the weights in the GGML app (#9511).

Struct-typed buffers. StorageFlattening emitted a struct-typed prefetch call. Struct-typed Exprs may only appear as field()'s argument or as a Store's value, so compilation failed. The prefetch is now of the buffer's bytes: the offset and strides are in bytes, and the record's bytes are added as the innermost dimension.

Lets that depend on the loop. InjectPrefetch computes the prefetched box by advancing the loop variable by the offset in the body under the prefetch. Lets between the loop and the prefetch kept their values from the current iteration. Bounds inference creates such lets, for example for a realization computed inside a GuardWithIf tile. In the test, a tile of 4 rows reads 6 rows of the input, but the prefetch covered only 2 rows (and none in the tail). Those lets are now re-wrapped around the body, so they are evaluated at the prefetched iteration too.

correctness_prefetch gains two tests, run on all of its targets:

  • test13 prefetches a struct-typed buffer. Without this PR it fails to compile.
  • test14 checks the GuardWithIf case above. Without this PR it finds 2 prefetched rows instead of 6.

Validation:

  • ctest -R 'correctness_(prefetch|struct_type)$' on this branch: passed.
  • ctest -L correctness at the top of the stack (apps/ggml: GGML quantized kernels on Approximations #9511): 704/705 passed. The failure is correctness_cross_compilation, which fails because my LLVM build has no Hexagon target.

Stacked on #9599.

Checklist

  • Tests added or updated (not required for docs, CI config, or typo fixes)
  • Documentation updated (if public API changed)
  • Python bindings updated (if public API changed)
  • Benchmarks are included here if the change is intended to affect performance.
  • Commits include AI attribution where applicable (see Code of Conduct)

@alexreinking alexreinking added the skip_buildbots Do not run buildbots on this PR. Must add before opening PR as we scan labels immediately. label Oct 25, 2026
@alexreinking
alexreinking added this pull request to stack #9613 October 25, 2026 21:05
@alexreinking
alexreinking force-pushed the alexreinking/prefetch-fixes branch from 95ddbd5 to 82a5fe1 Compare October 26, 2026 19:58
Base automatically changed from alexreinking/arm-prefetch-line-size to main October 26, 2026 23:05
@alexreinking
alexreinking force-pushed the alexreinking/prefetch-fixes branch from 82a5fe1 to d645519 Compare October 26, 2026 23:05
@alexreinking
alexreinking removed this pull request from stack #9613 October 28, 2026 03:25
@alexreinking
alexreinking added this pull request to stack #9629 October 28, 2026 03:25
Two bugs in prefetch lowering:

- StorageFlattening emitted a struct-typed prefetch call, which is
  rejected, since struct-typed Exprs may only appear as field()'s argument
  or a Store's value. Prefetch the buffer's bytes instead: the offset and
  strides in bytes, with the record's bytes as the innermost dimension.

- InjectPrefetch computed the prefetched box from the body under the
  prefetch with the loop variable advanced by the offset, but the lets
  between the loop and the prefetch (e.g. bounds inference's, of a
  realization within a GuardWithIf tile) still held their values at the
  current iteration. Re-wrap those lets so they're evaluated at the
  prefetched iteration too. Before, the test's 6-row prefetch was 2 rows.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@alexreinking
alexreinking removed this pull request from stack #9629 November 2, 2026 23:50
@alexreinking
alexreinking force-pushed the alexreinking/prefetch-fixes branch from d645519 to 95988cd Compare November 2, 2026 23:50
@alexreinking
alexreinking added this pull request to stack #9654 November 2, 2026 23:50
@alexreinking
alexreinking merged commit 468c1cb into main Nov 5, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip_buildbots Do not run buildbots on this PR. Must add before opening PR as we scan labels immediately.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants