Algorithm-level GPU test suite (matmul, attention, shared tiling)

[tetsuo-cpp]

Summary

Extend gpu_test/ beyond arithmetic/stack micro-ops with end-to-end numerical tests for real algorithms: naive matmul, shared-memory tiled matmul (when applicable), and scaled-dot-product attention.

Motivation

LIT covers translation and conversion thoroughly. gpu_test/test_kernels.py mostly checks +, DUP, simple stores, etc. Regressions in address spaces, barriers, float paths, or multi-param kernels can slip through until the GPT-2 demo is run by hand.

Algorithm tests also give a baseline for performance work (#10, #11, #12, stack flattening).

Proposed cases

Test	Validates
Naive matmul (small MNK)	multi-buffer params, GLOBAL-ID, loops
Tiled matmul + shared + BARRIER	S@/S!, synchronization
Attention (tiny seq/head)	f32 global, f64 shared, softmax, causal mask
Scalar + array mix	header + runner path
Reduced-width roundtrip	F32@/F32! correctness

Use numpy/torch reference implementations; assert allclose with documented tolerances.

Acceptance criteria

• At least matmul + attention GPU tests green on Vast (or documented local CUDA)
• Failures print expected vs actual diffs usefully
• Tests marked @pytest.mark.gpu and skip cleanly without API key / GPU
• Optional timing print under an env flag (not flaky asserts on perf)

Related

• GPU test infra / SSH fixes (Fix SSH timing issues in GPU test infrastructure #48 closed)
• Runner redesign (Replace warpforth-runner C++ CLI with Python cuda-python + JSON protocol #50)
• Demo attention (Naive attention kernel in Forth #44, GPT-2 interactive inference demo with WarpForth attention #46 closed)