Circle STARK proofs of LLM inference and exact classifier decisions.
What is proved · How it runs · Performance · Quickstart · Proof downloads · Documentation site
Hallmark proves two kinds of computation and checks them against one request:
- An LLM answer. A pinned, quantized TinyLlama-1.1B computes “The capital of France is Paris.” token by token, through end-of-sequence. SASH's Circle STARK prover covers every layer; the result is 23 linked proofs.
- Classifier decisions. A byte rule, an MLP and a CNN reach exact pass/block verdicts on the same prompt. Dedicated STWO Circle STARK circuits prove each one.
The verifier checks every proof against model and program identities the caller pinned before proving, then checks that both proofs refer to the same request. Circle STARK is the only proof system.
This is a fixed-case research integrity MVP: one prompt, one model, public witnesses, no hardware attestation. Where the proof ends lists what it does not claim.
| TinyLlama answer · run L1 | Classifier decisions · run C1 | |
|---|---|---|
| What is proved | 8 tokens through EOS, all 22 layers + head | 3 requests × 3 classifiers |
| Proofs | 23 linked segments | 9 independent proofs |
| Proving | 53.94 s GPU session on one H200 (547.75 s on CPU) | 24.06 s final STWO calls on an M4 Max |
| Verification | 3.40–3.53 s per check on a Mac, roots compiled in | 0.807 s for all nine |
| Proof size | 737.86 MB of JSON | 18.06 MB of JSON |
Each number has a stated boundary and none of them is an end-to-end latency. Read Performance before quoting one. The actual proofs, logs and machine records are public: download and verify them · artifacts · reference Discussion.
Prompt: Name the capital of France in one short sentence.
Response: The capital of France is Paris.
The chat template turns the prompt into 25 tokens. One causal block of 32 positions holds those 25 plus the first seven answer tokens. The proof covers the embedding, all 22 transformer layers and the final norm plus output head, as 23 segment proofs joined by 22 residual links. The head's argmax at positions 24–31 yields the eight answer tokens. Seven of them must equal the next input token, which closes the autoregressive loop; the caller checks that the eighth is EOS.
The relation is the disclosed quantization profile y-attnout-w12, not the
original floating-point model. The 23 segments form a linked bundle checked
together, not a single recursive proof. Exact LLM statement.
For a fixed integer classifier F, threshold t and the complete prompt x
(at most 96 UTF-8 bytes), the proof shows that verdict = 1 exactly when
F(x) < t, and that x matches a salted SHA-256 commitment. The weights are
constants compiled into the circuit; training is not proved. Each proof carries
100 public fields. The circuit recomputes the input digest; the caller rebuilds
the context digest and classifier identity from its own request.
A valid proof can say “block”. Proof validity and classifier approval are separate outcomes. The frozen MLP allows every admitted input (its integer score bound is [−5600, −2338] against threshold 0); it exercises the full MLP arithmetic while the rule and CNN provide the blocks. Model limits.
| Path | Cryptographic statement | Caller responsibilities |
|---|---|---|
| LLM — SASH Circle STARK | The enrolled quantized model computes the complete greedy answer through EOS: embedding, every transformer layer, final normalization and output argmax, with all segment and token links checked | Enroll model, shape and calibration roots and the exact token case before proving; check tokenization and detokenization outside the circuit |
| Classifiers — STWO Circle STARK | The rule, MLP or CNN arithmetic on the complete 96-byte input, plus its exact salted SHA-256 commitment | Enroll the circuit, preprocessing and backend identities before proving; rebuild expected public inputs and the complete roster (details) |
| Composition — host verification | Both verified relations refer to the same opened request | Check the shared prompt/response context; combine verified verdicts with all-pass |
The classifier caller currently reruns native inference to derive the expected verdicts, so this demo does not show verification being cheaper than a rerun. LLM statement · classifier public inputs · composition.
Your Mac downloads the pinned checkpoint, quantizes it, decodes greedily and extracts the witness. Three sequential CPU jobs then build the unchanged SASH prover, derive the model roots and compare them with the caller's enrollment before proving, and verify the bundle in fresh processes. The verifier never receives weights or the witness. SASH is fetched as hash-pinned upstream source and driven through CLI adapters; the CPU prover is unchanged upstream Rust. Source ownership and patches.
The join is host code over two verified results, not recursive aggregation. Classifiers read the prompt; the response is bound into their context, not classified. Architecture and every command.
Three named runs, three machines, three timing boundaries. L1 is one H200 CPU-then-GPU pair for the LLM; C1 is one fresh classifier run on a Mac; J1 is the composition. They are component measurements, not one fresh end-to-end latency, and a single run is not a distribution. MB and GB are decimal; GiB is binary. Every stage and raw file · how to read these numbers.
FlashProve, a second LLM prover for Mixture-of-Experts models, has its own H200 run on a synthetic model. It is set beside L1 in SASH and FlashProve compared; no row there is a like-for-like speed claim.
Same host, inputs and parameters, CPU then GPU, 2026-10-26: HF Jobs H200 profile with 23 provider vCPUs, 256 GB RAM and four Rayon threads. All 23 proofs and 22 links verify, and all 69 proof, statement and claim files match byte for byte across backends. Preparation and the generated-kernel build are reused.
| All 23 proof segments | CPU / SIMD | GPU / CUDA |
|---|---|---|
| Trace-generation timers | 31.84 s | 32.14 s |
| Proving-session timers | 547.75 s | 53.94 s |
| ↳ Interaction work, inside session timer | 60.72 s | 36.95 s |
| Inline verification timers | 77.62 s | 34.40 s |
| Complete prover segment commands | 679.49 s | 155.89 s |
| Separate full-grid CPU verifier on H200 host | 85.35 s | 84.32 s |
| Actual proof JSON, all 23 files | 737,858,641 bytes | 737,858,641 bytes |
| Sampled peak host RSS | 103.75 GB | 8.07 GB |
| Sampled peak GPU memory | — | 67.67 GiB |
The session timer starts after trace generation and includes commitments and interaction work, so do not add the nested interaction row. Complete commands also include loading, inline checks and serialization; the timer breakdown shows how they stack. Constraints run on CUDA with zero CPU fallback; witness and interaction generation stay on the CPU.
| Verification and enclosing stages | L1 | Boundary |
|---|---|---|
| Native Mac full-bundle check | 3.40 / 3.53 s | Per honest check on an M4 Max after compilation, shape roots compiled in |
| Fresh Linux full-bundle check | 77.40 / 77.61 s | Per honest check on a separate 8-vCPU / 32-GB worker, roots derived at runtime |
| Generated-kernel build job | 1,815.94 s | Earlier CPU job's running time; 77 kernel shapes |
| Model-root command | 113.86 s | Separate command before the segment runs |
| Complete cached-build benchmark | 2,086.78 s | Both backends, fixtures, transfers, checks and Mac suites |
The two verifiers share SASH's cryptography but preprocess differently, so their times are not a matched speedup. Linux rejects all 14 mutations; the Mac verifier rejects all 36. Download the LLM proofs (355 MB gzip of the 737.86 MB payload) · GPU method.
One fresh default run on 2026-10-28: 16 logical CPUs, 48 GiB RAM, four prover threads, populated dependency caches and an empty Rust target directory. It trains the MLP and CNN and proves all three classifiers on all three requests.
| Stage | C1 measurement | Boundary |
|---|---|---|
| MLP training/export | 1.95 s | Complete child command |
| CNN training/export/screen | 172.94 s | Complete child command |
| Rust prover / proof-only verifier builds | 24.82 / 9.57 s | Two locked Cargo commands |
| Host witness construction | 15.613 s | Nine witnesses, outside the prover processes |
| Preprocessing and model commitment | 7.882 s | Inside prover commands; repeated per proof |
| Final STWO proving calls | 24.061 s | Nine final calls; excludes the trace and commitment work before them |
| Complete prover commands | 50.028 s | Parsing, every engine stage and serialization |
| Fresh proof-only verifier commands | 0.807 s | Nine fresh processes; excludes the caller's native rerun |
| Actual serialized proofs | 18,063,779 bytes | Nine canonical JSON proof files |
| Proof campaign | 93.50 s | Enrollment, witnesses, proving, checks and host work |
| Complete observed controller | 302.96 s | Training, builds, proofs and checks; excludes uv bootstrap |
| Maximum sampled worker-tree RSS | 23.45 GB | Maximum across workers, not a sum |
Rows nest: preprocessing and final proving sit inside the prover commands. All 63 changed public inputs, 18 cross-request substitutions and three invalid rosters reject. Every C1 stage and each proof's time and size · download proofs and logs.
Three CPU/CUDA pairs of the unchanged C1 circuits; all 54 proofs pass the stock verifier and match the C1 proof bytes. Training, witness preparation and verification are excluded. Method, all 54 checks and resource records.
The retained-input composed command takes 266.29 s: native LLM workflow
216.87 s (fresh build, two honest checks, 36 mutations) and classifier controller
44.12 s (three new proofs, 6,020,695 bytes). The join re-verifies the classifier
proofs and rejects 16 changed bindings and rosters. Integrity passes; verdicts
[1, 1, 0] make release false. It reuses the L1 LLM proof bytes and is not
fresh GPU proving. Exact join and timings.
git clone https://github.com/AbdelStark/hallmark.git
cd hallmarkPython packages are not published; work from the checkout.
| I want to… | Command | Needs | Cost |
|---|---|---|---|
| Train, prove and verify classifiers | make e2e |
Apple Silicon Mac, 12 GiB free | Local only |
| Prove the TinyLlama answer | make e2e-llm |
Mac with 32 GB RAM, 16 GiB free, HF token and credits | ≤ $2.50 of HF CPU jobs |
| Verify retained LLM proofs | make verify-llm LLM_RUN=… |
Mac, Rust toolchain, 8 GiB free | Local, offline |
| Reproduce the H200 measurement | make benchmark-llm-gpu … |
A completed LLM run, HF credits | ≤ $5 of HF CPU/H200 jobs |
| Verify stored FlashProve proofs | make verify-flashprove |
Mac or Linux, C++17 compiler, 5 GB RAM | Local, no GPU |
| Prove FlashProve's small synthetic kit | make e2e-flashprove … |
HF token and credits | ≤ $3.50 of HF CPU/H200 jobs |
| Bind both to one request | make e2e-composed |
Both sets of prerequisites, 32 GiB free | Paid LLM jobs unless you reuse runs |
| Check software contracts only | make check-conformance |
Linux or macOS | Local, no proof |
All proof workflows need Git, Make, uv 0.9.18 and Rust
nightly-2025-07-14. Each run prints a fresh runs/ directory; failures exit
nonzero and keep their logs.
make e2eTrains the MLP and byte CNN, exports the integer graphs, builds the locked STWO prover and proof-only verifier, and generates nine Circle STARK proofs. Each is freshly verified, and 63 public-input mutations, 18 cross-request substitutions and three invalid rosters must reject. Expect the benign request to pass, the CNN to block the contextual one and the rule to block the marker one. Allow a few minutes plus the first Rust build, and room for the 30-GiB sampled worker cap. No GPU, token or pretrained weights are needed. Outputs and troubleshooting.
Set HF_TOKEN in the environment or an ignored .env, check your credits, then:
make e2e-llmDownloads the pinned 2.2-GB Apache-2.0 checkpoint, prepares the witness locally and starts three sequential paid HF CPU jobs under a $2.50 startup-inclusive ceiling. Success means 23 new proofs, two honest verifications and 14 rejected mutations. No local GPU is needed. Run guide.
To re-verify those proofs on your Mac, with no weights or credentials:
rustup toolchain install nightly-2025-07-14 --profile minimal # once
make verify-llm LLM_RUN=runs/llm-e2e-...Builds may download locked dependencies; verification itself runs offline. Read the shape-enrollment trust boundary, including optional signed parameters.
make benchmark-llm-gpu \
LLM_RUN=/path/to/completed/llm-e2e-run \
GPU_CREDIT_BALANCE_USD=YOUR_CHECKED_BALANCEStarts paid HF CPU/H200 jobs under a $5 single-run ceiling. It reuses the frozen public witness, builds generated CUDA kernels, compares SIMD and CUDA on one H200, and verifies the GPU proofs in fresh CPU and Mac processes.
GPU workloads can also run on generic Slurm clusters with Apptainer:
cd slurm
./hallmark-slurm.sh build-image --profile cuda12.8 --output hallmark-cuda12.8.sif
./hallmark-slurm.sh submit-llm --partition gpu --container hallmark-cuda12.8.sif \
--llm-run /data/$USER/llm-e2e-xyz --credit-balance 10.00Two CUDA profiles are available: cuda12.9 (CUDA 12.9.1, existing HF baseline,
driver 535+) and cuda12.8 (CUDA 12.8.1, native support for driver 570.x,
recommended for TDX confidential VMs). The cuda12.9 profile can run on driver 570.x
via CUDA forward compatibility but requires cuda-compat setup. Optional attestation
hooks are documented but off by default. Slurm guide.
GPU_BUILD_RUN=… reuses a matching build. Run plan, costs and failures.
make e2e-composedRuns both workflows and binds three new classifier proofs to the proved LLM
request. Pass LLM_RUN=… CLASSIFIER_RUN=… to reuse your own runs without new
cloud jobs; LLM proofs are re-verified and the three classifier proofs are
always new. Walkthrough and signed enrollment.
uv sync --locked --python 3.12
uv run --locked hallmark validate-data fixtures/smoke.jsonl
make check-conformanceExpect a dataset summary, then 156 matched software cases and 32 unavailable full-evidence obligations. Core CI also covers CPython 3.13 and 3.14.
make demo-tui
make demo-webA terminal and a local web workbench over the same workflow controller: live stages, process memory and CPU, logs, HF job status, verification results and retained artifacts. Paid workflows show their cost bounds before launch. Workbench guide.
The exact seed-17 classifiers used by the proofs are on Hugging Face, with model cards, JSON checkpoints, quantized weights, native exports, evaluation records and hash manifests:
| Model | Architecture | Card and weights |
|---|---|---|
| Byte MLP | 1,033 parameters; byte counts → 4 ReLU units → score | abdelstark/hallmark-byte-mlp |
| Contextual byte CNN | 4,161 parameters; byte embeddings → causal convolution → masked max → score | abdelstark/hallmark-contextual-byte-cnn |
Both are MIT-licensed synthetic research models trained from scratch, exported as Hallmark JSON/CBOR rather than Transformers models. Native inference does not generate a proof. TinyLlama is an upstream pretrained model, not a Hallmark release. Download and run the models · immutable revisions.
- One fixed case. The pipeline proves this TinyLlama answer to this prompt. Another prompt, length or model needs a new case and a new enrollment.
- Integrity, not privacy. Witnesses are public: there is no zero-knowledge or hardware-provenance claim. Caller enrollment is the trust anchor; a prover cannot authenticate its own parameters.
- Synthetic classifiers. Inputs are capped at 96 UTF-8 bytes and longer ones reject. Synthetic accuracy is not a useful safety policy: the frozen CNN blocks the ordinary LLM prompt in the composed demo.
- Unreviewed cryptography. Circle proofs use caller-enrolled public
preprocessing. Neither prover has independent security qualification, and
results say so:
qualified_backend: false,hiding_claim: false,hardware_claim: "not-accepted". - Bounded measurements. Proof sizes exclude enrollment and keys. GPU timings are single matched observations. Failures and source and environment hashes stay in the research record.
The MVP acceptance matrix separates completed integrity work from the original private/hardware v1 roadmap. Decision 0085 made Circle the default: lower workflow time in the recorded harness, at the cost of larger proofs, more memory and different security assumptions.
| I want to… | Read |
|---|---|
| Understand the proofs from scratch | Learning path |
| Read the measurements correctly | Performance |
| Download actual proofs and audit every stage | Reference runs |
| Trace commands, flows and source ownership | Architecture |
| Inspect the exact LLM relation | LLM proof guide |
| Inspect classifier circuits and public inputs | Classifier proofs · Circle details |
| Reproduce GPU timings | LLM on H200 · classifiers on A100 |
| Compare the two LLM provers | FlashProve · SASH and FlashProve compared |
| Run the joined workflow | Composed demo |
| Assess model and data quality | Results and limitations |
| Find any contract or research record | Documentation index |
| See completed work and open gates | Current state |
Build the documentation site with
make docs or preview it with make docs-serve; its toolchain is locked
separately (site maintenance). Figures are generated
from the evidence records with python scripts/render_figures.py.
uv sync --locked --python 3.12
make check
make package-check
# For training changes:
make check-train
make package-check-trainmake check covers the lockfile, lint, formatting, strict typing, tests,
documentation and software conformance. A core environment may skip optional
training; the train gate requires it. Package checks build and install fresh
wheel and sdist artifacts. CI checks a Circle gate proof in a separate verifier
process; it does not run paid GPU jobs or the full Mac demos.
| Path | Responsibility |
|---|---|
src/hallmark/ |
Contracts, native integer runtimes, training and binding |
scripts/ |
Experiment controllers, reproducible workflows, validation and figures |
research/probes/sash/ |
Pinned SASH adapters, patches and the fixed LLM case |
research/probes/circle/ |
STWO gate AIR, prover and proof-only verifier |
fixtures/ · tests/ |
Synthetic data, frozen protocols, arithmetic and rejection checks |
docs/ · research/ |
Guides, decisions, numeric evidence and the research ledger |
Start with CONTRIBUTING. Arithmetic or binding changes need rejection tests; data or evaluation changes need a new experiment identity. The supported API separates implemented interfaces from proposed ones. Keep raw evidence and failures; never commit credentials, private transcripts, checkpoints or local runs.
Read SECURITY for trust assumptions and private vulnerability reporting. Open an issue for reproducibility problems or scoped improvements, with the command, source revision and a sanitized failure log.
Hallmark is MIT licensed. Third-party software, data and model weights keep their own terms; see dependency notices. Historical evidence keeps its original licence labels and hashes (decision 0063).