FlashProve is a proof system for language-model inference. A provider commits to a model's weights once. For each reply it serves, it produces a proof that the reply's tokens satisfy an agreed rule, the acceptance policy, under the committed weights. A client checks the proof on a CPU, without the weights.
To build and run it, start with quickstart.md. To check a proof, read verifying.md. glossary.md defines the terms.
| Party | Program | Runs on | Directory |
|---|---|---|---|
| Provider | cp_commit commits to the weights, once |
one H100 | prover/ |
| Provider | cp_prove proves served replies, in batches |
one H100 | prover/ |
| Provider | a front end between clients and cp_prove. The example, proof_middleware.py, is middleware inside an unmodified vLLM |
the model server | integrations/ |
| Client | cp_verify checks one proof per call |
CPU | verifier/ |
| Client | check_proof.py runs cp_verify and compares the files and the result against the client's own values |
CPU | integrations/ |
| Client | lean-verify runs the executable form of verify, the Lean verifier that the soundness theorem sound_of_verify_at is about, on a real full-size proof |
CPU | tools/lean-verify/ |
cp_commit, cp_prove and cp_verify share one library, blocks/. cp_commit also runs
cp_verify on its own output. cp_prove does not.
Without an H100, you can check the example proofs in examples/ with cp_verify, and make and
check toy proofs with blocks/first_look.sh. Proving a model's reply needs
the GPU.
A proof is about the integer model: the network with quantized weights, computed in integers only.
A request file fixes the statement. The file holds a window of T token ids (the prompt followed by the reply) and two numbers, r and e. Positions are numbered from 0. At position t the model predicts token t + 1, so a proof over positions r to e − 1 checks tokens r + 1 to e, the checked tokens.
For example, with a prompt of 3 tokens and a reply of 2, r = 2 and e = 4, and the checked tokens are tokens 3 and 4, which are the reply. In general, a prompt of n tokens and a reply of m need r = n − 1 and e = n + m − 1.
If the prompt and reply are shorter than T, filler tokens pad the window, and the client's file must use the same filler as the front end's (how to build the file).
The programs support two acceptance policies, and the parameter file
(policy.txt) selects one.
- Exact match. Every checked token must be the committed integer model's top choice at its position.
- Clipped gap. Each checked token gets a cost: its score
gap plus a constant. At most
capof the costs may exceedC. Each cost is then clipped toC, and the clipped costs must average at mosttheta. So some of the checked tokens, up to a number thatcp_verifyprints, can be any token at all. Every other token costs at mostC, which rules out nothing whenCis large or the model's scores lie close together, and a verifier cannot tell when that is so (limits.md). This scoring follows Token-DiFR's at temperature 0 (arXiv 2511.20621), with four differences, which PUBLISHED-WORKS.md lists.
The commitment ID, a hash that identifies the commitment, covers the
parameter file but does not tell you which policy the file sets, so tell cp_verify which policy
you expect. Unless you pass -policy, it assumes exact match (when it rejects the
call).
A proof says nothing about tokens outside the range, with one exception. If the range runs to the end of the window (e = T), the last position predicts the token after the window, called the next token. The policy does not apply to it, so in this case the checked tokens end at token T − 1. Instead, under either policy, the proof states that the next token is the integer model's top choice.
verifying.md explains in full what an accepted proof means.
This is research code. These five cautions are the main ones; limits.md lists the known limits, and may be incomplete.
- Soundness. Nothing of soundness has been proved about the three programs themselves
(
cp_commit,cp_prove,cp_verify). The error bounds in bits that they print are the programs' own calculation, not a security level, and at the default setting some of the bits count proof of work (what the figures are worth). guide/what-is-proved.md says what is proved, and about what. - Weight hiding. By design, the programs mask proofs and a commitment's public files, apart from a declared list, and they do so by default. What is proved of it, and how far that reaches: guide/weight-hiding.md. What a provider must do: operators-duties.md.
- The committed model. The provider chooses the weights and writes the configuration, so no acceptance policy can show that the committed weights are the model the provider advertises, or even a sensible model.
- The integer model. A proof does not say how close the integer model is to the floating-point model.
- What has run. limits.md says which results come from these programs and which do not. The full-size runs used synthetic weights or a third party's 4-bit, 8-bit or 16-bit (BF16) conversion of Llama 4 Maverick's weights. On synthetic weights, and in the runs through vLLM, the requests were token IDs and not text. guide/performance.md gives what was measured.
Provider, once:
model files -> scales/, weight_sums/ -> quantization scales, checksum file
weights + configuration -> cp_commit -> public key directory, commitment ID (published)
-> secret key directory (kept secret)
For each reply:
client: prompt -> front end -> engine -> reply -> client
front end: request file (prompt and reply tokens, r, e) -> cp_prove
cp_prove, for each batch -> proof file -> front end
front end: cp_verify on every proof of the batch -> proof file -> client
Client, for each proof:
cp_verify on the inputs listed under "The files" -> YES or WITHIN LIMITS; any other result: not accepted
The steps below describe what the programs are designed to do.
- Prepare. The provider writes a base parameter file. The program in
weight_sums/writes a checksum file, and the provider addsfold.txt, a two-line file. Next, the tools inscales/, which read the checksum file, choose the model's quantization scales and write them intoKL.binand the final parameter file (policy.txt). Last,prover/tools/pack_layout.pywrites the layout file (layout.txt).policy.txt,KL.binandlayout.txtmake up the configuration.cp_commitandcp_proveneed the checksum file andfold.txtin the same directory as the model's weight files (weight_sums/README.md). Keep the checksum file as secret as the weights. - Commit.
cp_commitreads the weights and the configuration, encodes the weights as a weight table, and builds a Merkle tree over it. This first table is the anchor, and the commitment ID names it.cp_commitproduces a one-time proof that the anchor's entries are within their ranges, although for the embedding that range is wider than the model's own (limits-further.md).cp_committhen makes M leaf tables (-M) and ties each to the anchor with a proof called the link. Before it writes each public file, it hascp_verifycheck the file's bytes. - Publish. The provider publishes the configuration, the public key
directory (
ivk) and the commitment ID. If a provider could choose the commitment after seeing a prompt, it could commit to a small model that gives the answer it wants, and prove that honestly. So a client must have the ID before it sends its prompt, and must not get it through the server that it is checking, alone or with the files. Otherwise an accepted proof says nothing about which model the tokens were checked against. - Serve. The client sends a prompt, and an engine serves the reply. In ordinary serving, the expected mode, vLLM or any other engine serves the floating-point model. The proof is made afterwards, as one prefill pass over prompt and reply, under the clipped-gap policy (what a client learns under each policy). In integer serving, the engine runs a package designed to compute the integer model, so that a served reply can be proved under exact match.
- Prove. The front end writes a request file from the token ids that the engine reports that it
read and produced.
cp_provetakes a batch of 1 to 64 requests and picks one leaf table for it at random. If a batch runs to completion, each of its requests gets either a proof file and a token file, or an empty.failedfile. The token file holds the next token, or 0 if the range ends before the window does. - The provider's check. A response is a request's proof file and token file.
cp_proveruns no verifier on what it makes: once a batch's responses are staged, they get their final names. So the provider's front end runscp_verifyon every response of a batch before it gives any of them to a client. The serving scripts that ship (the vLLM integration,driver/measure_rate.py, the example scripts) hand on no proof thatcp_verifyhas not accepted, and the vLLM integration goes by the batch. This check is the provider testing itself, so clients must still verify. Nor is it a check on hiding: it catches only proofs thatcp_verifyrejects.cp_verifydoes not check that masks are present, and for most masks it cannot, so a proof whose mask was lost before the prover committed to the masked value still passes (limits.md). - Verify. The client builds its own request file from the token ids it sent and received, and
its own token file (the files). It then calls
cp_verifywith-idand, under clipped gap,-policy. The proof is accepted only if the exit code is 0 and the last line isYESorWITHIN LIMITS: ….cp_verifyprintsWITHIN LIMITSrather thanYESif any token of the request was checked under clipped gap. The last line does not show whether the result counts proof of work, or what the binding of the weights rests on: read the linesproof of work:andcommitment:too (when to accept). verifying.md has the commands, and its table lists the last lines and exit codes.
Why an anchor and leaf tables. The commitment ID is computed from the anchor, not from the leaf
tables, so it is the same for every leaf table of a commitment. Proofs are made against leaf tables
because each batch opens some columns of its table, and a table's masks cover only a fixed number of
opened columns, its limit (limits.md). As soon as
any leaf table runs out of room for another batch, cp_prove exits, and the service under that
commitment ends. So watch the counts, and make and publish a new
commitment before then. If anything has leaked by then, a new
commitment does not undo it.
One batch inside cp_prove. The top of prover/prover/service.cuh describes these steps.
- It takes the oldest requests from the spool directory and fixes their order.
- It samples 32 fresh random bytes. It uses these bytes, the seed and the requests to pick the batch's leaf table. It adds the batch's columns to that table's counter before it reads any of the table's files. Then it derives the key for the batch's randomness.
- It proves the requests,
-in-stepof them per pass over the weights. - It opens the columns of the leaf table that the batch needs, once for the whole batch. This takes two more passes over the weights.
- For each request, it stages either a proof file and a token file, or a
.failedfile. - All the batch's responses get their final names together.
In the secret key directory, cp_prove holds a lock while it runs and appends to the counter of the
batch's leaf table. It writes its results to the directory named with -out or -spool, and its
messages to stdout and stderr.
Why step 4 needs two passes. The first pass folds the table's rows at a point sampled only after
every request in the batch has finished its first chain. Next the prover
commits to the folded row, and only then samples the positions of the columns the second pass
computes. The order of the messages is at the top of blocks/iop/lin.h and blocks/iop/foldeq.h.
A front end must give each client only its own response. The example front end has no notion of clients. It hands a proof to anyone who has the proof ID and has the shared API key where vLLM was given one that the middleware can read (how it learns that key, and where it cannot: the operator's checklist; what was run: row H11.2 of the table). operators-duties.md covers the duties of a front end and an operator.
A client needs these inputs to run cp_verify.
cp_verify <configuration directory> <public key directory> <request file> <token file> <proof file> -id <commitment ID> [-policy <expected policy>]
| Input | Made by | The client gets it from | Trusted, or checked by the call |
|---|---|---|---|
commitment ID (-id) |
cp_commit |
the party that vouches for the model, not through the server you are checking; best where the commitment is published; before sending the prompt | trusted |
the expected policy (-policy): its kind and values |
the provider, which sets them in policy.txt |
the same source as the ID; before sending the prompt | trusted |
| tokenizer, chat template, end-token ids | the model's publisher | the place where the model is published; before sending the prompt | trusted; neither the ID nor a proof covers them |
the cp_verify program itself |
the client, who builds it | this repository | trusted; the ID does not cover any part of the program |
policy.txt, KL.bin, layout.txt: the configuration |
the provider | anyone | identified: the ID covers every byte. cp_verify tests it, but no test shows that the model it defines is the advertised one |
ivk/key.bin: the key file |
cp_commit |
anyone | checked by cp_verify: the ID covers the anchor's root and shape, its out-of-domain (OOD) answers (masked sums of the weights) and its OOD seeds, and the call checks the proofs in the file |
ivk/leaf/<k>.bin: the leaf file of the leaf table that the proof names |
cp_commit |
anyone | checked by cp_verify: the call checks its link to the anchor |
<name>.req: the request |
the client | its own record of the tokens sent and received | the client's own |
<name>.y: the token file |
the client (cp_prove writes its own with each proof) |
its own record: the next token it was served, or 0 if the range ends before the end of the window | the client's own |
<name>.proof: the proof file |
cp_prove |
the provider | checked |
The commitment ID is a single hash that covers every byte of the three configuration files, plus the
anchor's root and shape and its OOD answers and OOD seeds. The function
published_id in blocks/frame/heads.h computes the ID, and cp_commit and cp_verify both call
it. The ID does not cover the proofs inside key.bin, or the leaf files. cp_verify checks those
when it reads them (what is known about such files).
Never publish the secret key directory (isk), and never copy
or restore it. It holds the seed, the configuration as committed and, for each weight table, its
files and its counter. Anyone who reads the directory can remove the masks from opened columns and
test any guess at the weights, and that cannot be undone. A restored copy resumes from an old
counter, so more columns could be opened than the masks cover (backups and restored
copies).
The proof system is hash-based and non-interactive: an interactive oracle proof compiled with Merkle trees and Fiat-Shamir. This outline says what each part is designed to do. protocol.md describes the protocol, and guide/what-is-proved.md says what has been proved.
- Field and hash. Arithmetic is in the Goldilocks field, p = 2^64 − 2^32 + 1, and its cubic extension. A proof's Merkle trees and its chains, which are its Fiat-Shamir transcripts, use BLAKE2b. A second, fixed hash function, built on BLAKE2b's compression function, produces some digests that go into a proof's statement (hash mode).
- Weight commitment. Each row of a weight table holds weights together with random entries, called masks, and is encoded with a Reed-Solomon code. The table's columns are the leaves of a Merkle tree.
- A request's proof. A proof file holds three chains. The first belongs to the request. It works through the model segment by segment, reducing every step to claims about what the proof has committed to and about the weight table. The second is shared by the batch, and opens columns of the table once for all the batch's requests. The third again belongs to the request, and completes the request's part of that opening.
- Challenges. The verifier does not send any. Instead, each challenge is computed from a hash over its chain's first query and the earlier prover messages that enter the chain. The first query holds the digest of the configuration files, the plan of the chain's rounds, and the statement. Two things enter that hash late or never, and both are known limits. The next token does not enter the chain until just before the output head's part of the proof, so the statement in the first query omits the next token (SO-50). Opened columns, salts and Merkle paths never enter it (row SO-53 in the table of all rows).
| Directory | Contents |
|---|---|
blocks/ |
the library: field, hash, codes, the steps of the proof system, file formats, error accounting; toy proofs |
verifier/ |
cp_verify, its build script and its tests |
prover/ |
cp_commit and cp_prove: GPU sources (CUDA kernels and host code), build script, tests, and example configurations in prover/examples/ (its README maps its directories) |
scales/, weight_sums/ |
tools that choose a model's quantization scales and write the checksum file that cp_commit and cp_prove need |
driver/ |
the operator's side of the spool directory, driver/spool.py (not the same program as integrations/spool.py); driver/measure_rate.py, which measures the rate of proving (driver/README.md) |
tools/ |
lean-verify, which runs the Lean verifier's executable form on a full-size proof (a long build first: its README); the sandbox that the tests run in; an audit tool that re-derives a commitment's random values, or one batch's, from the secret key directory (it only works on synthetic weights) |
toy/ |
small test scripts, including quick_start_test.py |
tests/, run_suite.py, SUITE.md |
tests and shared test helpers, the script that runs them, and the table of tests |
examples/ |
four stored proofs on synthetic weights, two small and two at full size, each with what a client needs to check it; a script that runs cp_verify on altered files; stored toy files for blocks/; and quickstart-gpu.sh (examples/README.md lists them). This repository stores no false files for cp_verify made by cheating programs, so the trials with such files (row E28) cannot be repeated from it |
examples/maverick/ |
how to commit to public weights of Llama 4 Maverick, in a third party's 4-bit conversion: where to download them, their checksums, a script that goes from the download to a checked proof, and the commitment ID to compare the result with (its README); no weights |
docs/ |
the docs (where to read on) |
integrations/ |
the example front end for vLLM (proof_middleware.py) and its library spool.py, whose make-request command is for clients; the client's script, check_proof.py; an example client, ask_and_check.py; a demo, vllm/end_to_end/end_to_end.py; the package designed to let vLLM run the integer model, vllm/integer_model/; and the limits of all of these, LIMITS.md |
lean/ |
the Lean library (what it proves; its soundness theorem on verify: lean/THEOREM.md), and in lean/OnBytes/ Lean text that follows cp_verify's source (see below the table) |
licenses/ |
third-party licence texts |
paper/ |
paper.pdf is the paper: the protocol, what is proved of it in Lean and what is not, and the measurements |
Model weights are not part of the repository. Nor is any key file, leaf file or proof that was made from a real model's weights. THIRD_PARTY.md lists the values here that depend on a real model, such as checksums of its files.
lean/OnBytes/. This folder holds Lean text that writes cp_verify's
checks again, following cp_verify's source (what it holds; its
limits, and where it is known to differ from cp_verify). That second
writing is unproved, and this repository builds no stand-alone program from it; one test
(tools/accepts/) compiles it and has Lean load the compiled code while it evaluates files of
lean/Ask/. Other Lean files, those of verify's executable form among them, import definitions from
this folder; what verify takes from it is read with verify's statement
(lean/GAPS.md, 58). No theorem connects the second writing to cp_verify, and
the soundness theorem for the Lean verifier is not about it. These docs call the second writing
verify-on-bytes; where they tell a run of a stand-alone program built from it, that program was a
yardstick for tests: no soundness theorem reaches a call of it that checks a real proof, and no
theorem bounds the chance that it accepts a false file. Because it follows cp_verify's source
text, the two are not independent, one witness and its port, not two witnesses: where they agreed,
that is not independent evidence, and a mistake that both share does not show.
- File format: format/README.md points to the specification of the bytes and
the checks. Section 17 of
FORMAT.mddescribes what it leaves undefined and where it is known to differ from the programs. For bytes, hashed strings, ordering and checks, the specification takes precedence over every other page. If a program disagrees with it, please report a bug. - The
blocks/library: blocks/README.md.