Skip to content

Latest commit

 

History

History
321 lines (264 loc) · 95.6 KB

SUITE.md

File metadata and controls

321 lines (264 loc) · 95.6 KB

The test suite

This page lists the tests of this repository that anyone can run. Each test is one row of a table: what it needs, about how long it takes, the command that runs it and what its output must show. The page is also the input to run_suite.py, which reads only the tables.

No test uses a real model's weights; the tests that need weights use synthetic ones. A test that passes shows that the programs or routines it names gave the expected answers on synthetic inputs, or that a file or program that should be refused was refused. It shows nothing more about soundness or hiding.

The programs and every test program are built for one prime field, p = 2^64 − 2^32 + 1; a command may say --field 2, which names that field and changes nothing. Words such as key file, leaf file and commitment ID are defined in the glossary. The programs, their files and some tests' names say "record" for a table's OOD answers: see the glossary's entry "Record". Some test sources are run by no row; the head of nearly every one says how to build and run it. prover/tests/tail_gam_test.cu builds from prover/ with OUT=<a folder of your own> ./build.sh tail_gam_test. Add -fno-ipa-modref to a head's g++ line that lacks it (verifier/README.md says why). Never leave such a build's output in verifier/src: without -o, g++ writes a.out there, and the verifier's source digest, which covers every file in that folder, changes; a later run of the row build.serving then counts a cp_verify built before the change as built from other sources, and refuses it.

Run the tests

D=$(mktemp -d) && ./run_suite.py cpu "$D/work"    # the tests that need no GPU; the runner makes $D/work itself
D=$(mktemp -d) && ./run_suite.py gpu "$D/work"    # needs nvcc and one H100 80 GB
./run_suite.py all list-only --list    # prints the tests and runs nothing; no folder is made

In zsh, type the commands above without the words from # on, or run setopt interactivecomments first: by default, at its prompt zsh does not read # as the start of a comment, and takes those words as part of the command line.

If you run the tests as root, for example under sudo, first read the paragraphs below that begin «The sandbox is there» and «Under sudo»: they say what a test can then still do beyond your own files, and what changes for the files that the run makes.

  • cpu runs the tests of the kind cpu, except any test that needs a row of the kind nvcc, gpu or gpu-by-run: the runner names each such test in a line that begins left out; all runs them.
  • gpu runs the tests whose kind is nvcc, gpu or gpu-by-run, and the rows that they need.
  • long alone runs the tests of the kind long: integer_model.cpu and tools.lean_verify.full_size.
  • --only <regular expression> runs, among the tests of the kind asked for, those whose names the expression matches anywhere, together with the rows that they need.
  • --jobs <n> sets the number of job slots: each test takes the number in its «slots» column, and one slot stands for one core and about 2 GiB of memory. The runner uses that figure only to choose how many tests run at once; it does not limit a test's memory.

./run_suite.py --help-more gives every option and rule.

What the tests need: Linux with Landlock in its kernel, flock, make, g++ 13 with its address and undefined-behaviour sanitizer libraries (libasan, libubsan), Python 3 and realpath; for the GPU rows also nvcc 13.2.86 (a later toolkit is not supported) and one H100 80 GB. Some rows need more, and count as not run, or pass in part, without it: pytest, numpy, gguf 0.19.0, clang++, util-linux's prlimit, and Lean with the library built (tools/lean-verify/README.md, "What to type"; lean/CHECKING.md, «Before you start»). scales/README.md, «Try it», gives the lines that make a Python environment with numpy and gguf, and integrations/README.md, «Tests», adds pytest; run the suite from the shell where that environment is active. On a system with nothing beyond what the install line in docs/quickstart.md installs, a run of mode cpu ends at best with status 77, not 0. Some tests use the CPU's SHA instructions (sha256rnds2): on a CPU that cannot run them, such a test ends with status 77 and a line that says so, and counts as not run; weight_sums fails there. tools.confine fails where /dev/shm exists but your account cannot make a file in it. Under the runner each row has a home folder of its own, so for tools.lean_verify.full_size, where lake is elan's, elan downloads the Lean toolchain again (some 540 MB, 2.8 GB on disk) or, without the network, the row fails; confined, a write of Lake's below lean/.lake also fails the row, and so does a run by another account than the one that built the library.

The work folder must not exist yet, must lie outside this folder, and its path may hold only letters, digits and / . _ -, with no part that begins with -; the folder above it must exist. If another account might be able to change a folder on the way to it, or to BUILD, nothing runs and the runner ends with status 2, because that account could put another folder in its place while tests run. A work folder below a fresh mktemp -d folder, in your home folder or directly in /tmp passes. If the message names a folder of yours, chmod go-w closes it. Where new folders come out group-writable (a default umask of 0002, as on a bare Ubuntu 24.04), run umask 022 before you make the work folder, or chmod go-w those folders on its path that you have just made yourself. On a machine where you trust every account, such as a container of your own, --i-trust-every-account-on-this-machine lets the run go ahead past a folder that other accounts could change, but never past one that is missing or cannot be examined, and the first and last lines of the run say so. Even with that option, many tests fail on such a folder, because the scripts they start make the same check. Each test runs in a new folder inside the work folder. The programs and a few tests build into bin/ and the GPU programs' build folder, <work folder>/build or the folder that BUILD names. The row build.verifier builds the verifier into the repository's own bin/, so a cp_verify that you built there yourself is replaced by one built from this folder's sources, if that build succeeds.

By default each test runs confined by tools/confine.py, which uses the Linux kernel's Landlock: it may write, make or remove files only in its own folder, the run's shared folder, bin/, and, for rows that build, the GPU programs' build folder, and besides them files in /dev/shm. A test of the kind gpu or gpu-by-run may also write the GPU's device files (/dev/nvidia*) and the files of /proc. The sandbox never confines a file's mode, owner or times: a test may change them on any file of your account. On Linux 6.12 or later a test may signal only its own processes; before Linux 6.12 only a signal to every process of the account is refused, and under root that leaves every other process of the machine in reach (a test that ends processes by name would end every matching process on the machine). The runner's first and last lines say which of the two holds. A row's command that you type yourself, without the runner, runs outside the sandbox. On a kernel without Landlock, or before Linux 6.12 on a machine that is not x86_64 or aarch64, nothing runs unless you give --unconfined, and then none of this is enforced. What the sandbox does not cover is listed at the head of tools/confine.py.

The sandbox is there to limit what a faulty test can damage by accident. It is not a barrier against a test written to do harm, and when the suite runs as root it cannot be one. Run as root, a test can still change the owner, mode or times of any file on the machine (chmod, chown, touch), which the sandbox does not limit at all, and it keeps root's other powers, such as the network and the machine itself. Under root each test gets a mount namespace of its own, in which /proc/sys, /proc/sysrq-trigger, /proc/irq, /proc/bus and /proc/fs are read-only against a plain open by mistake, not against a test of the kind gpu or gpu-by-run that means to write there (a root command with such a test's rights has been seen to make them writable again: the head of tools/confine.py), and /dev/shm is an empty folder of the test's own (seen at work as root on one kernel, Linux 6.18; no test on a GPU was run as root: docs/limits-all-rows.tsv, row R1.1). Where root may not make such a namespace at all (root without CAP_SYS_ADMIN, as in Docker's and Kubernetes' containers by default), the runner's first line says so: /dev/shm is shared, and a test can remove, overwrite or empty any account's files there; and where those places of /proc are writable, a run that includes a test on the GPU stops before it starts anything, unless you give --unconfined. Where the namespace can be made but a later step fails, nothing runs as root; --unconfined runs the tests without the sandbox, on purpose.

Under sudo the runner runs as root, so what it makes belongs to root: the work folder with everything in it and, if this folder had no bin/ before, bin/ with the programs built there. Your own account cannot remove that work folder, or write in a bin/ that root made: in a later run without sudo, the tests that build into bin/ fail, and every test that needs one of them is not run. That this bin/ is root's does not protect it: whoever owns the copy can replace it, cp_verify included. Under sudo these tests act as root on names inside the copy, so whoever owns the copy can steer what root does there, as with any program run by sudo from someone's copy. To go on without root, start again in a new copy of the repository.

None of the tests' calls removes, moves, overwrites, empties, or changes the mode or owner of a file outside the work folder, the repository's bin/ and, for rows that build, the build folder that the runner names in its first line, except files and folders of their own in /dev/shm, with names that begin confine-test-, run-suite-test- or .check-, which tools.confine, sources.run_suite and, through integrations/proof_middleware.py, integrations.unit and integrations.cp_verify make and remove. This rests on a reading of the calls written in the repository's files that the tests run; tests/dangerous_calls_test.py checks the recursive deletions and the signals among them against tests/dangerous_calls.tsv, but not single removals, renames or emptying. It does not cover appends, changes of a file's times, or what outside programs that a test starts (such as nvcc, g++ or pip) do.

A test that runs past its time limit (five times its «minutes», and at least ten minutes, or at least an hour for a test on the GPU when GPU_PREFIX or GPU_LOCK is set) gets SIGTERM for every process still in its process group, and SIGKILL for any that are left twenty seconds later. A test of the kind gpu or gpu-by-run gets no signal at all, because killing a program while its GPU code runs can take the whole machine out of use: the runner names it in capitals, leaves it running, counts the test as failed, starts no further test on the GPU and finishes the others. Only a test of the kind gpu that is still waiting for the card's lock is ended as other tests are.

Each table has eight columns. «kind» is cpu (no GPU), nvcc (needs the CUDA compiler but no GPU), gpu, gpu-by-run (a test that holds the GPU only around each GPU program it starts), or long. «needs» lists the rows that must pass first. «word» is any on every row of this page. «slots» is how many job slots the row takes. «minutes» is a rough estimate of how long the row takes on one machine; it sets the order and the time limit. «passes» lists the exit statuses that count as a pass; after a space may follow a regular expression that the last line of output must match, and after each ;; another one that some line of the output must match. «command» is run by bash from this folder, with T set to this folder, W to the test's own folder (not made yet), S to the run's shared folder, B to the GPU programs' build folder and WORD to the version word.

The runner's own exit status:

  • 3: a test ended with status 3, whatever else happened.
  • 1: otherwise, a test failed, a test was not run for a reason of the run's own (a test that it needs failed or was not run for such a reason, a GPU program that an earlier test had left running still held the card, or the work folder was replaced or removed before it could start), a line was printed after the word LOOK, the work folder was replaced or removed during the run, or the runner itself got SIGTERM, SIGINT, SIGQUIT or SIGHUP.
  • 77: otherwise, the only tests that did not pass were not run and said why in their last lines, together with tests that could not run only because of those.
  • 0: otherwise.
  • 2: nothing ran: the call is wrong, no test is to run, the work folder or BUILD lies below a folder that another account might change, or the tests are to be confined and the work folder lies on /dev/shm's file system.

--list ends with 0. A test that would otherwise pass but whose last line begins with not run could not run on this machine, and counts as not run. A test that ends with status 77 where its row allows it, and whose last line does not begin with not run, ran only in part: it does not stop the status from being 0, and the runner's last line counts such tests, so read it.

Some test programs run through prover/tests/unit.sh, which writes the program's own lines to <program>.log in the folder that the row's command names. For those rows the runner sees only unit.sh's status and its few lines: a program's status 3 reaches the runner as 1, so read that log when such a row fails.

Some tests run a prover that argues for something false, made so by the test's own code, or inject faults into test programs, all on synthetic data, so that you can watch a check refuse the false proof or claim. Use nothing from the work folder for anything but these tests. The prover's sources also call test hooks, each inside three lines under TEST_HOOKS; the hooks' code is not part of this repository, so no build with that switch can be made from it.

What a test's exit status means

  • 0: everything ended as the test expected.
  • 1: something did not.
  • 2: a wrong call.
  • 3: cp_verify accepted a file or a statement that it should have refused, and the test did not expect that; or, in verifier.ws_build_fuzz, the verifier's layout routine gave a layout outside its bounds; or another program in that test ended with status 3, so read the test's output to tell which. Keep as they are the work folder, bin/ and the folder that BUILD named, if it was set; and report it: SECURITY.md in the top folder says how.
  • 77: the test, or a part of it, cannot be run on this machine.

Most tests that end with status 3 also print a line with the words WAS NOT REFUSED and keep what was accepted. That every test does so has not been checked. These do otherwise. verifier/tests/served_form_test.py ends with 3 without the words; it and what_a_yes_covers_test.py say only that a call ended "not as expected", so such a row is reported as ACCEPT without a line after LOOK. tests/new/run_new.py and examples/rejected-proofs/try_altered_copies.py have words of their own (ACCEPTED; WAS ACCEPTED). verifier/tests/clang_build_test.sh counts the status 3 of made_up_files_test.py as a failure of its own: it shows the line with the words and ends with status 1. Other tests tell an accept only as a failed expectation in words of their own, among them blocks/first_look.sh. Tests of many cases remove the files of the cases that were refused; the accepted ones stay. Not known: whether each test compares the last line on an expected accept; the tests have not been read one by one for that.

run_suite.py prints the lines that its own list of wordings matches after the word LOOK, whatever the row's status. The list is not complete, so a row that failed without a line after LOOK can still be such a find: read its output, which the runner keeps in screens/<test>.txt in the work folder (for a test that runs through prover/tests/unit.sh, also the program's own log). Some of these lines are also printed where a test failed for another reason. The runner searches only what a row's command prints: prover/tests/unit.sh and prover/tests/psc_rows.sh print only lines of their own, so a find in a test that runs through one of the two shows as a plain failure.

Builds

These rows build the programs and test programs that the other rows run, and prepare shared test data; a row named build.… passes when the build ends without an error, and most test nothing by themselves.

name kind needs word slots minutes passes command
build.attention_sumchecks_test nvcc - any 1 4 0 mkdir -p ${B:?} && cd prover && OUT=${B:?} ./build.sh --field 2 attention_sumchecks_test
build.base_kernels_test nvcc - any 1 10 0 : built$ prover/tests/unit.sh --field 2 build tests/base_kernels_test.cu ${S:?}/units.field2
build.blake2b_f_test nvcc - any 1 0.1 0 : built$ prover/tests/unit.sh --field 2 build tests/blake2b_f_test.cu ${S:?}
build.coins_field_rule_test nvcc - any 1 0.3 0 : built$ prover/tests/unit.sh --field 2 build tests/coins_field_rule_test.cu ${S:?}/coins_field_rule_f2
build.coins_own_gpu_test nvcc - any 1 1 0 : built$ prover/tests/unit.sh --field 2 build tests/coins_own_gpu_test.cu ${S:?}/coins_own_f2
build.compiler_sample nvcc - any 1 0.1 0 mkdir -p ${B:?} && cd prover && OUT=${B:?} ./build.sh --field 2 compiler_sample
build.cp_commit nvcc build.verifier any 2 10 0 mkdir -p ${B:?} && cd prover && OUT=${B:?} ./build.sh --field 2 cp_commit
build.cp_prove nvcc - any 2 10 0 mkdir -p ${B:?} && cd prover && OUT=${B:?} ./build.sh --field 2 cp_prove
build.engine_gpu_test nvcc - any 1 4 0 ^engine_gpu_test: built$ prover/tests/unit.sh --field 2 build engine/tests/engine_gpu_test.cu ${S:?}/engine.f2
build.engine_twins nvcc - any 2 4 0 ^engine_twins: field 2: built$ prover/tests/engine_twins.sh build 2 ${S:?}/engine_twins.f2
build.forest_lookup_test nvcc - any 1 5 0 mkdir -p ${S:?}/npsc.f2 && cd prover && OUT=${S:?}/npsc.f2 ./build.sh --field 2 forest_lookup_test
build.frame_as_the_verifier_reads nvcc - any 2 7.3 0 mkdir -p ${B:?} && cd prover && OUT=${B:?} ./build.sh --field 2 frame_as_the_verifier_reads
build.hf_code_test nvcc - any 1 1.5 0 prover/tests/unit.sh --field 2 build hash_commitment/tests/hf_code_test.cu ${S:?}/hf_code_test.field2
build.hf_commit_table_test nvcc - any 1 1 0 prover/tests/unit.sh --field 2 build hash_commitment/tests/hf_commit_table_test.cu ${S:?}/hf_commit_table_test.field2
build.hf_lin_side_test nvcc - any 1 3.5 0 mkdir -p ${S:?}/f2 && cd prover && OUT=${S:?}/f2 ./build.sh --field 2 hc_unit_test tests/hf_lin_side_test
build.hf_link_side_test nvcc - any 4 5 0 prover/tests/unit.sh --field 2 build hash_commitment/tests/hf_link_side_test.cu ${S:?}/hf_link_side_field2
build.kept_sums.forest_edges_test nvcc - any 1 5 0 : built$ prover/tests/unit.sh --field 2 build tests/kept_sums/forest_edges_test.cu ${S:?}/kept_sums_f2
build.last_run_test nvcc - any 1 4 0 ^last_run_test: built$ prover/tests/unit.sh --field 2 build engine/tests/last_run_test.cu ${S:?}/engine.f2
build.make_model cpu - any 1 0.1 0 mkdir -p ${B:?} && cd prover && OUT=${B:?} ./build.sh --field 2 make_model
build.npsc_checks cpu - any 1 2 0 ^psc_rows\.sh: 2 of 2 checkers built$ prover/tests/psc_rows.sh checks ${S:?}/npsc_checks
build.psc_bytes_layer_test nvcc - any 1 6 0 : built$ prover/tests/unit.sh build tests/psc_bytes_layer_test.cu ${S:?}/psc_bytes_layer
build.psc_layer_test nvcc - any 1 6 0 : built$ prover/tests/unit.sh build tests/psc_layer_test.cu ${S:?}/psc_layer
build.psc_layer_test.count nvcc - any 1 6 0 : built$ prover/tests/unit.sh --slot-count build tests/psc_layer_test.cu ${S:?}/psc_layer.count
build.serving nvcc build.verifier any 2 45 0 mkdir -p ${B:?} && cd prover && OUT=${B:?} ./build.sh --field 2 serving
build.stream_coins_test nvcc - any 1 2 0 ^stream_coins_test: built$ prover/tests/unit.sh --field 2 build engine/tests/stream_coins_test.cu ${S:?}/engine.f2
build.token_sumcheck_psc_test nvcc - any 1 3 0 mkdir -p ${B:?} && cd prover && OUT=${B:?} ./build.sh --field 2 token_sumcheck_psc_test
build.tree_arrays_cleared_test nvcc - any 1 1 0 ^tree_arrays_cleared_test: built$ prover/tests/unit.sh --field 2 build engine/tests/tree_arrays_cleared_test.cu ${S:?}/engine.f2
build.tree_hash_test nvcc - any 2 8 0 mkdir -p ${B:?} && cd prover && OUT=${B:?} ./build.sh --field 2 tree_hash_test
build.verifier cpu - any 1 2.0 0 verifier/build.sh --field 2 && test -x bin/cp_verify
build.verifier.target cpu - any 1 1.0 0 verifier/build.sh --field 2 - ${S:?}/bin-target-f2 && test -x ${S:?}/bin-target-f2/cp_verify
build.verifier.unit cpu - any 1 1.2 0 mkdir -p ${S:?}/verifier-unit-f2 && cd verifier/src && g++ -std=c++17 -O2 -fno-ipa-modref -w -pthread -I. -I../../blocks -I../../prover -DBLK_FIELD=2 -DDIGEST_WORDS=8 -DVF_SOURCE_DIGEST='"0000000000000000000000000000000000000000000000000000000000000000"' -DCP_VERIFY_AS_A_UNIT -c cp_verify.cpp -o ${S:?}/verifier-unit-f2/unit.o && g++ -std=c++17 -O2 -fno-ipa-modref -w -pthread -I. -I../../blocks -I../../prover -DBLK_FIELD=2 -DDIGEST_WORDS=8 ../tests/cp_configured_test.cpp ${S:?}/verifier-unit-f2/unit.o -o ${S:?}/verifier-unit-f2/cp_configured_test
build.wf_heavy_test nvcc - any 1 3 0 ^wf_heavy_test: built$ prover/tests/unit.sh --field 2 build engine/tests/wf_heavy_test.cu ${S:?}/wf_heavy_f2
kit.small_model cpu build.make_model any 1 0.2 0,77 ^(?!(?!^not run: kit[.]small_model: this CPU cannot run the SHA instructions: the program died with SIGILL, and so did a one-instruction program that uses sha256rnds2; run it on a CPU that can$)not run: ) ( ${B:?}/make_model ${S:?}/small_model $(cat prover/examples/target_small_file_f2/model.txt); s=$?; if [ $s = 132 ]; then exec prover/tests/unit.sh explain-sigill kit.small_model; fi; exit $s )
setup.target.full_kit cpu build.verifier.target any 1 0.1 0 ^pack_layout\.py: [1-9][0-9]* rows over F, [1-9][0-9]* rows over E, [1-9][0-9]* blocks$ mkdir -p ${S:?}/target.full_kit.f2 && cp prover/examples/target_full_f2/policy.txt prover/examples/target_full_f2/KL.bin ${S:?}/target.full_kit.f2/ && rm -f ${S:?}/target.full_kit.f2/layout.txt && ${S:?}/bin-target-f2/cp_verify -plan arrays ${S:?}/target.full_kit.f2 128 > ${W:?}.arrays; [ $? = 4 ] && python3 -B prover/tools/pack_layout.py --arrays ${W:?}.arrays $(awk '$1 == "group" && $2 == "link" { print $5 }' ${S:?}/target.full_kit.f2/policy.txt) 4 2>&1 > ${S:?}/target.full_kit.f2/layout.txt
setup.target.other_parameters_worlds gpu build.cp_commit,build.cp_prove,build.verifier any 4 19 0 ^other_parameters_worlds: 17 worlds made, 3 sisters left out$ ;; ^table-radius . LEFT OUT: the verifier's planning call ;; ^true . made ;; ^work-positions-other . made ;; ^work-shared-other . made ;; ^work-committer-other . made ;; ^true . two requests that cover the token at T - 1 verifier/tests/other_parameters_worlds.sh ${B:?}/hc/test_build prover/tests/configurations/pinned_no_proof_of_work ${S:?}/other-parameters-f2
setup.target.other_parameters_worlds.work gpu build.cp_commit,build.cp_prove,build.verifier any 4 19 0 ^other_parameters_worlds: 17 worlds made, 3 sisters left out$ ;; ^table-radius . LEFT OUT: the verifier's planning call ;; ^true . made ;; ^work-positions-other . made ;; ^work-shared-other . made ;; ^work-committer-other . made ;; ^true . two requests that cover the token at T - 1 verifier/tests/other_parameters_worlds.sh ${B:?}/hc/test_build prover/tests/configurations/pinned ${S:?}/other-parameters-f2-work
setup.target.small_kit cpu build.verifier.target any 1 0.1 0 ^pack_layout\.py: [1-9][0-9]* rows over F, [1-9][0-9]* rows over E, [1-9][0-9]* blocks$ mkdir -p ${S:?}/target.small_kit.f2 && cp prover/tests/configurations/pinned_no_proof_of_work/policy.txt prover/tests/configurations/pinned_no_proof_of_work/KL.bin ${S:?}/target.small_kit.f2/ && rm -f ${S:?}/target.small_kit.f2/layout.txt && ${S:?}/bin-target-f2/cp_verify -plan arrays ${S:?}/target.small_kit.f2 128 > ${W:?}.arrays; [ $? = 4 ] && python3 -B prover/tools/pack_layout.py --arrays ${W:?}.arrays $(awk '$1 == "group" && $2 == "link" { print $5 }' ${S:?}/target.small_kit.f2/policy.txt) 4 2>&1 > ${S:?}/target.small_kit.f2/layout.txt

Quick start

These rows run what docs/quickstart.md offers a first-time user: the two blocks of commands in its section 3 that build and run the example, not the block that checks the machine, and the example script prover/examples/run_target_example.sh on one GPU, and blocks/first_look.sh, which makes and checks toy proofs on a CPU alone.

name kind needs word slots minutes passes command
gpu.quick_start gpu-by-run - any 4 55 0 ^quick_start_test\.py: 2 as expected, 0 otherwise; mkdir -m 700 ${W:?} && python3 -B toy/quick_start_test.py --gpu ${W:?}/play
gpu.run_target_example gpu-by-run build.verifier,build.serving any 1 2 0 ^status 0 YES$ ;; ^proof of work: up to 32 bits counted in [1-9][0-9]* rounds?; with it the least round has [0-9]+\.[0-9][0-9] bits, without it 96\.2[0-9] bits$ ;; ^result-v1 status=0 .* work=32 .* least-round-without-work=96\.2[0-9] ;; ^commitment: (?!1 )[1-9][0-9]* closeness rounds of the one-time files rest on another source than the bound within a third of the distance; the table has a record: the binding of the weights rests on the theorem on codes \(ePrint 2025/2055, Theorem 4\.6\), taken as given mkdir -p ${W:?} && ( BUILD=${B:?}/hc prover/examples/run_target_example.sh target_small_f2_roomy ${W:?}/play > ${W:?}/log 2>&1; s=$?; tail -n 4 ${W:?}/log; grep '^result-v1' ${W:?}/play/cp_verify.log; echo "status $s $(tail -n 1 ${W:?}/log)" )
new.first_look cpu - any 8 2 0 /bent\.proof: [0-9]+ bytes: [^A] ;; /honest\.proof: [0-9]+ bytes: ACCEPTED$ mkdir -p ${W:?} && blocks/first_look.sh ${W:?}/look
sources.quickstart_gpu cpu - any 1 0.1 0 ^quickstart_gpu_test\.py: 5 as expected, 0 otherwise$ python3 -B examples/quickstart_gpu_test.py

Known answers

These rows compare routines, or stored reference files, with expected answers: published test vectors, values written in the format document or the protocol specification, or values from a separately written program (such programs are less independent of what they check than the name says: docs/limits-all-rows.tsv, rows E2, E2.4 and E2.8). In prover.code_test_values, prover.coin_keys.values, prover.coins_known_test, sources.format_vectors, t-s8_model, toy.mask_rules, verifier.forest_values, verifier.format_hash and verifier.kit_rounds the stored answers are the same program's own output, so the comparison with them shows only that they are unchanged. verifier.hiding_coins compares nothing with any file or value of the three programs. gpu.coins_own_gpu_test does not show that a program refuses to run when a self-test fails.

name kind needs word slots minutes passes command
commit.left_working_names cpu - any 1 0.2 0,77 ^(?!(?!.*?(?:: passes$))(?!^not run: [A-Za-z0-9_]+: this CPU cannot run the SHA instructions: the test died with SIGILL, and so did a one-instruction program that uses sha256rnds2; run it on a CPU that can$)) prover/tests/unit.sh both hash_commitment/tests/left_working_names_test.cpp ${W:?} "${W:?}/made"
gpu.blake2b_f_test gpu build.blake2b_f_test any 1 0.2 0 : passes$ mkdir -p ${W:?} && python3 -B verifier/tests/known_answers/blake2b_f.py 2000 1 > ${W:?}/f.calls.txt && prover/tests/unit.sh --field 2 run tests/blake2b_f_test.cu ${S:?} "${W:?}/f.calls.txt 2000"
gpu.coins gpu - any 2 1.7 0 ^coin_gpu_tests: all hold$ prover/tests/coin_gpu_tests.sh ${S:?}/coins
gpu.coins_own_gpu_test gpu build.coins_own_gpu_test any 1 0.2 0 ^coins_own_gpu_test: all hold$ ;; ^the long rows' streams: not played cd prover && ${S:?}/coins_own_f2/coins_own_gpu_test
gpu.hf_commit_table_test gpu build.hf_commit_table_test any 1 1 0 ^hf_commit_table_test: all 421 cases agree$ prover/tests/unit.sh --field 2 run hash_commitment/tests/hf_commit_table_test.cu ${S:?}/hf_commit_table_test.field2 && tail -n 1 ${S:?}/hf_commit_table_test.field2/hf_commit_table_test.log
gpu.kept_sums.forest_edges gpu build.kept_sums.forest_edges_test any 1 0.3 0 ^all 33 lines ok$ cd prover && ${S:?}/kept_sums_f2/forest_edges_test
gpu.tree_hash_test gpu build.tree_hash_test any 1 0.3 0 ^tree_hash_test: 13 vectors, 18 further lengths, two implementations: passes$ cd prover && ${B:?}/tree_hash_test
prover.code_test_values cpu - any 1 0.1 0 ^code_test_values\.py: 9 cases; 0 of them do not stand in prover/hash_commitment/tests/hf_code_test\.cu$ python3 -B prover/hash_commitment/tests/code_test_values.py --hold prover/hash_commitment/tests/hf_code_test.cu
prover.coin_keys cpu - any 1 0.2 0 ^coin_keys_test: 238 checks as expected, 0 otherwise$ mkdir -p ${W:?} && cd prover && g++ -std=c++17 -O2 -Wall -Wextra -fno-ipa-modref -DBLK_FIELD=2 -DBLK_HASH=1 -DDIGEST_WORDS=8 -I . -I ../blocks tests/coin_keys_test.cpp -o ${W:?}/coin_keys_test && ${W:?}/coin_keys_test
prover.coin_keys.values cpu - any 1 0.1 0 mkdir -p ${W:?} && cd prover && python3 -B tests/coin_keys_values.py > ${W:?}/values.txt && cmp ${W:?}/values.txt tests/coin_keys_values.txt
prover.coins_known_host_test cpu - any 1 0.3 0,77 ^(?!(?!^coins_known_host_test: passes$)(?!^not run: [A-Za-z0-9_]+: this CPU cannot run the SHA instructions: the test died with SIGILL, and so did a one-instruction program that uses sha256rnds2; run it on a CPU that can$)) ${T:?}/prover/tests/unit.sh --field 2 both tests/coins_known_host_test.cpp ${W:?}
prover.coins_known_test cpu - any 1 0.2 0 ^coins_known_test: passes$ bash ${T:?}/prover/tests/coins_known_test.sh ${W:?} 2
prover.coins_rfc8439_test cpu - any 1 0.2 0,77 ^(?!(?!^coins_rfc8439_test: passes$)(?!^not run: [A-Za-z0-9_]+: this CPU cannot run the SHA instructions: the test died with SIGILL, and so did a one-instruction program that uses sha256rnds2; run it on a CPU that can$)) ${T:?}/prover/tests/unit.sh --field 2 both tests/coins_rfc8439_test.cpp ${W:?}
prover.hash_test cpu - any 1 0.1 0,77 ^(?!(?!.*?(?:: passes$))(?!^not run: [A-Za-z0-9_]+: this CPU cannot run the SHA instructions: the test died with SIGILL, and so did a one-instruction program that uses sha256rnds2; run it on a CPU that can$)) ${T:?}/prover/tests/unit.sh --field 2 both tests/hash_test.cpp ${W:?}
prover.hc_chacha_test cpu - any 1 0.1 0 ^all 36 tests passed$ mkdir -p ${W:?} && cd prover && OUT=${W:?} ./build.sh --field 2 hc_host_test tests/hc_chacha_test && ${W:?}/hc/test_build/tests/hc_chacha_test
prover.hf_code_test.first_part cpu - any 1 0.5 0 ^0 of 137 checks failed$ mkdir -p ${W:?} && cd prover && g++ -O2 -fno-ipa-modref -std=c++17 -DBLK_FIELD=2 -I. -I../blocks -x c++ hash_commitment/tests/hf_code_test.cu -o ${W:?}/hf_code_test_cpu && ${W:?}/hf_code_test_cpu
prover.hf_params_test cpu - any 1 0.1 0 ^\d+ checks, 0 differ$ mkdir -p ${W:?} && cd prover && OUT=${W:?} ./build.sh --field 2 hc_host_test tests/hf_params_test && ${W:?}/hc/test_build/tests/hf_params_test
prover.hf_record_test cpu - any 1 0.1 0 ^\d+ checks, 0 fail$ mkdir -p ${W:?} && cd prover && OUT=${W:?} ./build.sh --field 2 hc_host_test tests/hf_record_test && ${W:?}/hc/test_build/tests/hf_record_test ${W:?}
prover.hmac_sha256_test cpu - any 1 0.1 0,77 ^(?!(?!.*?(?:: passes$))(?!^not run: [A-Za-z0-9_]+: this CPU cannot run the SHA instructions: the test died with SIGILL, and so did a one-instruction program that uses sha256rnds2; run it on a CPU that can$)) prover/tests/unit.sh --field 2 both tests/hmac_sha256_test.cpp ${W:?}
prover.masking_cells_test cpu - any 1 0.1 0,77 ^(?!(?!.*?(?:: passes$))(?!^not run: [A-Za-z0-9_]+: this CPU cannot run the SHA instructions: the test died with SIGILL, and so did a one-instruction program that uses sha256rnds2; run it on a CPU that can$)) prover/tests/unit.sh --field 2 both tests/masking_cells_test.cpp ${W:?}
prover.parameter_file_one_bits_test cpu - any 1 0.1 0,77 ^(?!(?!^parameter_file_one_bits_test: passes$)(?!^not run: [A-Za-z0-9_]+: this CPU cannot run the SHA instructions: the test died with SIGILL, and so did a one-instruction program that uses sha256rnds2; run it on a CPU that can$)) prover/tests/unit.sh --field 2 both tests/parameter_file_one_bits_test.cpp ${W:?} examples/target_small_f2/policy.txt
prover.parameter_file_one_place_test cpu - any 1 0.1 0,77 ^(?!(?!^parameter_file_one_place_test: passes$)(?!^not run: [A-Za-z0-9_]+: this CPU cannot run the SHA instructions: the test died with SIGILL, and so did a one-instruction program that uses sha256rnds2; run it on a CPU that can$)) prover/tests/unit.sh --field 2 both tests/parameter_file_one_place_test.cpp ${W:?} examples/target_small_f2/policy.txt
prover.public_weights_test cpu - any 1 0.2 0,77 ^(?!(?!.*?(?:: passes$))(?!^not run: [A-Za-z0-9_]+: this CPU cannot run the SHA instructions: the test died with SIGILL, and so did a one-instruction program that uses sha256rnds2; run it on a CPU that can$)) prover/tests/unit.sh --field 2 both tests/public_weights_test.cpp ${W:?} "${W:?}/written"
scales.policy_tables cpu - any 1 1.2 0 ^OK ;; ^Ran 23 tests? in python3 -B scales/tests/test_policy_tables.py
scales.table_ties cpu - any 1 0.8 0 ^OK: 26 checks$ python3 -B scales/tests/table_ties.py
sources.format_vectors cpu - any 1 0.1 0 ^format-vectors.txt: [1-9]\d* lines, as format_rules.py prints them$ mkdir -p ${W:?} && python3 -B docs/format/format_rules.py > ${W:?}/print && cmp ${W:?}/print docs/format/format-vectors.txt && echo "format-vectors.txt: $(wc -l < ${W:?}/print) lines, as format_rules.py prints them"
t-s8_model cpu - any 1 0.4 0 ^the model prints the stored file$ mkdir -p ${W:?} && cd tests && python3 -B s8_model.py -write ${W:?}/model.tsv && cmp ${W:?}/model.tsv s8_known_answers.tsv && echo "the model prints the stored file"
tools.audit.coins cpu - any 1 0.1 0,77 ^(?!(?!draw +as the known answer$)(?!not run: )) python3 -B tools/audit/coins_known.py ${T:?}
tools.lean_verify.claim cpu - any 1 0.1 0 ^read_claim_test\.py: 4 stored examples: all as expected$ python3 -B tools/lean-verify/read_claim_test.py ${T:?}
tools.lean_verify.full_size long - any 8 40 0,77 ^(?!(?!VERDICT: ACCEPT \(exact match\)$)(?!not run: tools[.]lean_verify[.]full_size: )) ( if ! command -v prlimit > /dev/null; then echo "not run: tools.lean_verify.full_size: prlimit (util-linux) is not on the PATH"; exit 77; fi; if [ ! -f lean/.lake/build/lib/lean/Johnson/PVR/ProgramGuard.olean ]; then echo "not run: tools.lean_verify.full_size: the Lean library is not built (tools/lean-verify/README.md, What to type, step 1)"; exit 77; fi; while read -r m; do if [ -z "$m" ]; then continue; fi; if [ "${m#\#}" != "$m" ]; then continue; fi; p=${m//.//}; f=; for o in lean/.lake/build/ir/$p.c.o.export lean/.lake/packages/*/.lake/build/ir/$p.c.o.export; do if [ -f "$o" ]; then f=1; fi; done; if [ -z "$f" ]; then echo "not run: tools.lean_verify.full_size: the native objects of the helper library are not built (tools/lean-verify/README.md, What to type, step 2)"; exit 77; fi; done < tools/lean-verify/native-modules.txt; mkdir -p ${W:?} && E=examples/full-size && TMPDIR=${W:?} exec tools/lean-verify/lean_verify.sh $E/config $E/ivk $E/example.req $E/example.y $E/example.proof $E/ID.txt )
tools.lean_verify.script cpu - any 1 3 0,77 ^(?!(?!lean_verify_test\.sh: 346 calls: all as expected$)(?!not run: tools[.]lean_verify[.]script: )) if ! command -v prlimit > /dev/null; then echo "not run: tools.lean_verify.script: prlimit (util-linux) is not on the PATH"; exit 77; fi; mkdir -p ${W:?} && tools/lean-verify/lean_verify_test.sh ${W:?}/play
tools.parser.small cpu build.verifier.unit any 1 1 0 ^parser_test\.py: accepted-proof: 3 files, 24 damaged originals, 33 damaged parsed, order and seeds files: all as expected$ python3 -B tools/parser/parser_test.py ${T:?} ${S:?}/verifier-unit-f2/unit.o ${T:?}/examples/accepted-proof ${W:?}
toy.mask_rules cpu - any 1 0.1 0 ^mask_rules.py: field 2: 3 known answers at the full threshold take a second try$ ;; field 1: of the 28 known answers 0 do not come out$ ;; field 2: of the 31 known answers 0 do not come out$ python3 -B toy/mask_rules.py
verifier.field_setting_test cpu - any 1 0.3 0 ^field_setting_test, setting 2: all passed$ mkdir -p ${W:?} && cd verifier && g++ -std=c++17 -O2 -pthread -fno-ipa-modref -I src -I ../blocks -I ../prover -DBLK_FIELD=2 -DDIGEST_WORDS=8 tests/field_setting_test.cpp -o ${W:?}/field_setting_test && ${W:?}/field_setting_test
verifier.forest_attention cpu - any 1 0.9 0 ^hc_parts_test.sh: hc_forest_attention_test: all as expected verifier/tests/hc_parts_test.sh --field 2 hc_forest_attention_test ${W:?}/play ${T:?}/verifier/tests/forest_weights.txt
verifier.forest_values cpu - any 1 0.1 0 lines made again, 0 differ from the file$ python3 -B verifier/tests/known_answers/forest_values.py --hold verifier/tests/forest_values.txt && python3 -B verifier/tests/known_answers/forest_values.py --weights --hold verifier/tests/forest_weights.txt
verifier.format_hash cpu - any 1 0.1 0 ^all as expected$ ;; ^57 lines checked, 0 with a different answer$ python3 -B verifier/tests/known_answers/format_hash.py
verifier.hiding_coins cpu - any 1 0.1 0,77 ^(?!(?!hiding_coins\.py: 0 of its own known answers do not hold$)(?!not run: )) python3 -B verifier/tests/known_answers/hiding_coins.py
verifier.kit_rounds cpu build.verifier.target any 1 0.2 0 ^kit_rounds_test\.py: 3 kits, [1-9]\d* kinds of rounds; 0 kits not as stored$ python3 -B verifier/tests/kit_rounds_test.py ${S:?}/bin-target-f2/cp_verify ${W:?} 2
verifier.kit_rounds_second cpu build.verifier.target any 1 0.2 0 ^kit_rounds_second\.py: 3 kits: 1219 stored lines, 403 computed: 397 the same, 6 known to claim less, 0 off the table KNOWN; of the 816 not computed the lowest has 152\.41 bits; 593 of 593 derived numbers as printed; 115 lists of positions, 0 too short or longer than need be; 0 faults in the pairing; log2 of the mask rows fitted in each key \(the ranges', the own test's\): 4 0, 0 0, 0 0: as expected$ python3 -B verifier/tests/kit_rounds_second.py ${S:?}/bin-target-f2/cp_verify ${W:?}/play
verifier.walker_on_a_world cpu build.verifier.unit,setup.target.other_parameters_worlds any 1 1 0 ^walker_on_a_world_test\.py: field 2: a proof of [1-9]\d* bytes, 1 control\(s\): all as expected$ ;; ^as expected +the-last-opening-cut-off \(\d+ bytes\): status 1: ;; ^as expected +the-first-opening-taken-out \(\d+ bytes\): status 1: ;; ^as expected +all-openings-taken-out \(\d+ bytes\): status 1: ;; ^as expected +a-second-opening-taken-out \(\d+ bytes\): THE LIMIT: read whole: status 0: python3 -B verifier/tests/walker_on_a_world_test.py ${T:?} ${S:?}/verifier-unit-f2/unit.o ${S:?}/other-parameters-f2/worlds/true ${W:?}/play 2
verifier.walker_on_a_world.work cpu build.verifier.unit,setup.target.other_parameters_worlds.work any 1 1 0 ^walker_on_a_world_test\.py: field 2: a proof of [1-9]\d* bytes, 1 control\(s\): all as expected$ ;; ^as expected +the-last-opening-cut-off \(\d+ bytes\): status 1: ;; ^as expected +the-first-opening-taken-out \(\d+ bytes\): status 1: ;; ^as expected +all-openings-taken-out \(\d+ bytes\): status 1: ;; ^as expected +a-second-opening-taken-out \(\d+ bytes\): THE LIMIT: read whole: status 0: python3 -B verifier/tests/walker_on_a_world_test.py ${T:?} ${S:?}/verifier-unit-f2/unit.o ${S:?}/other-parameters-f2-work/worlds/true ${W:?}/play 2
verifier.weights_twin_test cpu - any 1 2 0 ^weights_twin_test: ok \(48 worlds, 780 units, 979500 cells, 7389952 places without a cell; 5508 checks\)$ ;; ^0 - 1 = 0xffffffff00000000 in the verifier's field$ mkdir -p ${W:?} && cd verifier/src && g++ -std=c++17 -O2 -fno-ipa-modref -pthread -DDIGEST_WORDS=8 -DBLK_FIELD=2 -DBLK_HASH=1 -I . -I ../../blocks -I ../../prover ../tests/weights_twin_test.cpp -o ${W:?}/weights_twin_test && ${W:?}/weights_twin_test
verifier.query_width_bound cpu - any 1 0.1 0 ^query_width_bound_test: ok \(2043 checks; 959 of 1984 pairs \(H, dh\) with H in 1\.\.64 and dh = 2\^1\.\.2\^31 admitted\)$ mkdir -p ${W:?} && cd verifier/src && g++ -std=c++17 -O2 -fno-ipa-modref -Wall -Wextra -DDIGEST_WORDS=8 -I . ../tests/query_width_bound_test.cpp -o ${W:?}/query_width_bound_test && ${W:?}/query_width_bound_test
verifier.query_width_bound.sanitized cpu - any 1 0.1 0 ^query_width_bound_test: ok \(2043 checks; 959 of 1984 pairs \(H, dh\) with H in 1\.\.64 and dh = 2\^1\.\.2\^31 admitted\)$ mkdir -p ${W:?} && cd verifier/src && g++ -std=c++17 -O1 -g -fno-ipa-modref -fsanitize=address,undefined -fno-sanitize-recover=all -DDIGEST_WORDS=8 -I . ../tests/query_width_bound_test.cpp -o ${W:?}/query_width_bound_test && ${W:?}/query_width_bound_test
weight_sums cpu - any 4 3.5 0,77 ^run_tests: every step that ran passed weight_sums/tests/run_tests.sh

Refusals of altered, malformed and extreme files

These rows give altered, malformed, cut, crossed or extreme files, parameter files or calls to cp_verify itself in most rows, directly or through a script that runs the programs; in the others to the committer or the prover, to single routines of the verifier in a test program, to the tool prover/tools/pack_layout.py (tools.pack_layout.refusals), or to the library's example programs with a second verifier in Python (examples.protocol_toy_false_files): each row's command says which. Most check that each is refused; a few check only that the program ends cleanly, with a plan, a named refusal or a usage message and no crash. The common names of the verifier's refusals are explained in verifier/docs/output.md, section "Rejection names", and every name in docs/format/FORMAT.md, section 13.8.

No row on this page gives cp_verify a proof made by a cheating program. No one list gives, for each kind of false input, the test that tries it and what is left untried. No row on this page tests any of the six things that follow, nor these two: that a partial check, or a full check on honest files, names the condition of the standard parameter form that is not met, apart from next-token and id-named in a full check and no-proof-of-work in a full check and in both partial checks; a proof made with counted proof of work under a changed setting (docs/limits-all-rows.tsv, row E11). Many tests' files say in their head comment what the test leaves untried.

  • The last-round check that the word the prover sent, re-encoded, matches the symbols opened at the drawn positions. No run has shown this check to be the one that refused a false proof.
  • That a commitment to a word far from the code is refused, which is what the proximity test is for: the committer here always commits to a codeword, so no test can commit to a word that is none.
  • The check that a request's first chain and the shared chain use one set of codes, one shape and one table; without it a claim on another matrix was accepted in a test program.
  • That the verifier's lookup routines accept a lawful count at a repeated table row.
  • That a prover who retries its nonces against a word mixed from two codewords does no better than the design predicts.
  • The verifier's claim-by-claim approximate range check, which has its own test in the tree (verifier/tests/tail_twin_test.cpp) that a reader builds by hand.

verifier.call_accounting.examples runs on honest proofs alone. It does not show that an altered or forged proof cannot make the verifier hash more than it planned, and no other row tries it.

name kind needs word slots minutes passes command
examples.protocol_toy_false_files cpu - any 1 5 0 ^stored_false_test\.py: all 12 false files refused and all 4 honest files accepted, each with the words in LIST\.tsv$ mkdir -p ${W:?} && env -u SETTINGS make -s -C blocks OUT=${W:?}/build ${W:?}/build/f2h1/frame/handover_files.bin ${W:?}/build/f2h1/frame/handover_proof.bin > ${W:?}/build.txt 2>&1 && python3 -B examples/protocol-toy-false-files/stored_false_test.py ${W:?}/build/f2h1/frame ${W:?}/judged
gpu.one_changed_weight gpu-by-run build.verifier,build.serving,kit.small_model any 1 4 0 ^one_changed_weight, version H821: as foretold$ prover/tests/one_changed_weight.sh ${B:?}/hc ${W:?}/play prover/examples/target_small_file_f2 ${S:?}/small_model
gpu.record_altered_sets gpu-by-run build.verifier,build.serving any 1 1 0 ^record_altered_sets_test\.py: 7 calls on the honest set and its unaltered copy, 23 altered sets: all as expected$ ;; ^as expected +the honest set, the whole call: status 0: YES$ ;; ^as expected +its line result-v1 \(1 such line\(s\)\): the ID is the call's; status=0 bits=128 work=32 work-rounds=[1-9]\d* record=1$ ;; ^as expected +the honest set, the whole call with -no-proof-of-work: status 14: NOT-A-VERIFICATION$ ;; ^as expected +the unaltered copy, the whole call: status 0: YES$ ;; ^as expected +whole\.key-first-answer \(key\.bin: bit 0 of byte 40\): status 1: NO: model-statement$ ;; ^as expected +whole\.key-last-answer \(key\.bin: bit 0 of byte 6183\): status 1: NO: model-statement$ ;; ^as expected +whole\.leaf-first-answer \(leaf/0\.bin: bit 0 of byte 40\): status 1: NO: entry-refused$ ;; ^as expected +whole\.leaf-last-answer \(leaf/0\.bin: bit 0 of byte 6183\): status 1: NO: entry-refused$ ;; ^as expected +whole\.records-exchanged \(.*\): status 1: NO: model-statement$ ;; ^as expected +whole\.leaf-with-the-key-files-record \(.*\): status 1: NO: entry-refused$ ;; ^as expected +key\.records-exchanged \(.*\): status 1: NO: key-refused$ ;; ^as expected +leaf\.records-exchanged \(.*\): status 1: NO: entry-refused$ ;; ^as expected +whole\.key-after-the-record \(key\.bin: bit 0 of byte 6184\): status 1: NO: key-refused$ ;; ^as expected +whole\.key-middle \(.*\): status 1: NO: key-refused$ ;; ^as expected +whole\.leaf-middle \(.*\): status 1: NO: entry-refused$ ;; ^as expected +whole\.proof-middle \(.*\): status 1: NO: proof-refused$ mkdir -p ${W:?} && BUILD=${B:?}/hc prover/examples/run_target_example.sh target_small_f2_roomy ${W:?}/play && D=${W:?}/play && python3 -B verifier/tests/record_altered_sets_test.py ${B:?}/hc/cp_verify $D/config $D/ivk $D/in/example.req $D/out/example.y $D/out/example.proof $(sed -n "s/^commitment ID: //p" $D/cp_commit.log) ${W:?}/altered
gpu.refused_at_the_start.serving gpu-by-run build.verifier,build.serving any 1 2.5 0 ^refused_at_the_start: 30 scenes, all as expected$ prover/tests/refused_at_the_start.sh ${B:?}/hc prover/examples/target_small_f2_roomy ${W:?}/play prover/examples/target_small_f2/policy.txt
gpu.serving_links gpu-by-run build.verifier,build.serving,kit.small_model any 1 3 0 ^serving_links, version H821: 10 plays, all as expected$ prover/tests/serving_links.sh ${B:?}/hc ${W:?}/play prover/examples/target_small_file_f2 ${S:?}/small_model
prover.batch_opening_test cpu - any 1 1.5 0 ^batch_opening_test: ok \(9 views\)$ mkdir -p ${W:?} && cd prover && g++ -std=c++17 -O2 -Wall -Wextra -fno-ipa-modref -DBLK_FIELD=2 -DBLK_HASH=1 -DDIGEST_WORDS=8 -I . -I ../blocks tests/batch_opening_test.cpp -o ${W:?}/batch_opening_test && ${W:?}/batch_opening_test
tools.pack_layout.refusals cpu - any 1 0.5 0 ^pack_layout_refusals_test\.py: 45 cases: all as expected$ python3 -B prover/tests/pack_layout_refusals_test.py ${W:?}
verifier.call_accounting.examples cpu build.verifier.target any 2 0.5 0,77 ^(?!(?!call_accounting_test\.py: [1-9]\d* lines of the plan's accounting, 0 not as in the call$)(?!full-size: not run: the set is not in this tree$)) ;; ^the line covered: is, letter for letter, what the page's rules give: covered: r=60 e=64 T=64 .* policy=exact-match above-clip=0 \(\.\.\)$ ;; ^(?!(?!the line covered: is, letter for letter, what the page's rules give: covered: r=23 e=1024 T=1024 .* policy=exact-match above-clip=0 \(\.\.\)$)(?!full-size: not run: the set is not in this tree$)) ( absent= && mkdir -p ${W:?} && for x in accepted-proof full-size; do D=examples/$x && if [ $x = full-size ]; then n=0; for f in example.proof ivk/key.bin ivk/leaf/0.bin; do if [ -e $D/$f ]; then n=$((n+1)); fi; done; if [ $n = 0 ]; then absent=1; continue; fi; if [ $n != 3 ]; then echo "examples/full-size: $n of its three large files (example.proof, ivk/key.bin, ivk/leaf/0.bin) are in this tree: all three or none"; exit 1; fi; fi && echo "$D:" && ${S:?}/bin-target-f2/cp_verify $D/config $D/ivk $D/example.req $D/example.y $D/example.proof -id $(cat $D/ID.txt) > ${W:?}/$x.txt && python3 -B verifier/tests/call_accounting_test.py ${S:?}/bin-target-f2/cp_verify $D/config 128 ${W:?}/$x.txt $D/example.req && continue; echo "failed on examples/$x (status $?); the verifier's output is in ${W:?}/$x.txt"; exit 1; done; if [ -n "${absent:-}" ]; then echo "full-size: not run: the set is not in this tree"; exit 77; fi )
verifier.clang_build cpu build.verifier.target any 4 6 0,77 ^(clang_build_test\.sh: field 2: [1-9]\d* planning calls the same, 0 calls )?not[ ;] verifier/tests/clang_build_test.sh --field 2 ${S:?}/bin-target-f2/cp_verify ${W:?}
verifier.commitment_id.examples cpu build.verifier.target any 1 0.1 0,77 ^(?!(?!commitment_id_test\.py: all as expected$)(?!full-size: not run: the set is not in this tree$)) ;; ^commitment_id_test\.py: all as expected$ ;; ^examples/accepted-proof: the ID [0-9a-f]{64}$ ;; ^examples/accepted-proof-clipped-gap: the ID [0-9a-f]{64}$ ;; ^(?!(?!examples/full-size: the ID [0-9a-f]{64}$)(?!full-size: not run: the set is not in this tree$)) python3 -B verifier/tests/commitment_id_test.py ${S:?}/bin-target-f2/cp_verify examples/accepted-proof examples/accepted-proof-clipped-gap && n=0 && for f in example.proof ivk/key.bin ivk/leaf/0.bin; do if [ -e examples/full-size/$f ]; then n=$((n+1)); fi; done && if [ $n = 3 ]; then python3 -B verifier/tests/commitment_id_test.py ${S:?}/bin-target-f2/cp_verify examples/full-size; elif [ $n = 0 ]; then echo "full-size: not run: the set is not in this tree"; exit 77; else echo "examples/full-size holds $n of its three large files (example.proof, ivk/key.bin, ivk/leaf/0.bin): all three or none"; exit 1; fi
verifier.entries_and_parts cpu setup.target.other_parameters_worlds any 1 0.3 0 ^entries_and_parts_test: ok$ ;; ^2 the head turned to another copy's number, under which the proof's own entry lies: status 1: NO: (proof-refused)?(chain-refused)?$ ;; ^8 the honest entry of another copy under the proof's number: status 1: NO: (proof-refused)?(chain-refused)?$ ;; ^4 -leaf on a key that is hollow behind its root: status 3: python3 -B verifier/tests/entries_and_parts_test.py ${B:?}/hc/test_build/cp_verify ${S:?}/other-parameters-f2/worlds/true ${W:?} 0 3
verifier.entries_and_parts.work cpu setup.target.other_parameters_worlds.work any 1 0.3 0 ^entries_and_parts_test: ok$ ;; ^2 the head turned to another copy's number, under which the proof's own entry lies: status 1: NO: (proof-refused)?(chain-refused)?$ ;; ^8 the honest entry of another copy under the proof's number: status 1: NO: (proof-refused)?(chain-refused)?$ ;; ^4 -leaf on a key that is hollow behind its root: status 3: python3 -B verifier/tests/entries_and_parts_test.py ${B:?}/hc/test_build/cp_verify ${S:?}/other-parameters-f2-work/worlds/true ${W:?} 0 3
verifier.example_kits cpu build.verifier.target any 1 1 0 ^example_kits_test\.py: 7 parameter files \(1 bare, 5 work, 1 yes\), 12 plays with a line changed; 0 not as expected$ ;; ^asked for one bit above the file's target: [1-9]\d* files$ ;; ^out of range: the line record: 2 of 2 plays end NO: parameter-range \(policy\)$ ;; ^out of range: a group's first number: 16 of 16 plays end NO: parameter-group-shape \(policy\)$ ;; ^out of range: the table's fifth number: 1 of 1 plays end NO: parameter-table-shape \(policy\)$ python3 -B verifier/tests/example_kits_test.py ${S:?}/bin-target-f2/cp_verify ${W:?} 2
verifier.extreme_files cpu build.verifier.target any 8 0.4 0 ^extreme_files_test\.py: [1-9]\d* calls on [1-9]\d* files: planned [1-9]\d*, refused [1-9]\d*, usage \d+; fault 0, not-checked 0, slow 0, honest calls not as expected 0$ python3 -B verifier/tests/extreme_files_test.py ${S:?}/bin-target-f2/cp_verify ${W:?} prover/tests/configurations/pinned_no_proof_of_work --jobs 8
verifier.parameter_walk_width_count cpu build.verifier.target any 2 0.1 0 ^parameter_walk_width_count_test\.py: seed refused policy-norm \(before shape-heads\); walk status 1, [1-9]\d* files with H \* dh != d, 0 counted: as expected$ python3 -B verifier/tests/parameter_walk_width_count_test.py ${S:?}/bin-target-f2/cp_verify
verifier.extreme_files.full cpu build.verifier.target any 4 1.6 0 ^extreme_files_test\.py: [1-9]\d* calls on [1-9]\d* files: planned [1-9]\d*, refused [1-9]\d*, usage \d+; fault 0, not-checked 0, slow 0, honest calls not as expected 0$ python3 -B verifier/tests/extreme_files_test.py ${S:?}/bin-target-f2/cp_verify ${W:?} prover/examples/target_full_f2 --jobs 4 --every 8
verifier.extreme_proof_files cpu setup.target.other_parameters_worlds any 8 7 0 ^extreme_proof_files_test\.py: 3 honest calls and [1-9]\d* altered files: refused [1-9]\d*, usage \d+; fault 0, not-checked 0, slow 0, honest calls not as expected 0, not refused 0, second encodings or their control not as asked 0$ ;; ^the control p - 1 in the proof: [1-9]\d*: NO: proof-refused [1-9]\d*$ ;; ^second encodings in the key: [1-9]\d*: NO: key-refused [1-9]\d*$ ;; ^second encodings in the entry: [1-9]\d*: NO: entry-refused [1-9]\d*$ ;; ^second encodings in the key's record: 4: NO: key-refused 4$ ;; ^second encodings in the entry's record: 4: NO: entry-refused 4$ python3 -B verifier/tests/extreme_proof_files_test.py ${B:?}/hc/test_build/cp_verify ${W:?} ${S:?}/other-parameters-f2/worlds/true --jobs 8
verifier.forest_twin_test cpu - any 1 2 0 ^forest_twin_test: ok \(13 worlds: 7600 slots, 717 claims; in each, a host that forgets one kind of claims is refused by name, another padding number and every changed mask entry are noticed; 3 worlds with a call of nothing but padding refused by both texts\)$ ;; ^0 - 1 = 0xffffffff00000000 in the verifier's field, 0xffffffff00000000 in blocks/$ mkdir -p ${W:?} && cd verifier/src && g++ -std=c++17 -O2 -fno-ipa-modref -pthread -DDIGEST_WORDS=8 -DBLK_FIELD=2 -DBLK_HASH=1 -I . -I ../../blocks -I ../../prover ../tests/forest_twin_test.cpp -o ${W:?}/forest_twin_test && ${W:?}/forest_twin_test
verifier.hostile_leaf_files cpu build.verifier.unit,build.verifier.target any 4 3 0 ^hostile_leaf_files_test\.py: 186 changed sets, 372 calls: 186 refused, 0 NOT REFUSED, 0 otherwise; the unchanged set passes$ python3 -B verifier/tests/hostile_leaf_files_test.py ${T:?} ${S:?}/verifier-unit-f2/unit.o ${S:?}/bin-target-f2/cp_verify ${T:?}/examples/accepted-proof ${W:?}/play 4
verifier.la_equals_lt cpu build.verifier.target any 1 0.1 1 ^NO: shape-la$ mkdir -p ${W:?} && K=prover/tests/configurations/pinned_no_proof_of_work && sed 's/^la 8$/la 6/' $K/policy.txt > ${W:?}/policy.txt && grep -q '^la 6$' ${W:?}/policy.txt && grep -q '^lt 6$' ${W:?}/policy.txt && head -c 264 /dev/zero > ${W:?}/a.req && head -c 4 /dev/zero > ${W:?}/y.bin && ${S:?}/bin-target-f2/cp_verify -plan request ${W:?}/policy.txt $K/KL.bin 0 -layout $K/layout.txt -prompt ${W:?}/a.req -reply ${W:?}/y.bin
verifier.link_door cpu - any 1 1.0 0 ^link_door_test: 200 runs: ok$ mkdir -p ${W:?} && cd verifier/src && g++ -std=c++17 -O2 -fno-ipa-modref -pthread -DDIGEST_WORDS=8 -DBLK_FIELD=2 -DBLK_HASH=1 -I . -I ../../blocks -I ../../prover -o ${W:?}/link_door_test ../tests/link_door_test.cpp && ${W:?}/link_door_test
verifier.made_up_files.full cpu build.verifier.target,setup.target.full_kit any 4 0.2 0 ^made_up_files_test: ok$ ;; ^zero bytes, whole: all three stages ran: ;; ^one ID in all four calls: [0-9a-f]{64}$ python3 -B verifier/tests/made_up_files_test.py ${S:?}/bin-target-f2/cp_verify ${S:?}/target.full_kit.f2 ${W:?} 4
verifier.made_up_files.small cpu build.verifier.target,setup.target.small_kit any 4 0.1 0 ^made_up_files_test: ok$ ;; ^zero bytes, whole: all three stages ran: ;; ^one ID in all four calls: [0-9a-f]{64}$ ;; ^whole with another ID: status 1: NO: model-statement$ ;; ^another root, another ID: status 4$ python3 -B verifier/tests/made_up_files_test.py ${S:?}/bin-target-f2/cp_verify ${S:?}/target.small_kit.f2 ${W:?} 4
verifier.made_up_files.work cpu build.verifier.target any 4 0.1 0 ^made_up_files_test: ok$ ;; ^zero bytes, whole: all three stages ran: ;; ^one ID in all four calls: [0-9a-f]{64}$ python3 -B verifier/tests/made_up_files_test.py ${S:?}/bin-target-f2/cp_verify prover/examples/target_small_f2_roomy ${W:?} 4
verifier.one_chain_shell cpu - any 1 0.5 0 ^ok$ mkdir -p ${W:?} && cd verifier/tests && g++ -std=c++17 -O2 -fno-ipa-modref -DDIGEST_WORDS=8 -DBLK_FIELD=2 -DBLK_HASH=1 -I ../../blocks -I ../src -o ${W:?}/one_chain_shell_test one_chain_shell_test.cpp && ${W:?}/one_chain_shell_test
verifier.openings_left_out cpu build.verifier.unit,build.verifier.target any 4 1.5 0 ^openings_left_out_test\.py: 117 changed sets, 117 calls: 117 refused, 0 NOT REFUSED, 0 otherwise; the unchanged set passes$ python3 -B verifier/tests/openings_left_out_test.py ${T:?} ${S:?}/verifier-unit-f2/unit.o ${S:?}/bin-target-f2/cp_verify ${T:?}/examples/accepted-proof ${W:?}/play 4
verifier.other_parameters cpu setup.target.other_parameters_worlds any 8 3 0 ^other_parameters_test: ok$ ;; ^the control: 30 of 30 calls end with the honest status$ ;; ^the control's proof differs from the true world's: yes$ ;; , 0 calls in all not as foretold$ python3 -B verifier/tests/other_parameters_test.py ${B:?}/hc/test_build/cp_verify ${W:?} 4 14 15 ${S:?}/other-parameters-f2/worlds/true $(find ${S:?}/other-parameters-f2/worlds -mindepth 1 -maxdepth 1 -type d ! -name true ! -name '.*')
verifier.pending_row cpu - any 1 0.4 0 ^pending_row_test: 6 records: ok$ mkdir -p ${W:?} && cd verifier/src && g++ -std=c++17 -O2 -fno-ipa-modref -pthread -DDIGEST_WORDS=8 -DBLK_FIELD=2 -DBLK_HASH=1 -I . -I ../../blocks -I ../../prover -o ${W:?}/pending_row_test ../tests/pending_row_test.cpp && ${W:?}/pending_row_test
verifier.plan_record cpu - any 1 0.2 0 ^a stage's record of its accounting: \d+ checks, 0 failed: all as expected$ mkdir -p ${W:?} && g++ -O1 -fno-ipa-modref -std=c++17 -Wall -Wextra -DBLK_FIELD=2 -I verifier/src -I blocks -o ${W:?}/plan_record_test verifier/tests/plan_record_test.cpp && ${W:?}/plan_record_test
verifier.plan_tau cpu build.verifier.target any 1 0.1 0 ^plan_tau_test\.py: 13 cases: all as expected$ python3 -B verifier/tests/plan_tau_test.py ${S:?}/bin-target-f2/cp_verify prover/tests/configurations/pinned_no_proof_of_work ${W:?}
verifier.proof_fields_sweep cpu setup.target.other_parameters_worlds any 8 3 0 ^proof_fields_sweep_test: [1-9][0-9]* altered copies and 4 lengths: all refused$ D=${S:?}/other-parameters-f2/worlds/true && python3 -B verifier/tests/proof_fields_sweep_test.py ${B:?}/hc/test_build/cp_verify $D/config $D/ivk $D/in/example.req $D/out/example.y $D/out/example.proof $(cat $D/id.txt) ${W:?} 8
verifier.proof_fields_sweep.work cpu setup.target.other_parameters_worlds.work any 8 3 0 ^proof_fields_sweep_test: [1-9][0-9]* altered copies and 4 lengths: all refused$ D=${S:?}/other-parameters-f2-work/worlds/true && python3 -B verifier/tests/proof_fields_sweep_test.py ${B:?}/hc/test_build/cp_verify $D/config $D/ivk $D/in/example.req $D/out/example.y $D/out/example.proof $(cat $D/id.txt) ${W:?} 8
verifier.psc_twin_test cpu - any 1 1 0 ^psc_twin_test: ok \(trees of 2 to 32 leaves; 115 changed scalars and as many changed mask entries noticed\)$ ;; ^0 - 1 = 0xffffffff00000000 in the verifier's field, 0xffffffff00000000 in blocks/$ mkdir -p ${W:?} && cd verifier/src && g++ -std=c++17 -O2 -fno-ipa-modref -pthread -DDIGEST_WORDS=8 -DBLK_FIELD=2 -DBLK_HASH=1 -I . -I ../../blocks -I ../../prover -I ../../blocks/iop/tests ../tests/psc_twin_test.cpp -o ${W:?}/psc_twin_test && ${W:?}/psc_twin_test
verifier.record_altered_sets.example cpu build.verifier.target any 1 0.5 0 ^record_altered_sets_test\.py: 7 calls on the honest set and its unaltered copy, 23 altered sets: all as expected$ ;; ^as expected +the honest set, the whole call: status 0: YES$ ;; ^as expected +its line result-v1 \(1 such line\(s\)\): the ID is the call's; status=0 bits=128 work=32 work-rounds=[1-9]\d* record=1$ ;; ^as expected +the honest set, the whole call with -no-proof-of-work: status 14: NOT-A-VERIFICATION$ ;; ^as expected +the unaltered copy, the whole call: status 0: YES$ ;; ^as expected +whole\.key-first-answer \(key\.bin: bit 0 of byte 40\): status 1: NO: model-statement$ ;; ^as expected +whole\.key-last-answer \(key\.bin: bit 0 of byte 6183\): status 1: NO: model-statement$ ;; ^as expected +whole\.leaf-first-answer \(leaf/[0-9]+\.bin: bit 0 of byte 40\): status 1: NO: entry-refused$ ;; ^as expected +whole\.leaf-last-answer \(leaf/[0-9]+\.bin: bit 0 of byte 6183\): status 1: NO: entry-refused$ ;; ^as expected +whole\.records-exchanged \(.*\): status 1: NO: model-statement$ ;; ^as expected +whole\.leaf-with-the-key-files-record \(.*\): status 1: NO: entry-refused$ ;; ^as expected +key\.records-exchanged \(.*\): status 1: NO: key-refused$ ;; ^as expected +leaf\.records-exchanged \(.*\): status 1: NO: entry-refused$ ;; ^as expected +whole\.key-after-the-record \(key\.bin: bit 0 of byte 6184\): status 1: NO: key-refused$ ;; ^as expected +whole\.key-middle \(.*\): status 1: NO: key-refused$ ;; ^as expected +whole\.leaf-middle \(.*\): status 1: NO: entry-refused$ ;; ^as expected +whole\.proof-middle \(.*\): status 1: NO: proof-refused$ D=examples/accepted-proof && python3 -B verifier/tests/record_altered_sets_test.py ${S:?}/bin-target-f2/cp_verify $D/config $D/ivk $D/example.req $D/example.y $D/example.proof $(cat $D/ID.txt) ${W:?}/play
verifier.record_altered_sets.full cpu build.verifier.target any 4 5 0,77 ^(?!(?!record_altered_sets_test\.py: 7 calls on the honest set and its unaltered copy, 23 altered sets: all as expected$)(?!full-size: not run: the set is not in this tree$)) ;; ^(?!(?!as expected +the honest set, the whole call: status 0: YES$)(?!full-size: not run: the set is not in this tree$)) ;; ^(?!(?!as expected +its line result-v1 \(1 such line\(s\)\): the ID is the call's; status=0 bits=128 work=32 work-rounds=[1-9]\d* record=1$)(?!full-size: not run: the set is not in this tree$)) ;; ^(?!(?!as expected +the honest set, the whole call with -no-proof-of-work: status 14: NOT-A-VERIFICATION$)(?!full-size: not run: the set is not in this tree$)) ;; ^(?!(?!as expected +the unaltered copy, the whole call: status 0: YES$)(?!full-size: not run: the set is not in this tree$)) ;; ^(?!(?!as expected +whole\.key-first-answer \(key\.bin: bit 0 of byte 40\): status 1: NO: model-statement$)(?!full-size: not run: the set is not in this tree$)) ;; ^(?!(?!as expected +whole\.key-last-answer \(key\.bin: bit 0 of byte 49191\): status 1: NO: model-statement$)(?!full-size: not run: the set is not in this tree$)) ;; ^(?!(?!as expected +whole\.leaf-first-answer \(leaf/[0-9]+\.bin: bit 0 of byte 40\): status 1: NO: entry-refused$)(?!full-size: not run: the set is not in this tree$)) ;; ^(?!(?!as expected +whole\.leaf-last-answer \(leaf/[0-9]+\.bin: bit 0 of byte 49191\): status 1: NO: entry-refused$)(?!full-size: not run: the set is not in this tree$)) ;; ^(?!(?!as expected +whole\.records-exchanged \(.*\): status 1: NO: model-statement$)(?!full-size: not run: the set is not in this tree$)) ;; ^(?!(?!as expected +whole\.leaf-with-the-key-files-record \(.*\): status 1: NO: entry-refused$)(?!full-size: not run: the set is not in this tree$)) ;; ^(?!(?!as expected +key\.records-exchanged \(.*\): status 1: NO: key-refused$)(?!full-size: not run: the set is not in this tree$)) ;; ^(?!(?!as expected +leaf\.records-exchanged \(.*\): status 1: NO: entry-refused$)(?!full-size: not run: the set is not in this tree$)) ;; ^(?!(?!as expected +whole\.key-after-the-record \(key\.bin: bit 0 of byte 49192\): status 1: NO: key-refused$)(?!full-size: not run: the set is not in this tree$)) ;; ^(?!(?!as expected +whole\.key-middle \(.*\): status 1: NO: key-refused$)(?!full-size: not run: the set is not in this tree$)) ;; ^(?!(?!as expected +whole\.leaf-middle \(.*\): status 1: NO: entry-refused$)(?!full-size: not run: the set is not in this tree$)) ;; ^(?!(?!as expected +whole\.proof-middle \(.*\): status 1: NO: proof-refused$)(?!full-size: not run: the set is not in this tree$)) ( D=examples/full-size && n=0 && for f in example.proof ivk/key.bin ivk/leaf/0.bin; do if [ -e $D/$f ]; then n=$((n+1)); fi; done && if [ $n = 3 ]; then python3 -B verifier/tests/record_altered_sets_test.py ${S:?}/bin-target-f2/cp_verify $D/config $D/ivk $D/example.req $D/example.y $D/example.proof $(cat $D/ID.txt) ${W:?}/play; elif [ $n = 0 ]; then echo "full-size: not run: the set is not in this tree"; exit 77; else echo "examples/full-size: $n of its three large files (example.proof, ivk/key.bin, ivk/leaf/0.bin) are in this tree: all three or none"; exit 1; fi )
verifier.record_altered_sets.world cpu setup.target.other_parameters_worlds.work any 1 0.5 0 ^record_altered_sets_test\.py: 7 calls on the honest set and its unaltered copy, 23 altered sets: all as expected$ ;; ^as expected +the honest set, the whole call: status 0: YES$ ;; ^as expected +its line result-v1 \(1 such line\(s\)\): the ID is the call's; status=0 bits=128 work=32 work-rounds=[1-9]\d* record=1$ ;; ^as expected +the honest set, the whole call with -no-proof-of-work: status 14: NOT-A-VERIFICATION$ ;; ^as expected +the unaltered copy, the whole call: status 0: YES$ ;; ^as expected +whole\.key-first-answer \(key\.bin: bit 0 of byte 40\): status 1: NO: model-statement$ ;; ^as expected +whole\.key-last-answer \(key\.bin: bit 0 of byte 6183\): status 1: NO: model-statement$ ;; ^as expected +whole\.leaf-first-answer \(leaf/[0-9]+\.bin: bit 0 of byte 40\): status 1: NO: entry-refused$ ;; ^as expected +whole\.leaf-last-answer \(leaf/[0-9]+\.bin: bit 0 of byte 6183\): status 1: NO: entry-refused$ ;; ^as expected +whole\.records-exchanged \(.*\): status 1: NO: model-statement$ ;; ^as expected +whole\.leaf-with-the-key-files-record \(.*\): status 1: NO: entry-refused$ ;; ^as expected +key\.records-exchanged \(.*\): status 1: NO: key-refused$ ;; ^as expected +leaf\.records-exchanged \(.*\): status 1: NO: entry-refused$ ;; ^as expected +whole\.key-after-the-record \(key\.bin: bit 0 of byte 6184\): status 1: NO: key-refused$ ;; ^as expected +whole\.key-middle \(.*\): status 1: NO: key-refused$ ;; ^as expected +whole\.leaf-middle \(.*\): status 1: NO: entry-refused$ ;; ^as expected +whole\.proof-middle \(.*\): status 1: NO: proof-refused$ D=${S:?}/other-parameters-f2-work/worlds/true && python3 -B verifier/tests/record_altered_sets_test.py ${B:?}/hc/test_build/cp_verify $D/config $D/ivk $D/in/example.req $D/out/example.y $D/out/example.proof $(cat $D/id.txt) ${W:?}/play
verifier.rejected_proofs cpu build.verifier.target any 1 0.5 0 ^10 altered sets: all rejected$ ;; ^the files as they are +exit code 0 +YES$ ;; ^the files as they are +exit code 0 +WITHIN LIMITS: clipped-gap C= ( mkdir -p ${W:?}; r=0; for x in accepted-proof accepted-proof-clipped-gap; do python3 -B -u examples/rejected-proofs/try_altered_copies.py ${S:?}/bin-target-f2/cp_verify ${W:?}/$x --example $x; s=$?; if [ $s = 3 ]; then r=3; fi; if [ $r = 0 ]; then r=$s; fi; done; if [ $r != 0 ]; then echo "verifier.rejected_proofs: not as expected: see the lines above"; fi; exit $r )
verifier.req_chain_test cpu - any 1 1.5 0 ^req_chain_test: ok \([1-9][0-9]* runs had to be refused\)$ mkdir -p ${W:?} && cd verifier/src && g++ -std=c++17 -O2 -Wall -Wextra -fno-ipa-modref -DBLK_FIELD=2 -DBLK_HASH=1 -I . -I ../../blocks ../tests/req_chain_test.cpp -o ${W:?}/req_chain_test && ${W:?}/req_chain_test
verifier.sanitized_files cpu setup.target.other_parameters_worlds any 8 60 0,77 ^(?!(?!sanitized_walk_test.sh: field 2: checking calls: no report, 4 of 4 controls reported: all as expected$)(?!not run: this machine's g\+\+ cannot build a program with the two sanitizers)) ;; ^(?!(?!extreme_proof_files_test\.py: 3 honest calls and [1-9]\d* altered files: refused [1-9]\d*, usage \d+; fault 0, not-checked 0, slow 0, honest calls not as expected 0, not refused 0, second encodings or their control not as asked 0$)(?!not run: this machine's g\+\+ cannot build a program with the two sanitizers)) ;; ^(?!(?!the control p - 1 in the proof: [1-9]\d*: NO: proof-refused [1-9]\d*$)(?!not run: this machine's g\+\+ cannot build a program with the two sanitizers)) ;; ^(?!(?!second encodings in the key: [1-9]\d*: NO: key-refused [1-9]\d*$)(?!not run: this machine's g\+\+ cannot build a program with the two sanitizers)) ;; ^(?!(?!second encodings in the entry: [1-9]\d*: NO: entry-refused [1-9]\d*$)(?!not run: this machine's g\+\+ cannot build a program with the two sanitizers)) ;; ^(?!(?!second encodings in the key's record: 4: NO: key-refused 4$)(?!not run: this machine's g\+\+ cannot build a program with the two sanitizers)) ;; ^(?!(?!second encodings in the entry's record: 4: NO: entry-refused 4$)(?!not run: this machine's g\+\+ cannot build a program with the two sanitizers)) verifier/tests/sanitized_walk_test.sh --field 2 --jobs 8 --world ${S:?}/other-parameters-f2/worlds/true ${W:?}
verifier.sanitized_walk cpu - any 8 60 0,77 ^(?!(?!sanitized_walk_test.sh: field 2: no report, 4 of 4 controls reported: all as expected$)(?!not run: this machine's g\+\+ cannot build a program with the two sanitizers)) ;; ^(?!(?!parameter_walk\.py: [1-9]\d* files; fault 0,)(?!not run: this machine's g\+\+ cannot build a program with the two sanitizers)) ;; ^(?!(?!extreme_files_test\.py: [1-9]\d* calls on [1-9]\d* files: planned [1-9]\d*, refused [1-9]\d*, usage \d+; fault 0, not-checked 0, slow 0, honest calls not as expected 0$)(?!not run: this machine's g\+\+ cannot build a program with the two sanitizers)) verifier/tests/sanitized_walk_test.sh --field 2 --jobs 8 --steps 20 ${W:?} prover/tests/configurations/pinned_no_proof_of_work
verifier.served.form cpu build.verifier.target any 1 0.4 0 ^served_form_test.py: all as expected$ python3 -B verifier/tests/served_form_test.py ${S:?}/bin-target-f2/cp_verify prover/tests/configurations/pinned_no_proof_of_work ${W:?}
verifier.served.form.world.kit cpu setup.target.other_parameters_worlds any 4 0.5 0 ^served_form_test.py: world kit: all as expected$ D=${S:?}/other-parameters-f2/worlds/true && python3 -B verifier/tests/served_form_test.py --world kit ${B:?}/hc/test_build/cp_verify $D/config $D/ivk $D/in/example.req $D/out/example.y $D/out/example.proof
verifier.what_a_yes_covers cpu setup.target.other_parameters_worlds any 1 0.7 0 ^what_a_yes_covers_test\.py: 72 calls, 0 not as expected$ ;; ^as expected +-no-proof-of-work: status 0, YES python3 -B verifier/tests/what_a_yes_covers_test.py ${B:?}/hc/test_build/cp_verify ${S:?}/other-parameters-f2/worlds/true ${W:?}/play
verifier.what_a_yes_covers.work cpu setup.target.other_parameters_worlds.work any 1 0.7 0 ^what_a_yes_covers_test\.py: 72 calls, 0 not as expected$ ;; ^as expected +-no-proof-of-work: status 14, NOT-A-VERIFICATION python3 -B verifier/tests/what_a_yes_covers_test.py ${B:?}/hc/test_build/cp_verify ${S:?}/other-parameters-f2-work/worlds/true ${W:?}/play
verifier.ws_build_fuzz cpu - any 1 1 0 ^ refused w-shape-mW: [1-9]\d*$ ;; ^ws_build_fuzz: 300000 shapes: [1-9]\d* built mkdir -p ${W:?} && cd verifier/src && g++ -O1 -g -fno-ipa-modref -D_GLIBCXX_ASSERTIONS -fsanitize=address,undefined -fno-sanitize-recover=all -std=c++17 -pthread -I . -I ../../blocks -I ../../prover -DBLK_FIELD=2 -DBLK_HASH=1 -DDIGEST_WORDS=8 -o ${W:?}/ws_build_fuzz ../tests/ws_build_fuzz.cpp && ${W:?}/ws_build_fuzz 300000 1

GPU programs and kernels

These rows run the GPU programs and kernels on synthetic data: a few run the committer and the prover on the small example configurations, and most test one kernel or sub-proof at small sizes against a CPU reference, the C++ verifier's own routines compiled into a checking program (verifier/tests/psc_layer_check.cpp, forest_lookup_check.cpp; not a run of cp_verify), or both. gpu.frame_readings commits on made-up weights at the eight small stored configuration folders (six different configurations: the two under examples/ are copies of two under prover/examples/) and compares what the prover's code and the verifier's code derive from the public files (prover/tests/frame_as_the_verifier_reads.cu), once as they are and once with one bound turned, which must be seen. gpu.wf_heavy_test checks that the GPU buffers its routines give back hold only zeros, except buffers that the code marks as holding only public values; no test checks that mark. gpu.attention_sumchecks_test and gpu.token_sumcheck_psc_test do not try a value altered inside the sent proof.

name kind needs word slots minutes passes command
gpu.attention_sumchecks_test gpu build.attention_sumchecks_test any 1 0.5 0 ^ok: 137 of 137 checks hold, at field 2$ cd prover && ${B:?}/attention_sumchecks_test
gpu.base_kernels_test gpu build.base_kernels_test any 1 0.5 0 ^0 of 11 checks failed$ prover/tests/unit.sh --field 2 run tests/base_kernels_test.cu ${S:?}/units.field2 && tail -n 1 ${S:?}/units.field2/base_kernels_test.log
gpu.coin_contexts.services.not_full gpu build.verifier,build.cp_commit,build.cp_prove any 1 1 0 ^ALL AS EXPECTED$ ;; ^as expected plain: 10 generators under 3 keys, \d+ draws: nothing is cried$ python3 -B prover/tests/coin_contexts_listed.py services ${B:?}/hc/test_build prover/tests/configurations/pinned_no_proof_of_work ${W:?}/play shape 7 3 2
gpu.coins_field_rule_test gpu build.coins_field_rule_test any 1 0.3 0 ^all 8 lines ok$ ;; ^refused words among 2\^35 \(about eight are expected\): found \(8, 0\): ok$ cd prover && ${S:?}/coins_field_rule_f2/coins_field_rule_test
gpu.compiler_sample gpu build.compiler_sample any 1 0.1 0 ^compiler_sample: 324 values are the same on the GPU and on the CPU; the kernel has \d+ bytes of local memory per thread$ ${B:?}/compiler_sample
gpu.cp_commit.small gpu-by-run build.verifier,build.cp_commit any 2 10 0 . ;; ^whole: exit status 0$ ;; ^again: exit status 0$ ;; ^the whole run, twice under one test's seed: SAME FILES$ ;; ^whole: the closing line: 1 says NOT to be handed out, 0 says can be handed out$ ;; ^key-alone: exit status 0$ ;; ^the whole run's key and the key alone: SAME KEY$ ;; ^the key alone: entries in ivk: 0$ ;; ^cp_verify -key: status 3: PART PASSED: key$ ;; ^cp_verify -leaf, copy 0: status 3: PART PASSED: leaf$ ;; ^cp_verify -leaf, copy 1: status 3: PART PASSED: leaf$ ;; ^the two copies' entries: OTHER ENTRIES$ ;; ^second-form: exit status 2$ ;; ^second-form: 1 line with its refusal's name; entries in ivk: 0$ ;; ^short: exit status 7$ ;; ^short: files written: 0$ ;; ^drawn-c: exit status 0$ ;; ^drawn-c: 1 line every bounded site fits; isk.txt: 0 line seed test, 1 line ranges; the seed's file: 600 32$ ;; ^drawn-d: exit status 0$ ;; ^drawn-d: 1 line every bounded site fits; isk.txt: 0 line seed test, 1 line ranges; the seed's file: 600 32$ ;; ^two runs under drawn seeds: key\.bin: OTHER BYTES$ ;; ^two runs under drawn seeds: leaf/0\.bin: OTHER BYTES$ ;; ^drawn-c: the closing line: 1 says NOT to be handed out, 0 says can be handed out$ ;; ^drawn-bits: exit status 0$ ;; ^drawn-bits: the closing line: 1 says NOT to be handed out, 0 says can be handed out$ ;; ^kept-test: exit status 0$ ;; ^kept-test: isk/audit \(700\): entry\.0\.fresh 600 32 1 zeros; entry\.1\.fresh 600 32 1 zeros; key\.fresh 600 32 1 zeros; $ ;; ^kept-test against whole, isk/audit apart: SAME FILES$ ;; ^kept-test: the closing line: 1 says NOT to be handed out, 0 says can be handed out$ ;; ^kept-drawn: exit status 0$ ;; ^kept-drawn: isk/audit \(700\): entry\.0\.fresh 600 32 1 other; entry\.1\.fresh 600 32 1 other; key\.fresh 600 32 1 other; different files among them: 3$ ;; ^kept-drawn: the closing line: 1 says NOT to be handed out, 0 says can be handed out$ ;; ^kept-drawn: the head's line on the option: 1$ ;; ^kept-part: exit status 2$ ;; ^kept-part: the option with -leaves: 1 usage text; folders made: 0$ ;; ^without the option: folders audit in the other runs' isk: 0$ ;; ^the two drawn seeds: DIFFER$ ;; ^refuse-there: exit status 2$ ;; ^refuse-there: 1 line with its refusal's name$ ;; ^after refuse-there: NOTHING HAS CHANGED$ ;; ^refuse-left-isk: exit status 2$ ;; ^refuse-left-ivk: exit status 2$ ;; ^refuse-a-working-name: exit status 2$ ;; ^refuse-one-name: exit status 2$ ;; ^one name for both: 1 line; folders made: 0$ ;; ^a left working name: 2 lines with the refusal's name; the foreign files kept: 2; folders made: 0$ ;; ^a given name that ends in \.making: 1 line; folders made: 0$ ;; ^working names left by all the runs above: 0; the head's line in the whole run: 1$ ;; ^refuse-budget: exit status 2$ ;; ^budget-edge: exit status 0$ ;; ^refuse-budget-beyond: exit status 2$ ;; ^the budget's edge: the greatest M is above 2 and below 4000000: yes; one more: 1 line with the refusal's name; folders made: 0$ ;; ^refuse-budget: 1 line with its refusal's name; folders made: 0$ ;; ^relative names are printed from the root: 1 line at the start of a making, 1 in the refusal at a left working name$ ;; ^a left working name that cp_commit has not made: lines that tell the reader to remove something: 0$ ;; ^table-short: exit status 7$ ;; ^table-short: the table's file: there, mode 600, with bytes$ ;; ^table-good: exit status 0$ ;; ^table-good: the table's file: there, mode 600, with bytes$ ;; ^table-there: exit status 2$ ;; ^table-there: 1 line with its refusal's name; the file that stood: THE SAME BYTES; folders left: 0$ GL="$GPU_RUN" prover/hash_commitment/tests/cp_commit_small.sh ${B:?}/hc/test_build ${W:?}/run 2
gpu.engine_gpu_test gpu build.engine_gpu_test any 1 3 0 ^engine_gpu_test: field 2, hash 1: 584 trees compared, 0 differences$ ;; ^engine_gpu_test: passes$ prover/tests/unit.sh --field 2 run engine/tests/engine_gpu_test.cu ${S:?}/engine.f2 && tail -n 1 ${S:?}/engine.f2/engine_gpu_test.log
gpu.engine_twins gpu build.engine_twins any 1 2 0 ^engine_twins: field 2: 4 of 4 pairs of proofs equal, 8 of 8 runs accepted$ prover/tests/engine_twins.sh run 2 ${S:?}/engine_twins.f2
gpu.forest_lookup_test gpu build.forest_lookup_test,build.npsc_checks any 1 1.5 0 ^forest_lookup_check: 10 of 10 groups passed$ ;; ^forest_lookup_test: 10 of 10 groups run and written$ ;; ^0 - 1 = 0xffffffff00000000$ mkdir -p ${W:?} && cd prover && ${S:?}/npsc.f2/forest_lookup_test ${W:?} && ${S:?}/npsc_checks/forest_lookup_check.f2 ${W:?}
gpu.frame_readings gpu-by-run build.verifier,build.cp_commit,build.frame_as_the_verifier_reads any 1 25 0 ^frame_readings: 8 configurations, all as expected$ prover/tests/frame_readings.sh ${B:?}/hc/test_build ${B:?}/hc/test_build ${W:?}/play examples/accepted-proof-clipped-gap/config examples/accepted-proof/config prover/examples/target_small_f2 prover/examples/target_small_f2_gap prover/examples/target_small_f2_roomy prover/tests/configurations/pinned prover/tests/configurations/pinned_no_proof_of_work prover/tests/configurations/pinned_no_proof_of_work_roomy
gpu.hf_code_test gpu build.hf_code_test any 1 2 0 ^0 of 247 checks failed$ prover/tests/unit.sh --field 2 run hash_commitment/tests/hf_code_test.cu ${S:?}/hf_code_test.field2 && tail -n 1 ${S:?}/hf_code_test.field2/hf_code_test.log
gpu.hf_lin_side_test gpu build.hf_lin_side_test any 1 6 0 ^hf_lin_side_test: all passed$ ;; ^ S7 played: the CPU keeper of the same rows, masks and salts$ cd prover && ${S:?}/f2/hc/test_build/tests/hf_lin_side_test 1 5 7 9
gpu.hf_link_side_test gpu build.hf_link_side_test any 2 1 0 ^hf_link_side_test: all passed$ ;; ^toy 1: ;; ^toy 5: ;; ^toy 7: ;; ^toy 9: ;; ^ CPU engine: [0-9]+ \+ [0-9]+ columns prover/tests/unit.sh --field 2 run hash_commitment/tests/hf_link_side_test.cu ${S:?}/hf_link_side_field2 '1 5 7 9' && cat ${S:?}/hf_link_side_field2/hf_link_side_test.log
gpu.last_run_test gpu build.last_run_test any 1 2 0 ^last_run_test: passes$ prover/tests/unit.sh --field 2 run engine/tests/last_run_test.cu ${S:?}/engine.f2
gpu.other_weights_nothing_leaves gpu build.verifier,build.cp_commit,build.cp_prove any 1 3 0 ^other_weights_nothing_leaves.sh: as foretold$ ;; ^same: status 0; new files: ./out/example.proof ./out/example.y ; gone: none; moved against the commitment's: ./isk/leaf/0/count.bin $ ;; ^other: status 1; new files: ./out/example.failed ; gone: none; moved against the commitment's: ./isk/isk.txt ./isk/leaf/0/count.bin $ prover/tests/other_weights_nothing_leaves.sh ${B:?}/hc/test_build kit:prover/tests/configurations/pinned_no_proof_of_work ${W:?}/play
gpu.psc_bytes_layer_test gpu build.psc_bytes_layer_test any 1 0.1 0 ^psc_bytes_layer_test: 16 of 16 cases passed \(640 scalars and as many mask entries changed one at a time\)$ cd prover && ${S:?}/psc_bytes_layer/psc_bytes_layer_test
gpu.psc_layer_test gpu build.psc_layer_test,build.npsc_checks any 1 0.1 0 ^psc_layer_check: 17 of 17 cases passed \(816 scalars changed one at a time\)$ ;; ^psc_layer_test: 17 of 17 cases passed \(816 scalars and as many mask entries changed one at a time\)$ mkdir -p ${W:?} && cd prover && ${S:?}/psc_layer/psc_layer_test ${W:?} && ${S:?}/npsc_checks/psc_layer_check.f2 ${W:?}
gpu.psc_layer_test.count gpu build.psc_layer_test.count any 1 0.1 0 ^psc_layer_test: 17 of 17 cases passed \(816 scalars and as many mask entries changed one at a time\)$ ;; ^psc_layer_test: the slots' counters were read: in 17 of 17 cases every slot was written once and a second writing was counted$ cd prover && ${S:?}/psc_layer.count/psc_layer_test
gpu.public_weights gpu-by-run build.verifier,build.cp_commit,build.cp_prove,build.serving,build.make_model any 1 25 0 ^ALL AS EXPECTED$ python3 -B prover/tests/public_weights.py ${B:?}/hc ${B:?}/hc/test_build ${W:?}/play prover/examples/target_small_file_f2 ${B:?}/make_model prover/examples/target_small_f2_roomy prover/examples/target_small_f2
gpu.record_on_public_weights gpu-by-run build.verifier,build.serving,kit.small_model any 1 5 0 ^record_on_public_weights, version H821: all as expected$ prover/tests/record_on_public_weights.sh ${B:?}/hc ${W:?}/play prover/examples/target_small_file_f2 ${S:?}/small_model
gpu.serving_first_run gpu build.verifier,build.cp_commit,build.serving any 1 5 0 ^serving_first_run: all as expected$ prover/tests/serving_first_run.sh ${B:?}/hc ${B:?}/hc/test_build ${W:?}/play prover/examples/target_small_f2_roomy
gpu.stream_coins_test gpu build.stream_coins_test any 1 1 0 ^stream_coins_test: passes$ prover/tests/unit.sh --field 2 run engine/tests/stream_coins_test.cu ${S:?}/engine.f2
gpu.token_sumcheck_psc_test gpu build.token_sumcheck_psc_test any 1 0.5 0 ^ok: 184 of 184 checks hold, at field 2$ cd prover && ${B:?}/token_sumcheck_psc_test
gpu.tree_arrays_cleared_test gpu build.tree_arrays_cleared_test any 1 0.5 0 ^tree_arrays_cleared_test: ok \(two trees ;; ^tree_arrays_cleared_test: passes$ prover/tests/unit.sh --field 2 run engine/tests/tree_arrays_cleared_test.cu ${S:?}/engine.f2 && tail -n 1 ${S:?}/engine.f2/tree_arrays_cleared_test.log
gpu.wf_heavy_test gpu build.wf_heavy_test any 1 1 0 ^wf_heavy_test: passes$ prover/tests/unit.sh --field 2 run engine/tests/wf_heavy_test.cu ${S:?}/wf_heavy_f2

The proof library's own tests

The proof library in blocks/ is what the three programs are built on. Its own unit tests run on the CPU, at toy sizes, on synthetic data, and call the library directly; inside a request's sub-proofs cp_prove and cp_verify run implementations of their own of the masked sum-check, the look-ups and the range check, about which these tests say nothing (prover/README.md; docs/limits-all-rows.tsv, row SO-12).

second_writings.honest gives toy files to the library's C++ verifiers and to second verifiers in Python. It changes the first OOD answer in a toy key file and in a toy leaf file. Both altered files are refused at a proof of work, because the answers are hashed at the start of the file's hash chain, so this test never puts a false answer before the equations that tie the OOD answers to the table (Parts 9 and 10 of docs/format/toys/3-key-and-entry.md: the equations (Rec_i) and the record's rows). On synthetic data, in a run of the project's own, the second verifier of key files refused a false key file that is not part of this repository, so that a reader cannot repeat it, at the one batched linear check that holds these equations and those on the opened columns. By a reading of that verifier's code and of the outcome, its false pad had made one of these equations false, together with those on the opened columns. Such a check cannot say which equation decided, so that file does not show these equations refusing by themselves.

name kind needs word slots minutes passes command
blocks.tests cpu - any 8 13 0 ^run_new\.py: passed: tests$ ;; ^tests . passed . 89 tests ran \(89 sources, as listed in .*/blocks/tests/TESTS, 1 settings\) python3 -B tests/new/run_new.py ${T:?} ${W:?} --jobs 8 --settings f2h1 --only tests --tests blocks/tests/TESTS
new.cp_plan_files cpu - any 1 0.5 0 ^cp_plan_files\.py: 11 files: all as expected$ python3 -B blocks/numbers/tests/cp_plan_files.py ${W:?}/play
second_writings.honest cpu - any 4 5 0 ^ALL AS EXPECTED: 32 honest files accepted; 255 altered copies and 2 bent entries refused; each judged by two verifiers$ ;; ^bytes: all 409 proofs, files, parameters and public files have the sha256 that expected/sha256\.txt gives them$ bash tests/new/second/run_honest.sh ${W:?} 4

Checks of safeguards

These rows check safeguards, on the CPU, each in a folder of its own: the table of calls that end processes or delete recursively, the guarding lines in the programs' sources, the copies of the routine that refuses a work folder below a place that another account can change, cp_commit's file routines, the supervisor of the processes that cp_prove and cp_commit start, the routine that pins a thread near a GPU, what the tool that writes synthetic models removes after a failure, the routine behind cp_prove -clear, the runner itself, its sandbox, tools/confine.py, and the names under which cp_verify's NOT CHECKED may be the machine's fault.

name kind needs word slots minutes passes command
commit.exchanged_directories cpu - any 1 0.2 0,77 ^(?!(?!.*?(?:: passes$))(?!^not run: [A-Za-z0-9_]+: this CPU cannot run the SHA instructions: the test died with SIGILL, and so did a one-instruction program that uses sha256rnds2; run it on a CPU that can$)) prover/tests/unit.sh --field 2 both hash_commitment/tests/exchanged_directories_test.cpp ${W:?} "${W:?}/made"
commit.walked_parents cpu - any 1 0.2 0,77 ^walked_parents_test: (?=.)(?:passes)?(?:cannot be run on this machine: .*)?$ prover/tests/unit.sh --field 2 both hash_commitment/tests/walked_parents_test.cpp ${W:?} "${W:?}/made"
prover.gate_children cpu - any 1 1 0 ^ALL AS EXPECTED$ ;; ^cases as expected: 67; not: 0$ ;; ^as expected control: a parent that starts the script directly, killed while the children run: 4 processes still run two seconds later ;; ^as expected parent killed while the children run: no process is left$ ;; ^as expected as cp_commit: control: a parent that starts the script directly, killed while the children run: 4 processes still run two seconds later ;; ^as expected as cp_commit: parent killed while the children run: no process is left$ prover/tests/unit.sh --field 2 build tests/gate_children_parent.cpp ${W:?} && python3 -B prover/tests/gate_children.py ${W:?}/gate_children_parent ${W:?}/work
prover.guards_stand cpu - any 1 0.3 0 ^ALL AS EXPECTED$ ;; ^47 guards pinned in 25 files; 329 of 329 teeth bite; 8 places that make a stream, and 3 of 3 controls on them bite$ cd prover && python3 -B tests/guards_stand.py
prover.numa_pin_test cpu - any 1 0.1 0 ^numa_pin_test: 12 cases: passes$ mkdir -p ${W:?} && g++ -O2 -fno-ipa-modref -std=c++17 -Wall -Wextra -ftrivial-auto-var-init=pattern -I prover/tests/numa_pin_stand_in -I prover -o ${W:?}/numa_pin_test prover/tests/numa_pin_test.cpp && ${W:?}/numa_pin_test
prover.make_model_failures_test cpu - any 1 0.1 0 ^make_model_failures_test: 6 scenes: passes$ mkdir -p ${W:?} && g++ -O2 -fno-ipa-modref -std=c++17 -Wall -Wextra -pthread -I prover -o ${W:?}/make_model_failures_test prover/tests/make_model_failures_test.cpp && ${W:?}/make_model_failures_test ${W:?}/play
prover.spool_clear_test cpu - any 1 0.3 0,77 ^(?!(?!.*?(?:: passes$))(?!^not run: [A-Za-z0-9_]+: this CPU cannot run the SHA instructions: the test died with SIGILL, and so did a one-instruction program that uses sha256rnds2; run it on a CPU that can$)) prover/tests/unit.sh --field 2 both tests/spool_clear_test.cpp ${W:?}
sources.dangerous_calls cpu - any 1 0.3 0 ^dangerous_calls_test\.py: 38 samples as expected, 0 finding\(s\)(?:; 1 line of the table was not compared, because this tree has no lean/)?(?:(?<=\)); (?!1 )[1-9]\d* lines of the table were not compared, because this tree has no lean/)?$ python3 -B tests/dangerous_calls_test.py
sources.folder_look_copies cpu - any 1 0.1 0 ^folder_look_copies_test\.py: [1-9][0-9]* copies of open_above\(\), others_can_change\(\) and available_memory\(\), all unchanged$ python3 -B tests/folder_look_copies_test.py
sources.run_suite cpu - any 1 3 0 ^run_suite_test.py: 65 checks as expected, 0 otherwise$ python3 -B run_suite_test.py ${W:?}
tools.confine cpu - any 1 0.1 0,77 ^confine_test: \d+ checks as expected, 0 otherwise python3 -B tools/confine_test.py ${W:?}
verifier.not_checked_names cpu - any 1 0.1 0 ^ALL AS EXPECTED$ python3 -B verifier/tests/not_checked_names.py

The script that measures the rate of proving

This row tests driver/measure_rate.py, with which a reader can measure the prover's speed on a machine of their own. It runs on the CPU, in temporary folders of its own, against stand-ins, and starts none of cp_commit, cp_prove, cp_verify or a real nvidia-smi.

name kind needs word slots minutes passes command
driver.measure_rate cpu - any 2 1.2 0,77 ^(?!(?!OK$)(?!stood aside: below an open place: )) ;; ^(?!(?!Ran 50 tests? in)(?!stood aside: below an open place: )) python3 -B driver/tests/test_measure_rate.py

The integer model for PyTorch and vLLM

These rows run the tests of the scripts around vLLM (integrations/tests) and of the integer model's Python package (integrations/vllm/integer_model) that need no GPU and no vLLM. The integer model's row needs its own Python environment, made from integrations/vllm/requirements.txt and integrations/vllm/integer_model/tests/requirements.txt, and only ./run_suite.py long runs it.

name kind needs word slots minutes passes command
integer_model.cpu long - any 4 13 0,77 ^(?=.)(?:\d+ passed(?:, [1-4] skipped)?, \d+ deselected.* in \d.*)?(?:not run: .*)?(?:stood aside: below an open place: .*)?$ ( if python3 -c 'import pytest, torch, numpy, safetensors, transformers, gguf' 2>/dev/null; then mkdir -p ${W:?} && P=$(mktemp -d "${W:?}/pytest.XXXXXX") && cd integrations/vllm/integer_model && PYTHONDONTWRITEBYTECODE=1 python3 -m pytest -q -p no:cacheprovider --basetemp="$P/base" -m "not vllm and not prover"; else echo 'not run: python3 lacks pytest, torch, numpy, safetensors, transformers or gguf here'; exit 77; fi )
integrations.cp_verify cpu build.verifier any 4 6 0,77 ^(?=.)(?:68 passed in \d.*)?(?:full-size(?:, full-size-clipped-gap)?: not run: the set is not in this tree)?(?:full-size-clipped-gap: not run: the set is not in this tree)?(?:not run: .*)?(?:stood aside: below an open place: .*)?$ ;; ^(?:68 passed in \d)?(?:67 passed, 1 deselected in \d)?(?:66 passed, 2 deselected in \d)?(?:not run: )?(?:stood aside: below an open place: )?(?<=.) ( if python3 -c 'import pytest' 2>/dev/null; then mkdir -p ${W:?} && P=$(mktemp -d "${W:?}/pytest.XXXXXX") && k='' && s='' && for e in full-size full-size-clipped-gap; do n=0 && for f in example.proof ivk/key.bin ivk/leaf/0.bin; do if [ -e examples/$e/$f ]; then n=$((n+1)); fi; done && if [ $n != 3 ] && [ $n != 0 ]; then echo "examples/$e holds $n of its three large files (example.proof, ivk/key.bin, ivk/leaf/0.bin): all three or none"; exit 1; fi && if [ $n = 0 ]; then if [ $e = full-size ]; then t=test_full_size_example_is_verified; else t=test_full_size_clipped_gap_example_passes_within_its_own_limits; fi && k="${k:+$k and }not $t" && s="${s:+$s, }$e"; fi; done && if [ -n "$k" ]; then python3 -B -m pytest -q -p no:cacheprovider --basetemp="$P/base" -k "$k" integrations/tests/test_with_cp_verify.py integrations/tests/test_false_files_with_cp_verify.py && echo "$s: not run: the set is not in this tree" && exit 77; else python3 -B -m pytest -q -p no:cacheprovider --basetemp="$P/base" integrations/tests/test_with_cp_verify.py integrations/tests/test_false_files_with_cp_verify.py; fi; else echo 'not run: python3 lacks pytest here'; exit 77; fi )
integrations.unit cpu - any 2 8 0,77 ^(?=.)(?:\d{4,} passed(?:, (?!0)(?!2[1-9]\b)(?![3-9]\d\b)\d{1,2} skipped)?(?:, [1-9]\d* deselected)? in \d.*)?(?:not run: .*)?(?:stood aside: below an open place: .*)?$ ( if python3 -c 'import pytest' 2>/dev/null; then mkdir -p ${W:?} && P=$(mktemp -d "${W:?}/pytest.XXXXXX") && python3 -B -m pytest -q -p no:cacheprovider --basetemp="$P/base" -m "not root_only" --ignore=integrations/tests/test_with_cp_verify.py --ignore=integrations/tests/test_false_files_with_cp_verify.py integrations/tests; else echo 'not run: python3 lacks pytest here'; exit 77; fi )