Skip to content

perf(formatter): render compact SELECT/INSERT/UPDATE/DELETE through SQL() [W5-02] - #904

Merged
ajitpratap0 merged 8 commits into
mainfrom
perf/w5-02-compact-sql-fastpath
Nov 2, 2026
Merged

ajitpratap0 merged 8 commits into
mainfrom
perf/w5-02-compact-sql-fastpath

Conversation

@ajitpratap0

@ajitpratap0 ajitpratap0 commented Oct 31, 2026 •

Copy link
Copy Markdown
Owner

Summary

Wave-1 task W5-02 (FMT-1) from the 2026-11 performance audit. The prototype was confirmed by an independent verifier; Codex ported it, and Opus reviewed it and ran the full gate.

Compact formatting used to walk each statement with the visitor renderer, which builds many intermediate strings, even though each statement's SQL() already produces the same compact text.

  • Fast path. formatStatement now returns SQL() for SelectStatement, InsertStatement, UpdateStatement and DeleteStatement when all of these hold:
    • no clause newlines
    • no semicolon
    • no line width
    • no target dialect
    • keyword case is preserve or upper
  • Stays on the visitor renderer (now formatStatementSlow):
    • lowercase keywords
    • target-dialect rewrites
    • SetOperation, whose SQL() prints () for a nil side
    • AlterTable, whose SQL() drops commas (fixed separately in W5-02b)
  • Single-statement shortcut. For one statement with no comments and no Oracle execute slash, formatAST skips the parts slice and join. It still applies Unicode-identifier restoration and the statement terminator. This matches the general loop for one statement, including batch separators.

Test Plan

  • TestCompactFastDifferential. It compares the fast path with the slow path, forced by setting a line width because the visitor ignores line width.
    • Inputs: every .sql file under testdata/ (259 files: 89 realworld, 40 corpus, 130 other), parsed in each of 9 dialects, whole-file and split on ;\n.
    • Options: keyword case preserve, upper and lower; every target dialect; three indent settings.
    • Result on head ada67d9: files=259, statements=27289, comparisons=491202, fast_eligible=116040, 0 mismatches. fast_eligible counts only comparisons where the statement is one of the four allowlisted types, the dialect is empty and keyword case is not lower (the test fails if it is 0).
  • TestCompactFastOptions covers option combinations, including a newline_per_clause case (NewlinePerClause without AddSemicolon), so the formatter package alone catches removal of that guard (mutation-checked).
  • TestCompactFastFormatASTSingleStatement and TestCompactFastFormatASTHandBuiltComments guard the single-statement formatAST shortcut (terminator, execute slash, hand-built comments).
  • Nested statements. An extra review run forced nested rendering onto the slow path for the SQL literals in existing tests: 50,062 comparisons, no mismatches.
  • Gate (run outside the Codex sandbox):
Check Result
go run honnef.co/go/tools/cmd/staticcheck@latest ./pkg/formatter/... ✅
task check (includes W0-02 goldens, unchanged) ✅
task test:cbinding ✅
go test -count=1 ./pkg/compatibility/... ✅

Re-measured after merging wave 0 (origin/main 160c417). Local scripts/perf/ab.sh (same harness as task bench:ab): Apple M4, GOGC=off, -cpu 1, 10 interleaved base/head rounds, benchstat alpha 0.01, benchmark names from the wave-0 matrix (PLAN 2.2). The branch's own duplicate BenchmarkRender was dropped in favour of the matrix one. W0-02 goldens and the full task check pass with no golden change. Machine is shared, so numbers are indicative; the CI gate below is authoritative. Raw output: scratchpad/perf/wave1/w5-02/ab.txt.

Benchmark time B/op allocs/op
Render/compact/* (11) -27.2% to -49.0%, all p<0.001 -60% to -82% (geomean -75%) 4-42 -> 1
Fingerprint/* (11) -6.7% to -12.2% -7% to -16% -15% to -39%
Normalize/* (11) -6.0% to -13.2% -8% to -19% similar
geomean over the 33 -20.9%

The plan target was Render/compact/* <= -30% with 1 alloc and Fingerprint/complex <= -5%. Ten of 11 compact queries clear -30%; select_where reaches -27.2% (reported, not chased). Fingerprint/complex is -10.2%. Render/sql/* is unchanged (control).

CI perf gate (shadow, amd64, authoritative): PASS (table below is from head f2e016f; later commits only change tests, the gate on a237070/ada67d9d is noted at the end of this section) (check "A/B perf gate (shadow)" plus the sticky PR comment; A/A control: 0 of 32 benchmarks moved more than 3%).

  • Gated Normalize/simple_select -10.80%, Normalize/join -15.18%, Normalize/complex -15.21% (all p<0.001), B/op -14% to -17%, allocs 10->7, 40->23, 110->69.
  • Gated suite geomean -1.51% (p<0.001). No gated benchmark regressed >= 3%; allocs/op and B/op did not rise; instructions/op within +/-0.13%.
  • Gate on head ada67d9 (latest; the run on a237070 was cancelled by this push): PASS. Normalize/simple_select -11.83%, Normalize/join -16.75%, Normalize/complex -16.33% (all p<0.001), B/op -13.9% to -17.2%, allocs 10->7, 40->23, 110->69; suite geomean -1.63% (p<0.001); A/A control within +/-0.6%; no gated regression.
  • The default gated suite does not include Render/compact/*, so that target is covered by the local table above only (arm64, indicative); the amd64 evidence is the Normalize gain, which exercises the same fast path.

Breaking Changes

None. The output is byte-identical (see the differential test).

🤖 Generated with Claude Code

https://claude.ai/code/session_01DnbRnTFErhoMJdAjVEeCg2

…QL() [W5-02]

The compact formatter walked every statement with the visitor renderer,
building many intermediate strings, although the statements' SQL() methods
already produce the same compact spelling. formatStatement now returns
SQL() for SelectStatement, InsertStatement, UpdateStatement and
DeleteStatement when the options are compact (no clause newlines, no
semicolon, no line width, no target dialect, keyword case preserve or
upper). Lowercase keywords, target-dialect rewrites, SetOperation (nil
sides print "()" in SQL()) and AlterTable (SQL() drops commas) stay on the
visitor renderer, kept as formatStatementSlow. formatAST also skips the
parts slice and join for a single statement without comments or an
execute slash, still applying Unicode-identifier restoration and the
statement terminator.

Tests: a differential test compares the fast and slow paths over every
statement in the real-world corpora (27,383 statements parsed in each of
9 dialects) for preserve/upper/lower keyword case, every target dialect
and three indent settings: 492,894 comparisons, 0 mismatches.

M4, GOGC=off -cpu 1, 10 interleaved rounds, the 11 compare queries:
Render/compact -27.5% to -47.9% (geomean -39.3%), B/op -75%, allocs/op 1
for every query (-90%); Normalize -8.0% and Fingerprint -7.1% time,
allocs -37% to -39%.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DnbRnTFErhoMJdAjVEeCg2
@vercel

vercel Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
gosqlx-website Ready Ready Preview Nov 2, 2026 10:06am UTC

@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Claude finished @ajitpratap0's task in 21s —— View job


Review of PR #904: compact SQL() fast path

Verdict: looks good. I found no blocking issues.

I read the diff and compared the new single-statement shortcut in formatAST against the general loop. I did not run the tests or benchmarks. The numbers below are the ones in the PR description.

Correctness

  • Single-statement shortcut. With one statement, no comments and no execute slash, the general loop reduces to this:

    • RestoreUnicodeIdentifiers.
    • EnsureStatementTerminator.
    • strings.Join of one element, which is a no-op.

    The hasBatch branch also does nothing for one element, so a single BatchSeparator takes the same route. The shortcut matches the loop.

  • Fast-path guard in formatStatement. It is conservative. Lowercase keywords, any dialect rewrite, AddSemicolon, line width and clause newlines all fall through to formatStatementSlow. SetOperation and AlterTable are correctly excluded, for the reasons given in the PR.

  • AddSemicolon. The guard in formatStatement excludes it, but the formatAST shortcut still applies it afterwards through EnsureStatementTerminator. That is consistent. The semicolon test option covers this case.

Risks and suggestions (all non-blocking)

  1. Hidden coupling. The fast path relies on SQL() and the visitor renderer producing the same text. Future changes to either one could make them diverge without anyone noticing. TestCompactFastPathDifferential guards this well, but it reads testdata/realworld and testdata/corpus. Check that it still runs in CI and is not skipped under -short. A short comment at the formatStatement switch pointing to that test would help future editors.
  2. Nested statements. The PR says nested statements were checked once by a manual forced-slow-path run. That check is not in the repo. It may be worth making it a permanent test, for example subqueries and CTEs rendered through SQL().
  3. Typed nil statements. TestCompactFastNil covers typed-nil Select, Insert, Update and Delete statements. That depends on each SQL() method being nil-safe, so it is a good test to keep.
  4. Redundant check. The !opts.NewlinePerClause test in formatAST is already implied by the guard in formatStatement. It is harmless, but it prevents the shortcut from applying to single statements in non-compact styles. Those take the general path anyway, so the shortcut would gain nothing there.
  5. Benchmark gap. select_where reaches −27.5%, below the −30% target. The geomean is −39.3%, so this seems acceptable. It may be worth noting in the PR that the target was met in aggregate, not for every query.
  6. Merge ordering. The PR is marked wave-1 and should merge only after the W0-02 goldens and the W0-09 perf gate are on main. Please confirm both are in before merging.

Security: No concerns. The change adds no new input handling or I/O.

Test coverage: Strong. Beyond the differential and option-matrix tests, there are Unicode-identifier and nil tests, plus benchmarks.

@ajitpratap0
ajitpratap0 marked this pull request as ready for review November 1, 2026 18:46
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Nov 1, 2026
@github-actions

github-actions Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Claude finished @ajitpratap0's task in 20s —— View job


Review of PR #904: compact SQL() fast path

Verdict: looks good, with no blocking issues. I read the diff and the differential test. I did not run the tests or benchmarks, so the performance numbers are the ones in the PR description.

Correctness

  • Guard in formatStatement (pkg/formatter/render.go, around line 216). It is conservative. Every option that changes output falls through to formatStatementSlow: clause newlines, semicolon, line width, dialect and lowercase keywords. SetOperation and AlterTable stay on the slow path for the stated reasons.
  • Single-statement shortcut in formatAST. With one statement, no comments and no execute slash, it applies the same unicode restoration and termination as the general loop. strings.Join of one element is a no-op. This is correct.
  • Nil handling. formatStatement now calls stmtSQL(s) only inside a type switch. A nil interface never matches a case, so it reaches formatStatementSlow, which handles nil. Typed-nil statements are covered by TestCompactFastNil.

Suggestions (non-blocking)

  1. Test name mismatch. The PR description refers to TestCompactFastPathDifferential, but the test is TestCompactFastDifferential (compact_differential_test.go:23). Update the description or the name so anyone searching for it finds it. The test has no testing.Short() skip, so it also runs in the pre-commit hook's -short run. That is fine if the runtime is acceptable. It parses every fixture in 9 dialects, so please check how long it takes. A -short skip would be reasonable if it is slow.
  2. Redundant comparisons. When target is string(dialect), opts.Dialect != "", so formatStatement takes the slow path. That comparison is slow against slow and checks nothing. The same applies to KeywordLower. Only about 1/3 of the 492,894 comparisons exercise the fast path. Narrowing the matrix would cut runtime without losing coverage. Alternatively, count the comparisons where the fast path was actually taken and assert that the count is greater than zero, so a future guard change cannot make the test vacuous.
  3. Nested statements. The nested forced-slow-path check was a one-off run and is not in the repo. Here LineWidth = 80 keeps descendants on the slow path, but nothing checks that subqueries and CTEs inside a fast-path statement render identically. Consider a small table test with a subquery, CTE and INSERT … SELECT, comparing formatStatement against formatStatementSlow. In that test the nested statements should also be rendered through the visitor.
  4. Coupling comment. Add a one-line comment at the formatStatement switch pointing to TestCompactFastDifferential. It should say that any change to a SQL() method for these four statement types must keep that test green.
  5. Redundant check. !opts.NewlinePerClause in the formatAST shortcut is harmless but not needed for correctness. Non-compact styles also take the general path today. The check keeps the shortcut narrow, so I would leave it.
  6. Benchmark note. select_where reaches −27.5%, below the −30% target, while the geomean is −39.3%. The PR already says this, so no change is needed.
  7. Merge ordering. Confirm that the W0-02 goldens and the W0-09 perf gate are on main before merging, as the PR states.

Security: No concerns. The change adds no new input handling or I/O.

Test coverage: Strong. It has the differential matrix, the nil test, and the unicode-identifier and terminator paths.

@github-actions

github-actions Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Perf gate: PASS (shadow mode: report only, this check never fails)

Base 5407cc0dfa0e, head f3826e5aa319. 10 interleaved rounds, benchtime 200ms, -cpu 1, pinned to CPU 3 (SMT siblings 2-3) with taskset.

  • 3 benchmark(s) faster by 3% or more

Rules: time regresses at >= +3% with p < 0.01 (Mann-Whitney U), or a suite geomean >= +1.5% (per-round geomeans, same test); allocs/op must not increase; B/op must not rise more than 2%; instructions/op must not rise more than 1.5%. An A/A control with more than 10% of benchmarks moving over 3% is noisy: re-run once, then inconclusive (neutral).

Gated suite (GOGC=off, GOMAXPROCS=1)

benchmark time p A/A B/op allocs/op status
ParseE2E/simple_select +0.55% <0.001 +0.10% +0.00% 5 → 5 ok
ParseE2E/select_where +0.70% 0.023 +0.36% +0.00% 18 → 18 ok
ParseE2E/join -0.35% 0.472 +0.14% +0.00% 17 → 17 ok
ParseE2E/aggregate +0.68% 0.008 +0.30% +0.00% 18 → 18 ok
ParseE2E/subquery +1.34% <0.001 +0.30% +0.00% 14 → 14 ok
ParseE2E/cte +0.65% <0.001 +0.35% +0.00% 21 → 21 ok
ParseE2E/window +0.24% 0.173 +0.16% +0.00% 11 → 11 ok
ParseE2E/complex -0.64% 0.011 +0.24% +0.00% 50 → 50 ok
ParseE2E/insert -0.20% 0.028 +0.20% +0.00% 17 → 17 ok
ParseE2E/update -0.59% 0.404 -0.20% +0.00% 13 → 13 ok
ParseE2E/delete -0.47% 0.003 +0.05% +0.00% 5 → 5 ok
ParseRW/postgresql/medium +0.09% 0.393 -0.06% +0.00% 646 → 646 ok
ParseRW/mysql/medium +0.23% 0.315 -0.00% +0.00% 365 → 365 ok
ParseRW/mariadb/medium +0.28% 0.739 +0.01% +0.00% 523 → 523 ok
ParseRW/sqlserver/medium +0.11% 0.353 +0.00% +0.00% 936 → 936 ok
ParseRW/oracle/medium -0.22% 0.075 +0.12% +0.00% 1093 → 1093 ok
ParseRW/sqlite/medium -0.32% 0.496 -0.35% +0.00% 346 → 346 ok
ParseRW/snowflake/medium -0.12% 0.075 -0.09% +0.00% 777 → 777 ok
ParseRW/clickhouse/medium -0.62% 0.029 -0.37% +0.00% 384 → 384 ok
ParseFresh/simple_select +1.17% <0.001 -0.04% +0.00% 6 → 6 ok
ParseFresh/join +1.13% 0.002 +0.31% +0.00% 18 → 18 ok
ParseFresh/complex +1.21% <0.001 +0.57% +0.00% 51 → 51 ok
PoolGetPut/serial -0.01% 0.940 -0.04% +0.00% 0 → 0 ok
Tokenize/scratch/join -0.67% 0.003 +0.26% +0.00% 0 → 0 ok
Tokenize/scratch/complex -0.49% 0.006 -0.10% +0.00% 0 → 0 ok
Tokenize/scratch/insert -0.16% 0.545 +0.02% +0.00% 0 → 0 ok
Normalize/simple_select -12.88% <0.001 -0.29% -13.85% 10 → 7 faster, -allocs
Normalize/join -15.26% <0.001 -0.21% -17.15% 40 → 23 faster, -allocs
Normalize/complex -11.61% <0.001 -0.03% -16.24% 110 → 69 faster, -allocs
TransformParse/simple_select +0.00% 0.380 -0.03% +0.00% 5 → 5 ok
TransformParse/join +0.24% 0.623 +0.22% +0.00% 17 → 17 ok
TransformParse/complex +0.40% 0.165 -0.05% +0.00% 50 → 50 ok
geomean -1.20% <0.001 +0.06%

A/A control: 0 of 32 benchmarks moved more than 3%.

Instructions per operation (cmd/perfdriver)

case instructions/op status
e2e/aggregate +0.04% ok
e2e/complex -0.01% ok
e2e/cte +0.01% ok
e2e/delete +0.00% ok
e2e/insert +0.01% ok
e2e/join -0.01% ok
e2e/select_where +0.01% ok
e2e/simple_select +0.04% ok
e2e/subquery +0.04% ok
e2e/update -0.02% ok
e2e/window +0.07% ok
rw/postgresql/medium +0.04% ok
Secondary metric: default GC, GOMAXPROCS=1 (informational)
benchmark time p
ParseE2E/simple_select +0.24% 0.211
ParseE2E/select_where +1.04% 0.031
ParseE2E/join -0.06% 0.650
ParseE2E/aggregate +0.16% 0.739
ParseE2E/subquery +0.97% 0.005
ParseE2E/cte +0.46% 0.218
ParseE2E/window +0.29% 0.190
ParseE2E/complex -0.26% 0.075
ParseE2E/insert -0.67% 0.069
ParseE2E/update -0.65% 0.104
ParseE2E/delete -1.10% 0.471
ParseRW/postgresql/medium -0.75% 0.089
ParseRW/mysql/medium -0.01% 0.684
ParseRW/mariadb/medium +0.51% 0.105
ParseRW/sqlserver/medium +0.49% 0.280
ParseRW/oracle/medium -0.19% 0.481
ParseRW/sqlite/medium +0.22% 0.529
ParseRW/snowflake/medium -0.08% 0.436
ParseRW/clickhouse/medium +0.06% 0.496
ParseFresh/simple_select +1.14% 0.001
ParseFresh/join +0.64% <0.001
ParseFresh/complex +0.59% 0.010
PoolGetPut/serial -0.01% 0.818
Tokenize/scratch/join -0.54% <0.001
Tokenize/scratch/complex -0.72% 0.007
Tokenize/scratch/insert -0.09% 0.104
Normalize/simple_select -12.77% <0.001
Normalize/join -14.21% <0.001
Normalize/complex -11.30% <0.001
TransformParse/simple_select +0.13% 0.790
TransformParse/join -0.06% 0.791
TransformParse/complex -0.04% 0.684
geomean -1.22%
Environment
date:        2026-11-02T10:07:06Z
base:        5407cc0dfa0eed4034c385c29c29e709f842830d
head:        working tree of f3826e5aa319e45cd0afeb6bb98c6a6b80076a05
go:          go version go1.26.1 linux/amd64
kernel:      Linux 7.0.0-1017-azure x86_64
pinning:     pinned to CPU 3 (SMT siblings 2-3) with taskset
clocksource: tsc
load:        2.05, 0.58, 0.20
instr count: cachegrind
perf probe:  Error: No supported events found. The instructions:u event is not supported. 

Architecture:                            x86_64
CPU(s):                                  4
Model name:                              AMD EPYC 9V74 80-Core Processor
Thread(s) per core:                      2
Core(s) per socket:                      2
Socket(s):                               1
Hypervisor vendor:                       Microsoft
L1d cache:                               64 KiB (2 instances)
L1i cache:                               64 KiB (2 instances)
L2 cache:                                2 MiB (2 instances)
L3 cache:                                32 MiB (1 instance)
benchstat (primary metric)
goos: linux
goarch: amd64
pkg: github.com/ajitpratap0/GoSQLX/pkg/fingerprint
cpu: AMD EPYC 9V74 80-Core Processor                
                        │    base     │                head                 │
                        │   sec/op    │   sec/op     vs base                │
Normalize/simple_select   2.263µ ± 1%   1.972µ ± 1%  -12.88% (p=0.000 n=10)
Normalize/join            7.800µ ± 2%   6.610µ ± 1%  -15.26% (p=0.000 n=10)
Normalize/complex         19.10µ ± 1%   16.88µ ± 0%  -11.61% (p=0.000 n=10)
geomean                   6.960µ        6.037µ       -13.27%

                        │     base     │                 head                 │
                        │     B/op     │     B/op      vs base                │
Normalize/simple_select    1040.0 ± 0%     896.0 ± 0%  -13.85% (p=0.000 n=10)
Normalize/join            3.234Ki ± 0%   2.680Ki ± 0%  -17.15% (p=0.000 n=10)
Normalize/complex         8.945Ki ± 0%   7.492Ki ± 0%  -16.24% (p=0.000 n=10)
geomean                   3.086Ki        2.600Ki       -15.76%

                        │    base     │                head                │
                        │  allocs/op  │ allocs/op   vs base                │
Normalize/simple_select   10.000 ± 0%   7.000 ± 0%  -30.00% (p=0.000 n=10)
Normalize/join             40.00 ± 0%   23.00 ± 0%  -42.50% (p=0.000 n=10)
Normalize/complex         110.00 ± 0%   69.00 ± 0%  -37.27% (p=0.000 n=10)
geomean                    35.30        22.31       -36.80%

pkg: github.com/ajitpratap0/GoSQLX/pkg/gosqlx
                          │    base     │                head                │
                          │   sec/op    │   sec/op     vs base               │
ParseE2E/simple_select      1.443µ ± 0%   1.451µ ± 0%  +0.55% (p=0.000 n=10)
ParseE2E/select_where       2.909µ ± 1%   2.929µ ± 0%       ~ (p=0.020 n=10)
ParseE2E/join               4.443µ ± 0%   4.427µ ± 2%       ~ (p=0.469 n=10)
ParseE2E/aggregate          3.214µ ± 1%   3.236µ ± 2%  +0.68% (p=0.006 n=10)
ParseE2E/subquery           2.834µ ± 0%   2.872µ ± 2%  +1.34% (p=0.000 n=10)
ParseE2E/cte                3.867µ ± 0%   3.892µ ± 1%  +0.65% (p=0.000 n=10)
ParseE2E/window             2.530µ ± 0%   2.536µ ± 2%       ~ (p=0.171 n=10)
ParseE2E/complex            11.87µ ± 1%   11.79µ ± 1%       ~ (p=0.011 n=10)
ParseE2E/insert             1.967µ ± 0%   1.963µ ± 0%       ~ (p=0.024 n=10)
ParseE2E/update             1.794µ ± 1%   1.783µ ± 2%       ~ (p=0.402 n=10)
ParseE2E/delete             1.072µ ± 1%   1.067µ ± 0%  -0.47% (p=0.001 n=10)
ParseRW/postgresql/medium   143.3µ ± 0%   143.4µ ± 0%       ~ (p=0.393 n=10)
ParseRW/mysql/medium        76.91µ ± 0%   77.09µ ± 0%       ~ (p=0.315 n=10)
ParseRW/mariadb/medium      110.6µ ± 1%   110.9µ ± 1%       ~ (p=0.739 n=10)
ParseRW/sqlserver/medium    210.8µ ± 0%   211.0µ ± 0%       ~ (p=0.353 n=10)
ParseRW/oracle/medium       258.3µ ± 0%   257.7µ ± 1%       ~ (p=0.075 n=10)
ParseRW/sqlite/medium       67.09µ ± 0%   66.88µ ± 0%       ~ (p=0.494 n=10)
ParseRW/snowflake/medium    156.7µ ± 0%   156.5µ ± 0%       ~ (p=0.075 n=10)
ParseRW/clickhouse/medium   74.44µ ± 1%   73.97µ ± 1%       ~ (p=0.029 n=10)
ParseFresh/simple_select    7.800µ ± 1%   7.891µ ± 2%  +1.17% (p=0.000 n=10)
ParseFresh/join             17.00µ ± 1%   17.19µ ± 0%  +1.13% (p=0.002 n=10)
ParseFresh/complex          27.31µ ± 1%   27.64µ ± 1%  +1.21% (p=0.000 n=10)
PoolGetPut/serial           56.13n ± 1%   56.12n ± 2%       ~ (p=0.927 n=10)
geomean                     10.94µ        10.96µ       +0.21%

                          │     base     │                head                 │
                          │     B/s      │     B/s       vs base               │
ParseE2E/simple_select      17.19Mi ± 0%   17.09Mi ± 0%  -0.55% (p=0.000 n=10)
ParseE2E/select_where       33.77Mi ± 1%   33.53Mi ± 0%       ~ (p=0.022 n=10)
ParseE2E/join               27.05Mi ± 0%   27.14Mi ± 2%       ~ (p=0.424 n=10)
ParseE2E/aggregate          24.93Mi ± 1%   24.76Mi ± 2%  -0.69% (p=0.005 n=10)
ParseE2E/subquery           27.60Mi ± 0%   27.24Mi ± 2%  -1.31% (p=0.000 n=10)
ParseE2E/cte                30.58Mi ± 0%   30.38Mi ± 1%  -0.64% (p=0.000 n=10)
ParseE2E/window             35.43Mi ± 0%   35.35Mi ± 2%       ~ (p=0.171 n=10)
ParseE2E/complex            30.37Mi ± 1%   30.57Mi ± 1%       ~ (p=0.012 n=10)
ParseE2E/insert             36.84Mi ± 0%   36.93Mi ± 0%       ~ (p=0.027 n=10)
ParseE2E/update             29.24Mi ± 1%   29.42Mi ± 2%       ~ (p=0.361 n=10)
ParseE2E/delete             46.25Mi ± 1%   46.47Mi ± 0%  +0.46% (p=0.003 n=10)
ParseRW/postgresql/medium   25.04Mi ± 0%   25.01Mi ± 0%       ~ (p=0.424 n=10)
ParseRW/mysql/medium        35.19Mi ± 0%   35.11Mi ± 0%       ~ (p=0.342 n=10)
ParseRW/mariadb/medium      32.07Mi ± 1%   31.98Mi ± 1%       ~ (p=0.670 n=10)
ParseRW/sqlserver/medium    31.57Mi ± 0%   31.53Mi ± 0%       ~ (p=0.382 n=10)
ParseRW/oracle/medium       32.46Mi ± 0%   32.53Mi ± 1%       ~ (p=0.084 n=10)
ParseRW/sqlite/medium       30.79Mi ± 0%   30.88Mi ± 0%       ~ (p=0.492 n=10)
ParseRW/snowflake/medium    32.45Mi ± 0%   32.49Mi ± 0%       ~ (p=0.074 n=10)
ParseRW/clickhouse/medium   30.27Mi ± 1%   30.47Mi ± 1%       ~ (p=0.030 n=10)
geomean                     30.46Mi        30.44Mi       -0.07%

                          │      base      │                 head                  │
                          │      B/op      │     B/op      vs base                 │
ParseE2E/simple_select        816.0 ± 0%       816.0 ± 0%       ~ (p=1.000 n=10) ¹
ParseE2E/select_where       1.523Ki ± 0%     1.523Ki ± 0%       ~ (p=1.000 n=10) ¹
ParseE2E/join               2.305Ki ± 0%     2.305Ki ± 0%       ~ (p=1.000 n=10) ¹
ParseE2E/aggregate          1.812Ki ± 0%     1.812Ki ± 0%       ~ (p=1.000 n=10) ¹
ParseE2E/subquery           1.828Ki ± 0%     1.828Ki ± 0%       ~ (p=1.000 n=10) ¹
ParseE2E/cte                2.336Ki ± 0%     2.336Ki ± 0%       ~ (p=1.000 n=10) ¹
ParseE2E/window             1.281Ki ± 0%     1.281Ki ± 0%       ~ (p=1.000 n=10) ¹
ParseE2E/complex            6.399Ki ± 0%     6.399Ki ± 0%       ~ (p=1.000 n=10) ¹
ParseE2E/insert               824.0 ± 0%       824.0 ± 0%       ~ (p=1.000 n=10) ¹
ParseE2E/update               736.0 ± 0%       736.0 ± 0%       ~ (p=1.000 n=10) ¹
ParseE2E/delete               400.0 ± 0%       400.0 ± 0%       ~ (p=1.000 n=10) ¹
ParseRW/postgresql/medium   70.51Ki ± 0%     70.51Ki ± 0%       ~ (p=0.211 n=10)
ParseRW/mysql/medium        43.72Ki ± 0%     43.72Ki ± 0%       ~ (p=1.000 n=10) ¹
ParseRW/mariadb/medium      64.83Ki ± 0%     64.83Ki ± 0%       ~ (p=1.000 n=10)
ParseRW/sqlserver/medium    90.61Ki ± 0%     90.61Ki ± 0%       ~ (p=0.277 n=10)
ParseRW/oracle/medium       113.8Ki ± 0%     113.8Ki ± 0%       ~ (p=1.000 n=10) ¹
ParseRW/sqlite/medium       39.89Ki ± 0%     39.89Ki ± 0%       ~ (p=1.000 n=10) ¹
ParseRW/snowflake/medium    83.38Ki ± 0%     83.38Ki ± 0%       ~ (p=1.000 n=10) ¹
ParseRW/clickhouse/medium   37.13Ki ± 0%     37.13Ki ± 0%       ~ (p=1.000 n=10) ¹
ParseFresh/simple_select      896.0 ± 0%       896.0 ± 0%       ~ (p=1.000 n=10) ¹
ParseFresh/join             2.383Ki ± 0%     2.383Ki ± 0%       ~ (p=1.000 n=10) ¹
ParseFresh/complex          6.477Ki ± 0%     6.477Ki ± 0%       ~ (p=1.000 n=10) ¹
PoolGetPut/serial             0.000 ± 0%       0.000 ± 0%       ~ (p=1.000 n=10) ¹
geomean                                  ²                 +0.00%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                          │     base      │                 head                 │
                          │   allocs/op   │  allocs/op   vs base                 │
ParseE2E/simple_select       5.000 ± 0%      5.000 ± 0%       ~ (p=1.000 n=10) ¹
ParseE2E/select_where        18.00 ± 0%      18.00 ± 0%       ~ (p=1.000 n=10) ¹
ParseE2E/join                17.00 ± 0%      17.00 ± 0%       ~ (p=1.000 n=10) ¹
ParseE2E/aggregate           18.00 ± 0%      18.00 ± 0%       ~ (p=1.000 n=10) ¹
ParseE2E/subquery            14.00 ± 0%      14.00 ± 0%       ~ (p=1.000 n=10) ¹
ParseE2E/cte                 21.00 ± 0%      21.00 ± 0%       ~ (p=1.000 n=10) ¹
ParseE2E/window              11.00 ± 0%      11.00 ± 0%       ~ (p=1.000 n=10) ¹
ParseE2E/complex             50.00 ± 0%      50.00 ± 0%       ~ (p=1.000 n=10) ¹
ParseE2E/insert              17.00 ± 0%      17.00 ± 0%       ~ (p=1.000 n=10) ¹
ParseE2E/update              13.00 ± 0%      13.00 ± 0%       ~ (p=1.000 n=10) ¹
ParseE2E/delete              5.000 ± 0%      5.000 ± 0%       ~ (p=1.000 n=10) ¹
ParseRW/postgresql/medium    646.0 ± 0%      646.0 ± 0%       ~ (p=1.000 n=10) ¹
ParseRW/mysql/medium         365.0 ± 0%      365.0 ± 0%       ~ (p=1.000 n=10) ¹
ParseRW/mariadb/medium       523.0 ± 0%      523.0 ± 0%       ~ (p=1.000 n=10) ¹
ParseRW/sqlserver/medium     936.0 ± 0%      936.0 ± 0%       ~ (p=1.000 n=10) ¹
ParseRW/oracle/medium       1.093k ± 0%     1.093k ± 0%       ~ (p=1.000 n=10) ¹
ParseRW/sqlite/medium        346.0 ± 0%      346.0 ± 0%       ~ (p=1.000 n=10) ¹
ParseRW/snowflake/medium     777.0 ± 0%      777.0 ± 0%       ~ (p=1.000 n=10) ¹
ParseRW/clickhouse/medium    384.0 ± 0%      384.0 ± 0%       ~ (p=1.000 n=10) ¹
ParseFresh/simple_select     6.000 ± 0%      6.000 ± 0%       ~ (p=1.000 n=10) ¹
ParseFresh/join              18.00 ± 0%      18.00 ± 0%       ~ (p=1.000 n=10) ¹
ParseFresh/complex           51.00 ± 0%      51.00 ± 0%       ~ (p=1.000 n=10) ¹
PoolGetPut/serial            0.000 ± 0%      0.000 ± 0%       ~ (p=1.000 n=10) ¹
geomean                                 ²                +0.00%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                          │    base     │                head                │
                          │  sec/stmt   │  sec/stmt    vs base               │
ParseRW/postgresql/medium   11.94µ ± 0%   11.95µ ± 0%       ~ (p=0.393 n=10)
ParseRW/mysql/medium        9.614µ ± 0%   9.636µ ± 0%       ~ (p=0.289 n=10)
ParseRW/mariadb/medium      12.28µ ± 1%   12.32µ ± 1%       ~ (p=0.671 n=10)
ParseRW/sqlserver/medium    10.04µ ± 0%   10.05µ ± 0%       ~ (p=0.343 n=10)
ParseRW/oracle/medium       10.76µ ± 0%   10.74µ ± 1%       ~ (p=0.072 n=10)
ParseRW/sqlite/medium       11.18µ ± 0%   11.15µ ± 0%       ~ (p=0.493 n=10)
ParseRW/snowflake/medium    12.05µ ± 0%   12.04µ ± 0%       ~ (p=0.066 n=10)
ParseRW/clickhouse/medium   10.63µ ± 1%   10.57µ ± 1%       ~ (p=0.028 n=10)
geomean                     11.02µ        11.02µ       -0.07%

pkg: github.com/ajitpratap0/GoSQLX/pkg/sql/tokenizer
                         │    base     │                head                │
                         │   sec/op    │   sec/op     vs base               │
Tokenize/scratch/join      1.938µ ± 0%   1.925µ ± 0%  -0.67% (p=0.001 n=10)
Tokenize/scratch/complex   5.298µ ± 1%   5.272µ ± 0%  -0.49% (p=0.004 n=10)
Tokenize/scratch/insert    926.4n ± 0%   925.0n ± 1%       ~ (p=0.542 n=10)
geomean                    2.119µ        2.109µ       -0.44%

                         │     base     │                head                 │
                         │     B/s      │     B/s       vs base               │
Tokenize/scratch/join      62.00Mi ± 0%   62.42Mi ± 0%  +0.68% (p=0.001 n=10)
Tokenize/scratch/complex   68.05Mi ± 1%   68.38Mi ± 0%  +0.49% (p=0.004 n=10)
Tokenize/scratch/insert    78.23Mi ± 0%   78.36Mi ± 1%       ~ (p=0.517 n=10)
geomean                    69.11Mi        69.42Mi       +0.44%

                         │     base     │                head                 │
                         │     B/op     │    B/op     vs base                 │
Tokenize/scratch/join      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Tokenize/scratch/complex   0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Tokenize/scratch/insert    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
geomean                               ²               +0.00%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

                         │     base     │                head                 │
                         │  allocs/op   │ allocs/op   vs base                 │
Tokenize/scratch/join      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Tokenize/scratch/complex   0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Tokenize/scratch/insert    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
geomean                               ²               +0.00%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

pkg: github.com/ajitpratap0/GoSQLX/pkg/transform
                             │    base     │                head                │
                             │   sec/op    │   sec/op     vs base               │
TransformParse/simple_select   1.447µ ± 0%   1.447µ ± 1%       ~ (p=0.378 n=10)
TransformParse/join            4.801µ ± 1%   4.813µ ± 1%       ~ (p=0.616 n=10)
TransformParse/complex         13.18µ ± 1%   13.23µ ± 0%       ~ (p=0.165 n=10)
geomean                        4.507µ        4.516µ       +0.21%

                             │     base     │                 head                  │
                             │     B/op     │     B/op      vs base                 │
TransformParse/simple_select     816.0 ± 0%     816.0 ± 0%       ~ (p=1.000 n=10) ¹
TransformParse/join            2.305Ki ± 0%   2.305Ki ± 0%       ~ (p=1.000 n=10) ¹
TransformParse/complex         6.398Ki ± 0%   6.398Ki ± 0%       ~ (p=1.000 n=10) ¹
geomean                        2.273Ki        2.273Ki       +0.00%
¹ all samples are equal

                             │    base    │                head                 │
                             │ allocs/op  │ allocs/op   vs base                 │
TransformParse/simple_select   5.000 ± 0%   5.000 ± 0%       ~ (p=1.000 n=10) ¹
TransformParse/join            17.00 ± 0%   17.00 ± 0%       ~ (p=1.000 n=10) ¹
TransformParse/complex         50.00 ± 0%   50.00 ± 0%       ~ (p=1.000 n=10) ¹
geomean                        16.20        16.20       +0.00%
¹ all samples are equal

Run, raw data and benchstat

ajitpratap0 and others added 2 commits November 2, 2026 00:37
… fast path taken [W5-02]

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DnbRnTFErhoMJdAjVEeCg2
@github-actions

github-actions Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Claude finished @ajitpratap0's task in 19s —— View job


Review of PR #904: compact SQL() fast path

Verdict: looks good, no blocking issues. I read the diff to pkg/formatter/render.go and listed the tests. I did not run the tests or benchmarks. The performance numbers are the ones in the PR description and the CI perf comment.

Correctness

  • formatStatement guard (render.go:~216). It is conservative. Clause newlines, semicolon, line width, target dialect and lowercase keywords all fall through to formatStatementSlow. A nil interface matches no case, so it also reaches the slow path, which handles nil. SetOperation and AlterTable stay on the slow path.
  • Single-statement shortcut in formatAST. With one statement, no comments and no execute slash, it applies RestoreUnicodeIdentifiers and EnsureStatementTerminator, as the general loop does. The join of one element is a no-op. I did not check the batch-separator behaviour against the loop. The PR text says it matches.
  • Redundant check. !opts.NewlinePerClause in the shortcut is not needed for correctness. It keeps the shortcut narrow, so I would leave it.

Suggestions (non-blocking)

  1. Coupling comment. Add a one-line comment at the formatStatement switch. It should say that any change to the SQL() methods of these four statement types must keep TestCompactFastDifferential green.
  2. Test name. The PR description says TestCompactFastPathDifferential, but the test is TestCompactFastDifferential. Update the description.
  3. Vacuous-test guard. The matrix includes options where opts.Dialect != "" or KeywordLower. Those take the slow path, so they compare slow against slow. Count the comparisons where the fast path was actually taken and assert the count is above zero. A later guard change then cannot make the test pass trivially.
  4. Test runtime. The differential test parses every fixture in 9 dialects and has no testing.Short() skip. It therefore also runs in the pre-commit -short run. Check how long it takes. A -short skip, or a sampled subset in short mode, would be reasonable if it is slow.
  5. Nested statements. The forced-slow-path check on nested statements was a one-off run and is not in the repo. A small table test could cover a subquery, a CTE and INSERT … SELECT. It would compare formatStatement with formatStatementSlow.
  6. Benchmark note. select_where reaches −27.2%, below the −30% target. The PR already says so, so no change is needed.
  7. Merge ordering. Confirm that the W0-02 goldens and the W0-09 perf gate are on main. The PR text says the branch was merged with wave 0.

Performance: The shadow perf gate passes. Normalize/* improves 11–15% with fewer allocations. The ParseE2E, ParseRW, ParseFresh, Tokenize/scratch/* and TransformParse benchmarks show no regression. Instructions per operation stay within ±0.13%.

Security: No concerns. The change adds no new input handling or I/O.

Test coverage: Strong. It has the differential matrix, option tests, Unicode tests, a nil test, and single-statement and hand-built comment tests for formatAST.
· branch perf/w5-02-compact-sql-fastpath

…ble; guard NewlinePerClause [W5-02]

fast_eligible now counts only the four allowlisted statement types, and
TestCompactFastOptions gains a NewlinePerClause-only case so the formatter
package kills a mutation removing that guard.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DnbRnTFErhoMJdAjVEeCg2
@github-actions

github-actions Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Claude finished @ajitpratap0's task in 13s —— View job


Review of PR #904: compact SQL() fast path

Verdict: looks good, no blocking issues. I read the render.go diff. I did not run the tests or benchmarks. The performance numbers are the ones in the PR description and the CI perf comment.

Correctness

  • formatStatement guard. It is conservative. Clause newlines, semicolon, line width, target dialect and lowercase keywords all fall through to formatStatementSlow. A nil interface matches no case and reaches the slow path, which handles nil. SetOperation and AlterTable stay on the slow path for the stated reasons.
  • Single-statement shortcut in formatAST. With one statement, no comments and no execute slash, it applies RestoreUnicodeIdentifiers and EnsureStatementTerminator. I did not re-check the batch-separator behaviour against the general loop. The !opts.NewlinePerClause check is not needed for correctness, but it keeps the shortcut narrow, so I would leave it.

Suggestions (all non-blocking)

  1. Coupling comment. Add a one-line comment at the formatStatement switch. It should say that any change to the SQL() methods of these four statement types must keep TestCompactFastDifferential green.
  2. Description typo. The PR text says TestCompactFastPathDifferential, but the test is named TestCompactFastDifferential.
  3. Nested statements. The forced-slow-path check on nested statements was a one-off run and is not in the repo. A small table test could cover a subquery, a CTE and INSERT … SELECT, comparing formatStatement with formatStatementSlow.
  4. Test runtime. The differential test parses every fixture in 9 dialects and runs 492,894 comparisons. Check that it is acceptable under the pre-commit -short run, or add a -short skip or a sampled subset.
  5. Benchmark. select_where reaches −27.2%, below the −30% target. The PR already says so, so no change is needed.
  6. Merge ordering. Confirm that the W0-02 goldens and the W0-09 perf gate are on main. The branch has merged wave 0.

Performance: The shadow perf gate passes. Normalize/* improves 11–15% with fewer allocations. No other gated benchmark regressed, and instructions per operation stay within ±0.13%.

Security: No concerns. The change adds no new input handling or I/O.

Test coverage: Strong. It has the differential matrix, option tests, Unicode tests, a nil test, and single-statement tests for formatAST. The latest commits also assert that the fast path is actually taken and guard NewlinePerClause.
· branch perf/w5-02-compact-sql-fastpath

@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Claude finished @ajitpratap0's task in 13s —— View job


Review of PR #904: compact SQL() fast path

Verdict: looks good, no blocking issues. I read the pkg/formatter/render.go diff. I did not run the tests or benchmarks, so the performance numbers are the ones in the PR description and the CI perf comment.

Correctness

  • formatStatement guard. It is conservative. Clause newlines, semicolon, line width, target dialect and lowercase keywords all fall through to formatStatementSlow. A nil interface matches no case and reaches the slow path, which handles nil. SetOperation and AlterTable stay on the slow path.
  • Single-statement shortcut in formatAST. With one statement, no comments and no execute slash, it applies RestoreUnicodeIdentifiers and EnsureStatementTerminator as the general loop does. I did not re-check batch-separator handling against the loop line by line; the PR text and tests say it matches.
  • Redundant check. !opts.NewlinePerClause in the shortcut is not needed for correctness. It keeps the shortcut narrow, so I would leave it.

Suggestions (all non-blocking)

  1. Coupling comment. Add a one-line comment at the formatStatement switch. It should say that any change to the SQL() methods of these four statement types must keep TestCompactFastDifferential green.
  2. Description typo. Earlier bot comments refer to TestCompactFastPathDifferential. The test is TestCompactFastDifferential, which matches the PR text.
  3. Nested statements. The forced-slow-path check on nested statements was a one-off run and is not in the repo. A small table test could cover a subquery, a CTE and INSERT … SELECT, comparing formatStatement with formatStatementSlow.
  4. Test runtime. The differential test runs about 491k comparisons. Check that it is acceptable under the pre-commit -short run, or add a -short skip or a sampled subset.
  5. Benchmark. select_where reaches −27.2%, below the −30% target. The PR already says so, so no change is needed.

Performance: The shadow perf gate passes. Normalize/* improves 12–17% with fewer allocations. No other gated benchmark regressed, and instructions per operation stay within ±0.13%.

Security: No concerns. The change adds no new input handling or I/O.

Test coverage: Strong. It has the differential matrix, option tests, Unicode tests, a nil test, and single-statement tests for formatAST. The latest commits also assert that the fast path is actually taken and guard NewlinePerClause.
· branch perf/w5-02-compact-sql-fastpath

@ajitpratap0
ajitpratap0 merged commit f03332f into main Nov 2, 2026
55 of 56 checks passed
@ajitpratap0
ajitpratap0 deleted the perf/w5-02-compact-sql-fastpath branch November 2, 2026 10:23

This branch was successfully deployed

1 active deployment
Preview — 578d1561 Deployed Nov 2, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant