Add Func::approximate_by() and composable Approximations by alexreinking · Pull Request #9486 · halide/Halide

alexreinking · 2026-09-29T21:36:05Z

Adds Func::approximate_by() and composable Approximations, a way to splice a lossy transform such as a quantize/dequantize round trip into an existing call graph. Like rfactor() or a targeted Func::in(), approximate_by() eagerly replaces calls to a Func within the given consumers, here with decode(encode(f)).

Units are plain structs with encode()/decode(), held by a type-erased Approximation; Pointwise builds an elementwise unit from two lambdas.
Combinators: Compose, Parallel (positional or by port name), Permute, Choose, Identity, and the TrustedInverse escape hatch.
Layout units: BlockReshape (including multi-dimensional BlockReshape::tiles), StructLayout, PlanarFieldPack, LittleEndianScalarPack, all rank-polymorphic over trailing batch dimensions.
Ports and signatures: named wires; declared types, dimensions and ranges are checked at every call; describe() and check_ranges().
Derived ports (Derive): a port computed from other ports (e.g. a per-block sum of codes); decode reads it through its own Func so schedules can place it, and derived_ports_hold() checks it.
Port tags: user-defined string tags on ports, propagated through the combinators, so schedules select decode Funcs by role (decode_funcs(tag), tags_of(f)) without the core prescribing a vocabulary.
Results: a trace tree, encoded_by/decoded_by lookups, the encoded boundary for Pipeline::sever(), and decode_funcs() for eager_inline().
Tools (header-only): make_codec() builds encode/decode generators from one Approximation (using the new Generator::add_output(name, Func)); halide_approximation_testing.h provides seeded inputs, round-trip error reports and a property library.

The design rationale is in doc/Approximation.md. Tutorial lesson 25 builds GGML's Q4_0 from these pieces and checks it bit-for-bit against GGML's quantizer. This is a research-mode API.

Part of stack #9654. Uses Stage::hoist_invariants(groups) (#9508) and the eager_inline() contract (#9510), and sits on Func::precompute() (#9628); the GGML app (#9511) is the motivating user.

Tests: correctness_approximate_by, correctness_approximation_* (including _derived_ports and _tags), generator_aot_approximation_codec, tutorial lesson 25.

Authored by GitHub Copilot

codecov · 2026-09-29T23:56:15Z

Codecov Report

❌ Patch coverage is 80.67010% with 225 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.37%. Comparing base (04b0945) to head (8f01194).

Files with missing lines	Patch %	Lines
src/Approximation.cpp	82.34%	101 Missing and 84 partials ⚠️
src/Approximation.h	41.17%	19 Missing and 1 partial ⚠️
src/Func.cpp	73.33%	11 Missing and 5 partials ⚠️
src/Inline.cpp	81.81%	3 Missing and 1 partial ⚠️

Additional details and impacted files

@@                        Coverage Diff                        @@
##           alexreinking/stage-distribute    #9486      +/-   ##
=================================================================
+ Coverage                          72.26%   72.37%   +0.10%     
=================================================================
  Files                                262      264       +2     
  Lines                              82223    83352    +1129     
  Branches                           20113    20397     +284     
=================================================================
+ Hits                               59422    60323     +901     
- Misses                             16603    16721     +118     
- Partials                            6198     6308     +110

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Handles carry a label (explicit via labelled(), else the unit's name(), else its demangled type name), shared by every copy. Each encode/decode records a tree of the handle calls made beneath it, available as `trace` on EncodeResult/DecodeResult and encode_trace/decode_trace on ApproximationResult, and printable via operator<<. The flat stage-output lists are now derived from the tree. ApproximationResult::stage_ports() returns every stage-boundary Func for scheduling. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Units may declare an ApproximationSignature (statically, or contextually from their input ports for combinators) naming their ports and optionally constraining each port's type and dimensionality; the handle validates Funcs against it in both directions. Port names flow through Compose, Apply, Identity, Permute, and Choose, and are recorded in the trace, so Apply can select a port by name (with arities taken from the inner stage) and results can be queried with encoded_by(stage, "name"). Undeclared units pass names through when arity is preserved and fall back to positional names otherwise. describe() (and operator<<) renders an Approximation's structure and signatures without running it. The core components declare signatures where their port types are known. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…ions ApproximationTesting.h (header-only, in Halide.h) provides seeded, platform-independent input distributions (including adversarial ones), verify_round_trip() error reports, and a property library (lossless, bounded_error, within_declared_bound, idempotent_requantize, zero- and sign-preservation, outputs_within_declared_ranges) run by check_property() over seeded trials, with the failing seed reported for reproduction. Properties can be conditioned on input ranges: ports may declare value ranges (preconditions on encode inputs, guarantees on encode outputs), default generators respect them, check_ranges()/describe() flag unmet preconditions statically, and Property::at(stage) checks one stage of a composition on the values that actually reach it from upstream. Units may also declare lossless() or a per-element error_bound(); the core components declare these where they hold. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Renames doc/ApproximationDesign.md to doc/Approximation.md, rewrites it to describe the API as it now exists (the duck-typed handle, unit forms, discovered intermediates, the trace tree, ports and signatures, and ApproximationTesting.h), and links it from the Guides toctree. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Decoding took its output port names from the unit's signature resolved without an encode-side context, so Permute (whose context-free signature is positional) lost the names it received, and a by-name Apply after it failed in decode. Units may now provide decoded_ports(encoded); Permute, Identity, Apply, Compose, TrustedInverse and Choose do. The testing helpers could not read back Tuple-valued or struct-typed encoded Funcs but still dereferenced them. Properties needing encoded values now fail with a clear message instead; the others still apply. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

It is a test-time helper that no pipeline needs, so it now lives next to halide_image_io.h as tools/halide_approximation_testing.h, installed with Halide::Tools, rather than in the monolithic Halide.h. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Apply's positional and by-name forms are replaced by Parallel, a product combinator: Parallel{a, b} gives each child a consecutive slice sized by its signature, and Parallel{{"codes", a}, {"scale", b}} routes ports by name, passing the rest through in place. Port names now identify a wire in both directions: names flowing from upstream win over a unit's declared defaults, and decode's outputs mirror encode's inputs, including for a stand-alone decode. This makes by-name routing work across StructLayout and removes the decoded_ports() hook. Compose now lists stages in encode order (innermost first). Also: the testing header compiles against a plain Halide.h, and SymmetricBlockQuantize's TruncateHalfUpWithOffset rounding adds qmax + 0.5 in one step, as GGML does. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Builds GGML's Q4_0 block format from Approximation components, splices it into a dot product with approximate_by, splits the quantizer off with compute_offline, and checks it bit-for-bit against a C++ transcription of GGML's reference quantizer, plus property checks. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Quantization math (SymmetricBlockQuantize, AdditiveRadixSplit, BinaryAlphabetPack) is GGML-specific and belongs in client code. StorageCast and AdditiveOffset are dropped in favor of inline Pointwise units, which gain with_types(), with_ranges() and with_lossless(). The remaining generic layout units (BlockReshape, StructLayout, PlanarFieldPack, LittleEndianScalarPack) move into Approximation.h/.cpp, and ApproximationComponents.h is deleted. Tests and the tutorial define their own quantizers. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Approximation default labels used typeid(T) unconditionally, which breaks consumers built with -fno-rtti. Fall back to a generic label. parallel_fork.cpp declared an unscoped enum value Parallel under using namespace Halide, which is now ambiguous with Halide::Parallel. Make the enum scoped. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Derive unit default labels from __PRETTY_FUNCTION__/__FUNCSIG__ instead of typeid so -fno-rtti builds keep real type names. Skip error-path tests when exceptions are disabled, use relative float tolerances in approximate_by, and address clang-tidy findings in Approximation and lesson 25. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

BlockReshape, PlanarFieldPack and StructLayout now pass trailing dimensions through unchanged, so one scheme value approximates a row or a matrix of rows. Their signatures are contextual: dimensionalities follow the input ports, and are unknown without context. StructLayout's record_dimensions is now optional and inferred from its inputs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

decode_funcs() lists the decode side's Funcs that eager_inline() accepts as they are scheduled now (replacement and the decode root's intermediates, producers first), so one consumer.eager_inline() call folds the inlinable decode chain in. The eager_inline() checks are factored into Internal::eager_inline_obstacle(), which returns the reason rather than erroring; eager_inline()'s messages are unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

make_codec() packages the configure()/sever()/adopt pattern for codec Generators: it approximates an ImageParam of values by a scheme, gives the result a default schedule, severs it at the encoded Funcs, and returns both halves with stable port names. adopt_encoder() and adopt_decoder() declare either half's ports on a Generator. Header-only, so it adds no library API. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Leave stage ports inline by default, as Halide does; the encode side's stage boundaries need not be materialized for a codec. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…d dims Func names are made unique per process, so a codec Generator's configure() renamed its Funcs (decoded$1, codes$2) on its second run in a multi-target build, and the codec helper rejected them. Add GeneratorBase::add_output(const std::string &name, const Func &existing), have Codec record its port names and adopt with them, and build the approximation_codec AOT test multi-target to cover it. StructLayout now rejects an inferred record dimensionality below one (e.g. a 0-D scalar slot) with a user error instead of crashing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

BlockReshape::tiles({b0, b1, ...}) splits each leading dimension by its own block, within-tile indices first: (x0, x1, rest...) <-> (x0 % b0, x1 % b1, x0 / b0, x1 / b1, rest...). Trailing dimensions pass through; types and ranges pass through; lossless. tiles({b}) is BlockReshape(b). Lets a schedule vectorize across a whole tile, e.g. the 2x2 sub-tiles of a matrix-multiply register block. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The 2x2 tile is one dense 4-wide store only if the output's tile rows are adjacent; without a stride constraint it lowers to two 2-wide stores. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

What's left of the within-block index for the last extent is already in range, and the % made bounds inference widen a piece of a block to the whole block (e.g. a stage at a loop over 4-element pieces of a 32-element block realized all 32). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add Derive{port, from, psi}, a lossless stage whose encode appends a port defined as psi(from...) and whose decode drops it, so the round trip and its idempotent are unchanged. Stages after it may encode the port (e.g. round it); the stored value then approximates psi of the decoded ports within the bound of the one lossy stage that follows Derive on the port. ApproximationResult::derived_ports() / derived_port(name) (and derived_ports(enc_trace, dec_trace)) expose each DerivedPort: the decode-side from-ports and stored port, psi instantiated on the decode side, and the declared relation (exact, a bound, or why there is none). match_derived_port(term, port) decides whether a Func equals the port's definition under real-valued semantics: port calls are atoms, pure Funcs are expanded, sum reductions become sums over alpha-renamed RVars, + - * are ring operations with invariant factors pulled out of sums, and casts are the identity only when provably exact; any rounding is opaque. Unifying port calls yields the port arguments. Func::approximate_by(const DerivedPort &, consumers[, DerivedPortCheck]) replaces calls to an outlined term in the consumers with a read of the stored port, after the match. It is lossy, bounded by the port's declared bound. DerivedPortCheck::Runtime additionally require()s the declared relation where the port is read. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

derived_ports_hold() checks, on encode(x), that every derived port holds its declared relation (exactly, or within the bound of the lossy stage that follows Derive). The round trip now also reads back Derive's from-ports, and realizes derived ports that no later stage reads (dead in the decode) so the check is not vacuous. closed() is a structural check, run in tests and never enforced on construction: every encode and decode output of every stage must reach its stage's inputs, and no other Func of the round trip, Buffer or ImageParam (scalar Params are allowed). It reports the stage path and the offending Func, catching units whose decode reads a value captured from encode. test/correctness/approximation_derived_ports covers a blockwise int8 scheme with a derived block-sum port: introspection, substitution in a consumer of an outlined block sum, the argument mapping, rejections (wrong extent, wrong port, a rounding in between, a stored-value substitution, missing bounds), the run-time check on a corrupted buffer, idempotence of the consistent scheme versus one deriving the sum from the unrounded scale, derived_ports_hold, and closed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Schedules and boundary handling tag indices with likely() and promise_clamped(), which are value-preserving. Strip them from the arguments of port calls before comparing them, so a term indexed through such tags still matches the derived port's definition. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

DerivedPort::stored and ::from are documented as decode-side Funcs, but when an aliasing unit (Identity, Permute, a Parallel pass-through) or nothing follows Derive, they were the encoder's own Funcs. Pipeline::sever rewrites calls, not handles, so after severing the encoded ports the derived port's definition no longer matched the consumer's term, or a substitution re-read the offline encoder. Add Derive::decode_side, which wraps each `from` port and the derived port in a fresh `<port>_decoded` Func; Approximation::decode hands those to Derive::decode and records them as the decode trace's inputs, so derived_ports() always returns decode-side Funcs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Ports of an approximation may now carry tags (ApproximationTags, a set of strings) naming what their values are, in a vocabulary chosen by the user; the core prescribes none. Units that split values into components declare tags on their signatures' outputs; the framework carries them through the combinators: tags flow into input ports, untagged outputs inherit those of the same-named input, else of the input at the same position, else the union of the inputs' tags. Derive takes an optional tag set for its port. Every trace node records the tags of its ports, inputs and intermediates, and ApproximationResult answers decode_funcs(tag), tags_of(f) and owns(f), so schedules can select decode Funcs by component instead of by type or by probing their definitions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

match_derived_port took a derived port's arguments only from definition calls whose index argument is one of its Vars, so a port whose definition indexes the codes by an expression of a Var, e.g. per-16 sums s(t, b) = sum(codes(16 * t + r, b)), never matched a term reading the same sums: no call determined t. Accept definition arguments affine in one Var, c * v + w (c a nonzero integer constant, w free of v, possibly using the sum's binders and the definition's other Vars). The term's argument e proposes v = (e - w) / c when the division is exact and leaves no binder and no Var of the definition; Vars in w are replaced by their own candidates, iterated to a fixpoint. The definition's Vars are renamed apart from the term's, and both sides' binders are named by nesting depth as the normalizer's keys name them. Candidates remain proposals: the existing check that the definition at them normalizes to the term is unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

alexreinking added this pull request to stack #9487 September 29, 2026 21:36

alexreinking mentioned this pull request Sep 29, 2026

Add apps/ggml: Halide reimplementation of GGML quant kernels #9417

Closed

3 of 5 tasks

alexreinking force-pushed the alexreinking/approximations branch from 15927f1 to bf8c475 Compare September 29, 2026 21:40

alexreinking changed the title ~~Add Func::approximate_by() directive and Approximation combinators~~ Add Func::approximate_by() and composable Approximations Sep 30, 2026

alexreinking force-pushed the alexreinking/approximations branch from c2ea4a9 to b1a6b31 Compare September 30, 2026 05:36

alexreinking mentioned this pull request Sep 30, 2026

Filter std:: instantiations out of Halide.dll's export list #9489

Merged

2 of 5 tasks

alexreinking force-pushed the alexreinking/approximations branch 2 times, most recently from cccab1a to 891b76b Compare September 30, 2026 19:41

alexreinking force-pushed the alexreinking/approximations branch 2 times, most recently from 84a4a8d to ef2ee20 Compare October 2, 2026 17:22

alexreinking force-pushed the alexreinking/approximations branch 2 times, most recently from 03c20e6 to e319cdb Compare October 2, 2026 19:40

Base automatically changed from alexreinking/compute-offline to main October 5, 2026 17:19

alexreinking force-pushed the alexreinking/approximations branch 2 times, most recently from 9ea2609 to cdab38c Compare October 6, 2026 17:05

alexreinking removed this pull request from stack #9487 October 6, 2026 17:11

alexreinking changed the base branch from main to alexreinking/stage-distribute October 6, 2026 17:13

alexreinking added this pull request to stack #9509 October 6, 2026 17:13

alexreinking force-pushed the alexreinking/approximations branch from cdab38c to 48ea8fd Compare October 6, 2026 17:47

alexreinking removed this pull request from stack #9509 October 6, 2026 17:47

alexreinking changed the base branch from alexreinking/stage-distribute to alexreinking/eager-inline-schedules October 6, 2026 17:47

alexreinking added this pull request to stack #9512 October 6, 2026 17:47

alexreinking mentioned this pull request Oct 6, 2026

apps/ggml: GGML quantized kernels on Approximations #9511

Draft

alexreinking removed this pull request from stack #9512 October 6, 2026 19:33

alexreinking force-pushed the alexreinking/approximations branch from 48ea8fd to ca134f5 Compare October 6, 2026 19:33

alexreinking changed the base branch from alexreinking/eager-inline-schedules to alexreinking/hoist-invariants-bounds October 6, 2026 19:34

alexreinking added this pull request to stack #9522 October 6, 2026 19:34

alexreinking and others added 30 commits November 2, 2026 18:48

Add Approximation to the Makefile build

8c7164d

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Rename compute_offline to sever in approximations

1c58426

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Remove version qualifiers from approximation design

d4fdb35

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

make_codec: compute_root only Funcs with update definitions

e98023c

Leave stage ports inline by default, as Halide does; the encode side's stage boundaries need not be materialized for a codec. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

approximation_components: constrain the tiles test's output stride

91eee2d

The 2x2 tile is one dense 4-wide store only if the output's tile rows are adjacent; without a stride constraint it lowers to two 2-wide stores. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Format shared inline schedule validation

f58deb5

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

approximations: document derived ports, matching, and the new properties

7802069

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Navigation Menu

Repository navigation

Add Func::approximate_by() and composable Approximations - #9486

Add Func::approximate_by() and composable Approximations#9486
alexreinking wants to merge 34 commits into
alexreinking/func-precomputefrom
alexreinking/approximations

alexreinking commented Sep 29, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Sep 29, 2026 •

edited

Loading

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

alexreinking commented Sep 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Sep 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

alexreinking commented Sep 29, 2026 •

edited

Loading

codecov Bot commented Sep 29, 2026 •

edited

Loading