[GLUTEN-11708][VL] Enable Bloom filter might_contain to subfield filter translation - #12793
Merged
Merged
Conversation
zhztheplayer
force-pushed
the
wip-bloom-pushdown
branch
from
August 17, 2026 13:16
38b3802 to
e4ff8db
Compare
zhztheplayer
marked this pull request as ready for review
August 17, 2026 13:25
marin-ma
approved these changes
Aug 17, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Enables optional translation of Spark might_contain (Bloom filter) expressions into Velox subfield filters, controlled by a new static configuration flag.
Changes:
- Added a new static config
spark.gluten.sql.columnar.backend.velox.scan.bloomFilterPushdown.enabled(Scala + C++ + docs). - Extended
SparkExprToSubfieldFilterParserto parsemight_contain(bloomFilter, xxhash64_with_seed(seed, field))into a Bloom-filter-backedFilter. - Wired the new config into Velox backend initialization when registering the parser.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| docs/velox-configuration.md | Documents the new Velox scan Bloom filter pushdown config flag. |
| cpp/velox/operators/functions/SparkExprToSubfieldFilterParser.h | Adds config-driven enablement for Bloom filter translation. |
| cpp/velox/operators/functions/SparkExprToSubfieldFilterParser.cc | Implements constant-evaluation + might_contain parsing and a Bloom-filter-backed filter type. |
| cpp/velox/config/VeloxConfig.h | Introduces C++ config key + default for enabling Bloom filter pushdown. |
| cpp/velox/compute/VeloxBackend.cc | Passes the config flag into the parser registration. |
| backends-velox/src/main/scala/org/apache/gluten/config/VeloxConfig.scala | Adds the corresponding static Spark/Gluten config entry and getter. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| << "cannot push down to subfield filter"; | ||
| return std::nullopt; | ||
| } | ||
| auto seed = seedValue->as<SimpleVector<int64_t>>()->valueAt(0); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #11708