[GLUTEN-12850][VL] Support multiple Dev Container configurations - #12853
Draft
felipepessoto wants to merge 1 commit into
Draft
[GLUTEN-12850][VL] Support multiple Dev Container configurations#12853felipepessoto wants to merge 1 commit into
felipepessoto wants to merge 1 commit into
Conversation
- Keep the dynamic-link environment as the default for daily development. - Add a static-link configuration for portable jars and vcpkg reproduction. - Share post-create setup while printing variant-specific build instructions. - Persist separate ccache, Maven and vcpkg binary-cache volumes. - Select the arm64 vcpkg triplet automatically in the static environment. - Document configuration selection, limitations and safe switching. - Make the JDK 17 switch explicit for Spark 4.1 builds so Maven does not run the Java 17 release target under JDK 8. Generated-by: GitHub Copilot CLI gpt-5.6-sol Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends Gluten’s Dev Container support to a multi-configuration setup (dynamic-link as the default plus an additional static-link option), sharing common post-create initialization while updating developer documentation to describe configuration selection, intended use cases, and build/test workflows (including explicit JDK 17 switching for Spark 4.x in the dynamic image).
Changes:
- Adds a new
.devcontainer/velox-static/devcontainer.jsonconfiguration for static-link packaging and vcpkg reproduction while keeping.devcontainer/devcontainer.jsonas the default dynamic-link workflow. - Updates the shared
.devcontainer/post-create.shto tailor printed instructions and arm64 static vcpkg environment setup based on the selected configuration. - Refreshes developer docs to describe configuration choice in VS Code/Codespaces and to document build/test commands for each configuration.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| docs/developers/NewToGluten.md | Updates Dev Container references to the new multi-configuration layout under .devcontainer/. |
| docs/developers/dev-container.md | Documents configuration selection and provides updated dynamic/static build and test instructions (including Spark 4.x JDK 17 guidance). |
| .devcontainer/devcontainer.json | Marks the default dynamic-link config as the daily-dev option and sets GLUTEN_DEV_CONTAINER_VARIANT=velox-dynamic. |
| .devcontainer/velox-static/devcontainer.json | Introduces a static-link configuration with separate persisted caches (including /var/cache/vcpkg) and higher host requirements. |
| .devcontainer/post-create.sh | Shares setup across configs while printing variant-specific instructions and adding arm64 static vcpkg environment exports. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
felipepessoto
marked this pull request as draft
August 22, 2026 03:02
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes are proposed in this pull request?
Fixes #12850.
Follow-up to #12778, which made the dynamic-link image the default development environment.
.devcontainer/devcontainer.jsonas the default dynamic-link configuration for daily Velox development and Spark unit tests..devcontainer/velox-static/devcontainer.jsonfor portable jar packaging and vcpkg issue reproduction./var/cache/vcpkgbinary cache.The Spark 4.1 instructions now explicitly switch the dynamic environment from its default JDK 8 to JDK 17:
buildbundle-veloxbe.shadds the Java 17 Maven profile and release target for Spark 4.x, but a Maven profile cannot switch the JDK that is already running. Without this step Scala fails with'17' is not a valid choice for '-release'.How was this patch tested?
Parsed both configurations with
@devcontainers/cli 0.88.0 read-configuration.Ran both post-create variants and verified the dynamic Spark 4.1 instructions, static packaging instructions, unknown-variant failure and idempotent arm64 environment setup.
Built Spark 4.1 under OpenJDK 17 with the same Maven command used by
buildbundle-veloxbe.sh:BUILD SUCCESSin 4:23, producinggluten-package-1.8.0-SNAPSHOT-4.1.jar.Validated the final static command end to end on CentOS 9/GCC 12/JDK 17 using the Maven repository and vcpkg binary cache extracted from
apache/gluten:vcpkg-centos-9:./dev/buildbundle-veloxbe.sh --enable_vcpkg=ON --build_arrow=OFF \ --spark_version=3.5The build processed 124 vcpkg packages, completed in 24:55 and produced
cpp/build/releases/libgluten.soplus the 88 MBgluten-velox-bundle-spark3.5_2.12-linux_amd64-1.8.0-SNAPSHOT.jar.Ran
bash -n, the license-header check,git diff --checkand VS Code diagnostics with no errors.Was this patch authored or co-authored using generative AI tooling?
Generated-by: GitHub Copilot CLI gpt-5.6-sol