Skip to content

fix(spark): align e2e operator chart with controller - #935

Closed
KrishSharma4406 wants to merge 1 commit into
kubeflow:mainfrom
KrishSharma4406:fix-spark-e2e-operator-chart
Closed

KrishSharma4406 wants to merge 1 commit into
kubeflow:mainfrom
KrishSharma4406:fix-spark-e2e-operator-chart

Conversation

@KrishSharma4406

@KrishSharma4406 KrishSharma4406 commented Nov 6, 2026 •

Copy link
Copy Markdown

What this PR does / why we need it:

The Spark Examples E2E workflow builds the Spark Operator controller image (ghcr.io/kubeflow/spark-operator/controller:local) from current master, but hack/e2e-setup-cluster.sh installed published chart 2.1.0. This causes RBAC drift: the master controller includes spark-operator#3240
requiring networkpolicies create/patch permissions, while chart 2.1.0 lacks them, causing SparkApplication submissions to fail with a 403 error (reported in #933).

Change:

  • hack/e2e-setup-cluster.sh: Added SPARK_OPERATOR_CHART_PATH support to install from a local Helm chart directory when provided, while preserving the published 2.1.0 chart as the default fallback when unset.

  • Makefile: Forwarded SPARK_OPERATOR_CHART_PATH through test-e2e-setup-cluster.

  • .github/workflows/test-spark-examples.yaml: Pointed SPARK_OPERATOR_CHART_PATH to ./spark-operator/charts/spark-operator-chart so the controller image and its Helm chart/RBAC are drawn from the exact same checkout.

Allows hack/e2e-setup-cluster.sh to use SPARK_OPERATOR_CHART_PATH to install the Spark Operator from a local chart path instead of the published chart.

In test-spark-examples.yaml, points the chart path to the checkout of kubeflow/spark-operator so that the controller image and its RBAC are kept in sync.

Closes kubeflow#933

Signed-off-by: Krish Kumar <krishsharma4406@gmail.com>
Copilot AI balanced review requested due to automatic review settings November 6, 2026 18:45
@github-project-automation github-project-automation Bot moved this to Ready to Merge in Kubeflow SDK Nov 6, 2026
@google-oss-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign electronic-waste for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

🎉 Welcome to the Kubeflow SDK! 🎉

Thanks for opening your first PR! We're happy to have you as part of our community 🚀

Here's what happens next:

  • If you haven't already, please check out our Contributing Guide for repo-specific guidelines and the Kubeflow Contributor Guide for general community standards
  • Our team will review your PR soon! cc @kubeflow/kubeflow-sdk-team

Join the community:

Feel free to ask questions in the comments if you need any help or clarification!
Thanks again for contributing to Kubeflow! 🙏

@thisis-Shitanshu

Copy link
Copy Markdown

Thanks @KrishSharma4406 for working on this.

#934 was opened earlier for the same issue and is already under review, so I'd prefer to keep the fix consolidated there rather than maintain two overlapping implementations.

Your changes around skipping the Helm repo update for local-chart mode and reporting the local chart path are useful, and I've asked for those refinements to be incorporated into #934.

Thanks again for picking this up.

@tariq-hasan

Copy link
Copy Markdown
Member

Thanks for the contribution @KrishSharma4406!

Let me close this in favor of #930 since it covers a much bigger scope and addresses several issues that all need to be resolved together.

Would appreciate your reviews on that PR @thisis-Shitanshu and @KrishSharma4406!

/close

@google-oss-prow google-oss-prow Bot closed this Nov 7, 2026
@google-oss-prow

Copy link
Copy Markdown
Contributor

@tariq-hasan: Closed this PR.

Details

In response to this:

Thanks for the contribution @KrishSharma4406!

Let me close this in favor of #930 since it covers a much bigger scope and addresses several issues that all need to be resolved together.

Would appreciate your reviews on that PR @thisis-Shitanshu and @KrishSharma4406!

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

4 participants