Skip to content

fix: compare waiting container reason to PodInitializing - #1396

Open
jim-relation wants to merge 1 commit into
kubeflow:notebooks-v1from
jim-relation:fix/jwa-podinitializing-status
Open

jim-relation wants to merge 1 commit into
kubeflow:notebooks-v1from
jim-relation:fix/jwa-podinitializing-status

Conversation

@jim-relation

Copy link
Copy Markdown

get_status_from_container_state compares a list literal to a string:

waiting_state = container_state["waiting"]
if ["reason"] == 'PodInitializing':

["reason"] has no receiver, so Python parses it as a one-element list literal rather than a subscript. The comparison is therefore always False, the WAITING branch is dead code, and every waiting container falls through to WARNING. A notebook that is just starting up shows a warning reading PodInitializing: No available message for container state. instead of a waiting spinner reading PodInitializing.

Introduced in kubeflow/kubeflow#7585, which extracted container_state["waiting"] into waiting_state. The other lines in that hunk picked up the new local; this one lost its old receiver without gaining it. flake8 sees valid syntax, so linting never flagged it.

Present on notebooks-v1, v1.11-branch and the v1.11.0 tag. The v2 backend has no equivalent code path.

Fix

Compare waiting_state.get("reason"). .get rather than ["reason"] because status.containerState is a corev1.ContainerState (notebook-controller/api/v1beta1/notebook_types.go:43) and ContainerStateWaiting.Reason is +optional with omitempty, so the key can be absent — the pre-#7585 subscript could raise KeyError on a valid Notebook.

Tests

test_no_message_container_state asserted the buggy output, so it was holding the defect in place. It now uses ImagePullBackOff, which keeps its original missing-message coverage.

Added test_pod_initializing_container_state, which fails against the unfixed source, and test_waiting_container_state_with_message for the warning path with both fields set, which had no coverage.

make unittest passes (12 tests) and flake8 is clean on both files.

Not fixed here

kubelet uses ContainerCreating for this same placeholder state when a pod has no init containers, and PodInitializing when it has them — see defaultWaitingState in pkg/kubelet/kubelet_pods.go. Notebooks without init containers will still show a warning while starting. That predates #7585, which had the same single-reason check, so I have left it out to keep this change focused. Glad to widen the predicate to both reasons here if you would rather have it in one PR.


Discovered by Claude while consuming this status output; patch and analysis also by Claude Code.

`get_status_from_container_state` compared a bare list literal to a
string, so the condition was always False:

    if ["reason"] == 'PodInitializing':

The refactor in kubeflow/kubeflow#7585 extracted
`container_state["waiting"]` into `waiting_state` but dropped the
receiver on this one line, leaving a list literal instead of a
subscript. The WAITING branch became unreachable and every waiting
container fell through to the WARNING branch, so notebooks that were
simply starting up rendered as "warning" with "PodInitializing: No
available message for container state." instead of "waiting" with
"PodInitializing".

Compare `waiting_state.get("reason")` instead. `.get` rather than a
subscript because `status.containerState` is a `corev1.ContainerState`
and `ContainerStateWaiting.Reason` is `+optional` with `omitempty`, so
the key can legitimately be absent.

`test_no_message_container_state` asserted the buggy output, so it is
retargeted at a non-initializing reason to keep covering the
missing-message default. Two tests are added: one pinning the
PodInitializing path that regressed, and one for the warning path with
both reason and message, which had no coverage.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Jim Broadbent <jim.broadbent@relationrx.com>
@github-project-automation github-project-automation Bot moved this to Needs Triage in Kubeflow Notebooks Sep 10, 2026
@google-oss-prow google-oss-prow Bot added the area/backend area - related to backend components label Sep 10, 2026
@google-oss-prow

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign thesuperzapper for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@google-oss-prow google-oss-prow Bot added area/frontend area - related to frontend components area/v1 area - version - kubeflow notebooks v1 size/M labels Sep 10, 2026
@christian-heusel

Copy link
Copy Markdown
Member

Hey, thanks a lot for contributing to Kubeflow Notebooks! 🎉

Out of interest, did you create your GitHub account just for this contribution? 😆

/ok-to-test

@jim-relation

Copy link
Copy Markdown
Author

Hey, thanks a lot for contributing to Kubeflow Notebooks! 🎉

Out of interest, did you create your GitHub account just for this contribution? 😆

/ok-to-test

Corporate account. Happy to confirm in DM that I'm also an account with some other FOSS contributions

@jim-relation

Copy link
Copy Markdown
Author

@christian-heusel Can this please be reviewed? Is the policy that you aren't fixing bugs in notebooks-v1 at this stage?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/backend area - related to backend components area/frontend area - related to frontend components area/v1 area - version - kubeflow notebooks v1 ok-to-test size/M

Projects

Status: Needs Triage

Development

Successfully merging this pull request may close these issues.

2 participants