Skip to content

(PA-9119) Diagnose dnf5 metadata staleness and fix versioncmp() truthiness bug - #851

Merged
joshcooper merged 1 commit into
mainfrom
PA-9119/main/dnf5-metadata-refresh
Aug 19, 2026
Merged

joshcooper merged 1 commit into
mainfrom
PA-9119/main/dnf5-metadata-refresh

Conversation

@span786

@span786 span786 commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Problem

acceptance/tests/test_upgrade_puppet8_to_puppet9.rb failed deterministically on Fedora 41: Package[puppet-agent] (ensure => 'latest') found no update immediately after Yumrepo['pc_repo'] created the repo pointing at the newer puppet9-nightly build, in the same catalog run.

Notice: .../Yumrepo[pc_repo]/ensure: created
...
Debug: Package[puppet-agent](provider=dnf): Yum didn't find updates, current version (8.21.0.13...) is the latest
...
Minitest::Assertion: Expected nil to be truthy.

Confirmed via a live failing build: Jenkins build #1, PLATFORM=fedora41-64default.

Tracked in PA-9119 and PUP-12151.

Root cause

A dnf5 metadata-cache staleness bug: dnf5's on-disk repo metadata cache can lag a repo that was created or changed earlier in the same Puppet transaction. PA-9119's investigation deterministically reproduced this (100% correlation across 6 trials tied to cache-warm-before-repo-exists), ruled out publish-lag, and pinned it to Puppet's dnf/yum provider never forcing a metadata refresh when checking for updates.

The actual fix lives in puppet-private: puppetlabs/puppet-private#217 (adds a dnf5-gated --refresh to check_updates/available_versions). This PR is the module-side half: better failure diagnostics for this class of bug, plus an unrelated bug found while investigating.

What's in this PR

  1. Acceptance test diagnostics (test_upgrade_puppet8_to_puppet9.rb): on a post-upgrade version-assertion failure, log the repo/package state (the real --debug decision lines, dnf repolist, dnf list --showduplicates) before teardown destroys the evidence, so a recurrence is diagnosable from CI logs alone. Guarded so it only runs on RPM-family agents (checked with a non-raising test -d /etc/yum.repos.d, not facter/fact_on — this branch exists specifically for the case where the agent's facter is broken, and fact_on has no accept_all_exit_codes equivalent, so it would raise in exactly that scenario). Also fixed the pre-existing nil.match? crash risk: puppet_agent_version_on returns nil (not a non-matching string) when facter aio_agent_version itself fails, which would otherwise raise before the diagnostics or even the assert could run.

  2. manifests/install.pp bug fix: found during PA-9119's investigation —

    if $facts['os']['name'] == 'Fedora' and versioncmp($facts['os']['release']['major'], '41') {

    versioncmp() returns an Integer, and Puppet treats every Integer (including 0 and negatives) as truthy, so this was unconditionally true for every Fedora release, not just >= 41. Fixed to versioncmp(...) >= 0. This restores the pre-existing behavior for Fedora < 41 (explicit 'yum' provider) that a recent change (d058ce3, introducing dnf5 handling for Fedora >= 41) unintentionally dropped for all Fedora releases. Added a regression test (spec/classes/puppet_agent_spec.rb) confirmed to fail against the buggy condition.

Testing performed

  • bundle exec rake spec — 1135 examples, 0 failures
  • bundle exec rake lint / rake syntax / rake metadata_lint — clean
  • bundle exec rubocop on both modified Ruby files — clean
  • Manually verified the new install.pp regression test fails against the pre-fix if versioncmp(...) condition (reverted, confirmed red, restored)

Related

  • puppetlabs/puppet-private#217 (the actual provider fix this test diagnoses)
  • PA-9119
  • PUP-12151

Base branch note: stacked on #847 (PA-8998, already open) since this test file's current state depends on it; the diff will narrow to just this PR's changes once #847 merges.

🤖 Generated with Claude Code

@span786
span786 requested review from a team and bastelfreak as code owners August 18, 2026 16:14
@span786
span786 marked this pull request as draft August 18, 2026 16:15
@span786
span786 marked this pull request as ready for review August 18, 2026 16:17
@joshcooper

Copy link
Copy Markdown
Contributor

@span786 I assume this is just for testing and can be closed?

Base automatically changed from PA-8998-require-puppet9-master to main August 19, 2026 15:17
…iness bug

The puppet8->puppet9 upgrade acceptance test failed deterministically
on Fedora 41: Yumrepo[pc_repo] is created, then Package[puppet-agent]
(ensure => latest) is managed in the same catalog run and finds no
update, even though the newer build is published and reachable. Root
cause is a dnf5 metadata-cache staleness bug in Puppet's dnf/yum
package provider, fixed separately in puppet-private (PUP-12151).

- Add failure diagnostics (repo/package state, the real --debug
  decision lines, dnf repolist/list output) so a recurrence is
  actionable from CI logs alone, without warming the cache the
  diagnostics are trying to observe and without raising on non-RPM
  platforms or when facter itself is broken (the exact scenario these
  diagnostics exist to capture).
- Fix a versioncmp() truthiness bug in manifests/install.pp found
  during this investigation: versioncmp() returns an Integer, and
  Puppet treats every Integer (including 0 and negatives) as truthy,
  so `if versioncmp(...)` was true for every Fedora release, not just
  >= 41. Fedora releases below 41 were silently getting an unset
  package provider (relying on dnf auto-selection) instead of the
  intended explicit 'yum' provider.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@joshcooper
joshcooper force-pushed the PA-9119/main/dnf5-metadata-refresh branch from 5d94536 to 5daa42c Compare August 19, 2026 15:17
@joshcooper
joshcooper merged commit 76f8944 into main Aug 19, 2026
9 checks passed
@joshcooper
joshcooper deleted the PA-9119/main/dnf5-metadata-refresh branch August 19, 2026 15:53
@joshcooper

Copy link
Copy Markdown
Contributor

@span786 I merged the wrong PR initially, sorry about that. Still getting used to GH stacks

@span786

span786 commented Aug 19, 2026 •

Copy link
Copy Markdown
Contributor Author

No worries! Opened a revert PR for the acceptance-test diagnostics added here (not needed now that the actual root cause is fixed in puppet-private#220) — the unrelated install.pp versioncmp() fix from this PR is left in place: #853

joshcooper added a commit that referenced this pull request Aug 19, 2026
Revert dnf5 upgrade-test diagnostics added in #851
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants