Skip to content

Match Pulp entities exactly when removing a product - #1248

Merged
anfimovdm merged 2 commits into
masterfrom
fix-product-removal-pulp-cleanup
Jul 29, 2026
Merged

anfimovdm merged 2 commits into
masterfrom
fix-product-removal-pulp-cleanup

Conversation

@anfimovdm

Copy link
Copy Markdown
Contributor

Summary

Fixes AlmaLinux/build-system#506 — product removal could delete Pulp entities belonging to other products.

remove_product() fetched distributions with a name__startswith search on pulp_base_distro_name ({owner}-{product}) and then deleted every href in the result. That prefix also matches every product whose name starts with the same string, so removing user/almalinux deleted the distributions of user/almalinux-extras too.

While fixing that, two related problems in the same block:

  • The guard meant to skip repositories that are absent from Pulp compared a repository name against a distribution name. Pulp names a distribution {repo_name}-distro (PulpClient.create_rpm_distro), so the comparison never matched, continue always fired, and the product's Pulp repositories were never deleted — only its distributions.
  • The sign key repository of a community product ({owner}-{product}-sign-key-repo) is a Pulp file repository. Only the RPM distribution endpoint was queried, so it and its distribution were left behind on every removal.

Changes

  • alws/crud/products.py — remove_product() builds an exact {repo.name}-distro → repository mapping from db_product.repositories and filters the prefix-search result through it. The RPM and file distribution endpoints are queried concurrently, and each repository is deleted together with its distribution, so the "in DB but not in Pulp" case is still handled — now via the distribution's actual existence instead of a comparison that could never be true.
  • alws/utils/pulp_client.py — new get_file_distros(), the file-distribution counterpart of get_rpm_distros(), using the same paginated __get_entities path.
  • tests/test_api/test_products.py — regression test: simulates both distribution endpoints, asserts every product repository and distribution (including the sign key file repo) is deleted, and that a similarly-named product's distributions in both endpoints are untouched.
  • tests/fixtures/pulp.py — new get_file_distros fixture; test_user_product_remove needs it now that removal queries that endpoint.

Testing

./run-tests.sh — 85 passed, 10 skipped.

The new test was verified to fail against the unfixed code:

>       assert foreign_distro["pulp_href"] not in deleted_hrefs
E       AssertionError: assert '/pulp/api/v3/distributions/rpm/rpm/foreign/' not in [...]

That run also showed the second bug directly — only distribution hrefs were collected, no repository hrefs.

Product removal looked up Pulp distributions with a name__startswith
search on "{owner}-{product}", so it also matched the distributions of
every product whose name starts with the same string, and deleted them.
Removing "user/almalinux" wiped the distributions of
"user/almalinux-extras" as well.

The guard that was supposed to skip repositories missing from Pulp
compared a repository name against a distribution name. Pulp names a
distribution "{repo_name}-distro", so the comparison never matched and
the product repositories were never deleted at all.

Both are fixed by building the exact distribution name to repository
mapping from the product repositories and narrowing the prefix search
down to it. Repository and distribution are now deleted together, so
repositories absent from Pulp are still skipped.

The sign key repository of a community product is a file repository and
was never cleaned up either, since only the RPM distribution endpoint
was queried. Both endpoints are checked now.

Fixes AlmaLinux/build-system#506
Matching a distribution by its "{repo_name}-distro" name leaves behind
whatever does not follow the convention, and matching a repository by
its href from our db leaves behind whatever the db does not know about.

Removal now lists the product repositories and distributions from pulp,
picks the repositories by the exact names the product has, and picks the
distributions by the repository they serve, falling back to the name.
The sign key file repository is listed the same way.
@anfimovdm

Copy link
Copy Markdown
Contributor Author

Follow-up after testing on a real Pulp: the src repository survived removal.

The first version still identified a distribution by its name ({repo_name}-distro) and a repository by the pulp_href stored in our DB. Both are assumptions rather than facts — a distribution whose name doesn't follow the convention is never matched, and a repository our DB has a stale href for is never found.

Removal now asks Pulp instead:

  • repositories and distributions are listed from all four endpoints (repositories/rpm/rpm, repositories/file/file, distributions/rpm/rpm, distributions/file/file) with the {owner}-{product}- prefix;
  • a repository is selected when its name is exactly one of the product's repository names, and it is deleted by the href Pulp reports, not the one in our DB;
  • a distribution is selected when the repository it serves is one of those, with the {repo_name}-distro name as a fallback for distributions serving a publication rather than a repository.

get_file_repositories() was added to PulpClient alongside get_file_distros().

The regression test now gives the src repository's distribution a name that breaks the convention and asserts it is still deleted, and adds a similarly-named foreign repository to both listings to confirm exact-name selection excludes it.

./run-tests.sh — 85 passed, 10 skipped.

@anfimovdm
anfimovdm merged commit 8ab1456 into master Jul 29, 2026
3 checks passed
@anfimovdm
anfimovdm deleted the fix-product-removal-pulp-cleanup branch July 29, 2026 21:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix product removal for products that start with the same name

1 participant