feat/prune age

[alanpeixinho]

Summary

Add prune_db, a manual management command to delete old data and reclaim space.

• Deletes checkouts, builds and tests older than a given age (e.g. --older-than "30 days").
• Manual cascade (models use DO_NOTHING): pruning a checkout removes its builds, pruning a build removes its tests — even when children are newer.
• --origins scopes the age filter; cascade stays origin-agnostic so no orphans are left.
• Batched deletes (--batch-size) that each commit on their own to keep locks short.
• --dry-run and a per-table confirmation prompt showing exactly how many rows will be affected (skip with --yes).
• Extracts parse_interval from update_db into a shared helper.

Usage

python manage.py prune_db --older-than "30 days" --dry-run                                                                                                                                                         
python manage.py prune_db --older-than "30 days" --origins maestro                                                                                                                                          

How to Test

• In a local database create newly seeded data poetry run python3 manage.py seed_data --clear --yes
• Run command poetry run python3 manage.py prune_db --dry-run with different arguments for older-than parameter.
• Verify the number of selected rows matches the database.
• Run command poetry run python3 manage.py prune_db with different arguments for older-than parameter.
• Verify the rows are properly deleted.

[felipebergamin]

a minor but I think that would be nice to validate the batch_size. Currently we are able to pass a negative value to the database.

django.db.utils.DataError: LIMIT must not be negative

[alanpeixinho]

Done.

[felipebergamin]

Worth noting in docs that aggregate tables are not touched by this command.

[alanpeixinho]

Done

[tales-aparecida]

@nuclearcat how long/far-back do we keep the raw KCIDB json payloads?

[mentonin]

Django has a built-in parser django.utils.dateparse.parse_duration that could be used here

[alanpeixinho]

I took a look on the built-in parser. Unfortunately it has different standards from what we are following.

• standard: "30 00:00:00" (%d %H:%M:%S)
• ISO 8601: "P30D"
• postgres: "3 days 04:05:06" (only works for days)
  I believe we might benefit more of keeping our custom parser, we could even extend it to include weeks or months, especially in a scenario where we are working with longer ranges.
  What you think?

[nuclearcat]

Each batch re-evaluates the full nested predicate. Add on top batch might be too small (we need to check how many tests per day we have, how long take deletion of 10k).
Consider materializing the ids once into a CREATE TEMP TABLE tobedeleted_... AS SELECT id .., and then batch-deleting by joining against it. Temp tables survive across the per-batch commits within the same session, so this fits the existing design.

[alanpeixinho]

Each batch re-evaluates the full nested predicate. Add on top batch might be too small (we need to check how many tests per day we have, how long take deletion of 10k). Consider materializing the ids once into a CREATE TEMP TABLE tobedeleted_... AS SELECT id .., and then batch-deleting by joining against it. Temp tables survive across the per-batch commits within the same session, so this fits the existing design.

Nice catch @nuclearcat . I have included a temp table to store ids before batch deleting.