Skip to content

feat: prepare external API persistence foundation - #750

Open
bballdavis wants to merge 6 commits into
DialmasterOrg:devfrom
bballdavis:feature/external-api-foundation
Open

bballdavis wants to merge 6 commits into
DialmasterOrg:devfrom
bballdavis:feature/external-api-foundation

Conversation

@bballdavis

@bballdavis bballdavis commented Jul 31, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • adds the durable persistence foundation for external API policy, channel grants, requests, and quota accounting
  • preserves existing API keys as legacy_download while keeping current creation, hashing, validation, usage tracking, deletion, and single-video behavior unchanged
  • creates new tables directly in their final shape and registers their Sequelize associations normally

Part of #568.

Scope boundaries

This phase exposes no external routes, authentication middleware, request execution, management endpoints, rate-limit enforcement, feature flags, Swagger contract, or frontend UI.

Migration safety

  • rebased onto the current dev baseline
  • four focused migrations extend API keys and create channel grants, external requests, and usage buckets
  • obsolete pre-normalization compatibility markers have been removed
  • existing duplicate channel rows are left untouched; this PR adds no channel identity constraint or consolidation
  • catalog-table and query-specific request indexes are deferred to the runtime PR, where they can be reviewed with the consuming queries and EXPLAIN evidence
  • existing keys receive safe legacy defaults, and rollback deactivates external-role keys before removing their policy fields
  • partial migration and rollback states tolerate missing policy columns and indexes

Validation

  • backend and frontend lint plus TypeScript checks passed
  • 197 backend suites and 5,656 tests passed with zero skipped; backend line coverage is 90.71%
  • 277 frontend suites and 5,155 tests passed with zero skipped; frontend line coverage is 89.06%
  • the dedicated migration lifecycle passed on MariaDB 10.3 and MySQL 8.0 (one executed test per engine, zero skipped), covering legacy-key preservation, duplicate channels, partial recovery, final schema and collation validation, rollback, and reapply
  • documentation checks and build passed; the required All Checks workflow passed on PR head 5358ac1b
  • package.json adds only the dedicated database test script; package-lock.json remains aligned with current dev

@dialmaster dialmaster left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main blockers for this PR (see my other comments):

  • the global brace-expansion override breaks minimatch
  • the channel dedupe migration can leave an install unable to start
  • the eight 20260726 no-op migrations need to go
  • squashing the migrations together (eg, no need for multiple migrations for the same new table)

Comment thread package.json Outdated
Comment thread migrations/20260908107000-unique-channel-identity.js Outdated
Comment thread migrations/20260908107000-unique-channel-identity.js Outdated
Comment thread migrations/20260908100000-add-external-api-key-policy.js
Comment thread migrations/20260908103000-expand-external-request-types.js Outdated
Comment thread migrations/20260908104000-add-external-catalog-indexes.js Outdated
Comment thread server/models/index.js Outdated
@bballdavis
bballdavis force-pushed the feature/external-api-foundation branch from 27ba03e to c5ed12d Compare September 15, 2026 02:44
@bballdavis
bballdavis force-pushed the feature/external-api-foundation branch from c5ed12d to e2d0a7f Compare September 15, 2026 03:44

@dialmaster dialmaster left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my comment on line 23 of migrations/20260908100000-add-external-api-key-policy.js, just one last minor thing and then I think this will be good to approve :)

Comment thread migrations/20260908100000-add-external-api-key-policy.js Outdated
Comment thread migrations/__tests__/externalApiDatabase.integration.test.js Outdated
Comment thread docs/AUTHENTICATION.md Outdated
Comment thread migrations/20260908106000-create-external-api-usage-buckets.js Outdated
bballdavis added a commit to bballdavis/Youtarr that referenced this pull request Sep 29, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants