Skip to content

perf: start the apply sooner and run Terraform with parallelism 20 - #27

Merged
DaRoszko merged 1 commit into
mainfrom
perf/faster-apply
Oct 29, 2026
Merged

DaRoszko merged 1 commit into
mainfrom
perf/faster-apply

Conversation

@DaRoszko

Copy link
Copy Markdown
Contributor

What

Cuts the time of every platform and tenants apply. There are two changes, and neither alters isolation, control flow or permissions.

  1. The admin check is now a step, not a job. It is the first step of resolve, before any App token is minted or anything is read, so the order and the refusal are unchanged. This saves one job start and its queue gap, about 7 s per run (run 40928903653 had access 18:14:25 to 18:14:30 and resolve starting 18:14:32).
  2. Terraform plan and apply run with -parallelism=20 (the default is 10).

Why it is safe

  • Permissions: the check now runs in a job whose token also has deployments: read. The check does not use it, and anyone who can read the repository can already read it. No secret is exposed before the check: the App key reaches only the token step that comes after it.
  • Re-runs: the apply job still re-checks the actor on every attempt.
  • Pause gate stays its own job: AIP_PAUSED exists only in the {env}-events Environment, and a job has one Environment. Merging the gate would mean moving the switch, which is a control change.
  • No dependency on the job name: no ruleset, caller or required check refers to apply / access (checked across tenants, deployments, infra and workflows).

Why parallelism 20

  • Measured reads: the Infradev refresh (20 teams, about 60 ARM reads in 13 s at parallelism 10) runs at about 2 s per read.
  • ARM headroom: 20 in flight is about 9 reads a second. ARM allows 25 a second per principal per subscription, with a bucket of 250 (Learn, Understand how Azure Resource Manager throttles requests, checked 29 October 2026). At 50 teams (about 200 reads) that stays well inside the limit.
  • Why not 25: 25 would save only about 3 to 5 s at 50 teams and leave less headroom for the backend and retries.
  • Concurrent writes: the providers already serialise writes to one parent. azurerm locks the Foundry account for deployments and projects, and azuread locks the application for federated credentials (provider source checked), so concurrent writes to the same account or registration do not conflict.

Tests

  • tests/unit/workflows.test.js: the check comes before the token step and before resolve-apply, and there is no access job.
  • tests/integration/scripts/apply.test.js: plan and apply pass -parallelism=20.
  • Results: all unit tests pass (469); the workflows and apply.sh suites pass (97); lint, shellcheck and actionlint pass.

Release

perf: gives a patch release. It reaches tenants and deployments at @v2 with no change to callers' inputs, secrets or permissions.

The admin check is now the first step of the resolve job, before any token is minted or anything read, instead of a job of its own: one job start fewer (about 7 s) with the same order and the same refusal. Terraform plan and apply run with -parallelism=20 (default 10): measured refresh runs at about 2 s per ARM read, so 20 is about 9 reads a second, well inside ARM's 25 a second per principal for 50 teams; azurerm and azuread serialise writes to one Foundry account or one registration, so concurrent writes do not conflict. The pause gate stays its own job, as AIP_PAUSED lives only in the {env}-events Environment.
@DaRoszko
DaRoszko requested a review from a team as a code owner October 29, 2026 21:40
@DaRoszko
DaRoszko requested a balanced review from Copilot and removed request for a team October 29, 2026 21:40

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues block the workflow startup optimization and Terraform concurrency changes.

No open issues 🎉

What changed in this PR

Reduces startup overhead and increases Terraform concurrency in the reusable apply workflow used by tenants and deployments, without changing caller inputs or secrets.

Changes:

  • Moves the initial admin check into the first step of resolve, removing a separate job while retaining the apply-time recheck.
  • Sets Terraform plan and apply parallelism to 20.
  • Updates tests to verify admin-check ordering and the Terraform arguments.
File Description
tests/​unit/​workflows.test.js Verifies the admin check precedes token creation and resolution, with no separate access job.
tests/​integration/​scripts/​apply.test.js Checks that plan and apply pass -parallelism=20.
scripts/​apply.sh Sets and uses parallelism 20 for Terraform plan and apply.
.github/​workflows/​apply-state.yml Merges the initial admin check into resolve and updates rerun comments.

Review effort: Balanced

@DaRoszko
DaRoszko merged commit 4534e21 into main Oct 29, 2026
3 checks passed
@DaRoszko
DaRoszko deleted the perf/faster-apply branch October 29, 2026 21:58
ai-platform-orchestrator Bot added a commit that referenced this pull request Oct 29, 2026
🤖 I have created a release *beep* *boop*
---


##
[2.0.4](v2.0.3...v2.0.4)
(2026-10-29)


### Performance Improvements

* start the apply sooner and run Terraform with parallelism 20
([#27](#27))
([4534e21](4534e21))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: ai-platform-orchestrator[bot] <333351233+ai-platform-orchestrator[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants