Skip to content

feat(changelog): Add configurable AI summaries for CLI and Actions - #902

Merged
BYK merged 21 commits into
masterfrom
codex/luna-changelog-summaries
Oct 15, 2026
Merged

BYK merged 21 commits into
masterfrom
codex/luna-changelog-summaries

Conversation

@MathurAditya724

@MathurAditya724 MathurAditya724 commented Oct 13, 2026 •

Copy link
Copy Markdown
Member

Adds opt-in AI summaries to automatic changelogs in the CLI and GitHub Actions. Choose release, category, and scope summaries independently; hidden lower-level digests are reused for higher levels. Both surfaces use one HTTP client, defaulting to Copilot Responses with GPT-6 Luna, with custom Responses or Chat Completions endpoints supported.

changelog:
  policy: auto
  aiSummary:
    levels: [release]
    minItems: 5
    style: Concise and friendly for developers. # optional
    llm:
      endpoint: https://api.githubcopilot.com/responses
      model: gpt-6-luna
      apiKeyEnv: GITHUB_TOKEN

An aiSummary object enables generation; omission or aiSummary: false disables it. Omitted or empty levels use [release]. Summaries appear when original entry counts exceed minItems (default 5), including descendants for categories and the entire changelog for release summaries. A threshold of 0 includes every nonempty selected group. Release-only output leaves originals expanded. Category prose and scope headings/summaries stay visible, with originals folded once per group. Release overviews are plain prose in up to three paragraphs and include every category, and source-linked breaking/security facts survive reductions.

Optional style supplies 1–500 characters of tone guidance at every generation stage while retaining factual, coverage and inert-output safeguards. Craft enables all three levels in its own .craft.yml with minItems: 2 and a concise, friendly writing style for dogfooding. Generation still requires an authorized credential and, for built-in Actions tokens, the organization policy described below; denied requests retain original output.

Connection settings live under llm; credentials come from the named environment variable (default GITHUB_TOKEN). Per-field precedence is explicit CLI flags, structured CRAFT_AI_LLM, scalar environment overrides, then effective project/workspace configuration. The Action accepts llm and an explicit llm_api_key; reusable release workflows forward both. Repository/organization variables are mapped in the calling workflow. Credentials are excluded from argument tracing. Craft sends HTTP requests directly, with no additional runtime installation or release artifact.

Copilot model access depends on the supplied credential. Explicit built-in workflow-token use requires copilot-requests: write and “Allow use of Copilot CLI billed to the organization,” with organization billing. The llm_api_key input selects a separate credential without replacing the release token. Copilot's direct inference backend is outside GitHub's public REST API contract. Custom endpoints receive no Copilot-specific headers, and failures never switch endpoints or providers.

Requests have fixed input/output, concurrency, count, and deadline limits. Strict structured-response validation and escaped generated prose keep summaries inert. Missing credentials, errors, timeouts, refusals, or incomplete coverage retain the entire original output. Dry runs, highlighted previews, --no-changelog, and version analysis make zero model calls; version-bump analysis remains independent.

Validation: 301 focused regressions, lint, type checking, formatting and the documentation build pass locally. The E2E runs under pnpm test in the regular Node 22/24 suites. It builds privately and exercises paths containing spaces, both HTTP formats, configuration precedence and original-output fallback. Regressions cover every level combination, visible scope headings, mixed groups, retained scope labels in reused digests, bounded writing styles and a ten-category prose overview. Final-head CI passes on f512a005383a10841b4bd3fe56e20925cfbe1670, including both Node suites, the native smoke test, container build, security scans and automated reviews. The two publication jobs are skipped for pull requests. Documentation includes release, category, scope and combination examples.

Live CLI samples passed Copilot Luna and OpenRouter Luna requests. The OpenRouter sample on b3769873 passed eight HTTP 200 responses with the revised config, preserving every original entry and the version bump while dry-run, missing-token and aiSummary: false cases made zero calls. Source metadata was a fixture; inference was live. The isolated built-in Actions-token run received HTTP 403 because the required organization policy was disabled. The real CLI and composite Action preserved original entries after the denial; Git pushes stayed local and publishing was stubbed. This is fallback proof, not successful workflow-token inference. Live OpenRouter inference in Actions remains untested. No organization policies were changed, and the live-test workflow is outside this PR.

Generate opt-in summaries in the actual changelog serializer with one bounded structured request. Preserve original entries and keep releases independent of model availability.

Inspired by #688; uses OpenRouter instead of the retired GitHub Models service.
@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-10-15 19:18 UTC

Reuse bounded scope and category digests for complete release overviews. Preserve original entries and critical facts, and keep version analysis AI-free.

Read CRAFT_AI_API_KEY and default to release-only summaries.
@MathurAditya724 MathurAditya724 changed the title feat(changelog): Add Luna summaries through OpenRouter feat(changelog): Add hierarchical Luna summaries through OpenRouter Oct 13, 2026
@MathurAditya724 MathurAditya724 changed the title feat(changelog): Add hierarchical Luna summaries through OpenRouter feat(changelog): Add hierarchical Luna summaries with Copilot Oct 13, 2026
@MathurAditya724 MathurAditya724 changed the title feat(changelog): Add hierarchical Luna summaries with Copilot feat(changelog): Add configurable Luna summaries with Copilot Oct 13, 2026
Remove the alternate provider and process documents. Keep caller release tools unchanged by installing the SDK with a private, integrity-pinned pnpm distribution.
Ship the pinned SDK and native runtime with release artifacts instead of installing packages in callers. Keep the runtime outside the ordinary npm package and retain immutable nightly compatibility.
@MathurAditya724 MathurAditya724 changed the title feat(changelog): Add configurable Luna summaries with Copilot feat(changelog): Add Luna summaries with a bundled Copilot runtime Oct 14, 2026
Comment thread src/__tests__/copilot-runtime.test.ts Fixed
Keep critical PR titles in model input when custom entries replace them, reject negative claims in critical facts, and neutralize generated protocol links. Share artifact integrity enforcement and exclude runtime assets from unpublished CDN registry links.
Bundle the JavaScript client and stdio factory while retaining the matching native runtime, authentication libraries, and license notices. Verify structured sessions and cancellation against a loopback fixture in the platform matrix.
@MathurAditya724 MathurAditya724 changed the title feat(changelog): Add Luna summaries with a bundled Copilot runtime feat(changelog): Add Luna summaries with a minimal Copilot runtime Oct 14, 2026
Ship only native runtime assets separately and launch them through the SDK stdio path option. Preserve license notices and verify the embedded SDK and standalone Craft across platforms.
Comment thread .github/scripts/ai-summary-smoke.mjs Fixed
Comment thread .github/scripts/ai-summary-smoke.mjs Fixed
@MathurAditya724 MathurAditya724 changed the title feat(changelog): Add HTTP summaries with Copilot Luna defaults feat(changelog): Add configurable AI summaries for CLI and Actions Oct 14, 2026
Comment thread .github/workflows/build.yml Outdated
Comment thread docs/src/content/docs/configuration.md Outdated
Comment thread docs/src/content/docs/configuration.md Outdated
Comment thread docs/src/content/docs/configuration.md Outdated
Comment thread docs/src/content/docs/configuration.md Outdated
Comment thread docs/src/content/docs/configuration.md Outdated
Comment thread docs/src/content/docs/configuration.md

@BYK BYK left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also enable this feature in Craft itself as part of the PR for demonstration purposes. I'd say we should go with minItems: 2 or less here.

Comment thread .github/workflows/build.yml Outdated
Comment thread docs/src/content/docs/configuration.md Outdated
Retain scope labels in parent requests and fold each original entry once.
Enable summary dogfooding with minItems: 2 and run the CLI E2E in the ordinary Node suites.
@MathurAditya724

Copy link
Copy Markdown
Member Author

Enabled AI summaries in Craft’s own .craft.yml in d392453, using levels: [release, category, scope] and minItems: 2 (groups with more than two original entries). The rendering now keeps category and scope prose visible and folds each original group once. All 294 focused regressions pass, including the standalone CLI E2E.

Generation still requires an authorized credential and the documented Copilot permission/policy for built-in workflow-token access; denied access retains the original changelog. No organization policies were changed.

@BYK BYK left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember talking about adding a "style" config option too or a prompt override/addendum to give some personality to the output when desired. Where's that?

Comment thread docs/src/content/docs/configuration.md Outdated
Allow bounded tone guidance at every summary level while preserving the base safeguards.
Render validated release statements as up to three paragraphs without category labels.
@MathurAditya724

Copy link
Copy Markdown
Member Author

Added changelog.aiSummary.style in f512a00 and enabled a concise, friendly developer style in Craft’s own config. It accepts 1–500 characters of tone guidance, applies through both CLI and Actions at every generation stage, and retains the factual, coverage, length, structured-response and inert-prose safeguards. The setting and examples are documented; schema and standalone HTTP E2E regressions pass as part of 301 focused tests.

@BYK
BYK merged commit aaddaa4 into master Oct 15, 2026
26 checks passed
@BYK
BYK deleted the codex/luna-changelog-summaries branch October 15, 2026 19:17

This branch was successfully deployed

1 active deployment
false — f512a005 Deployed Oct 15, 2026 by MathurAditya724 via Build #3075
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants