Skip to content

fix(ai-red-teaming): make extraction/membership/inversion work in the TUI - #138

Merged
rdheekonda merged 1 commit into
mainfrom
fix/airt-extraction-surface-metrics
Sep 3, 2026
Merged

fix(ai-red-teaming): make extraction/membership/inversion work in the TUI#138
rdheekonda merged 1 commit into
mainfrom
fix/airt-extraction-surface-metrics

Conversation

@rdheekonda

Copy link
Copy Markdown
Contributor

Problem

Running the traditional-ML privacy attacks through the TUI agent did not match the SDK:

  • Extraction ran with an empty query pool -> 0 target queries, producing a bogus "0% fidelity, cloned" finding. The SDK path worked because the notebook explicitly fetches /pool and passes it in.
  • Membership inference had the same latent gap (needs member/non-member sets).
  • Model inversion had no agent tool at all - the dispatch map only had extraction/membership/evasion, so the agent could not run it (SDK-only).
  • Even when extraction ran, the agent's summary showed only fidelity/agreement/queries and reported per-class fidelity, the budget curve, and transfer as "missing follow-ups" - though the SDK computes them and they are persisted to the finding's metrics_detail.

Evasion was unaffected (it perturbs a single input, no external dataset).

Fix (capability-only)

  • Extraction (generate_extraction_attack): derive the query pool from the target's sibling /pool endpoint when none is given; fail loudly on an empty pool instead of a 0-query clone; surface the full SDK result to the agent (soft fidelity, KL divergence, per-class fidelity, fidelity-vs-budget curve, surrogate accuracy); actually run the transfer test (measure_transfer defaults on, exposed as a tool param).
  • Membership (generate_membership_attack): derive member/non-member sets from /members and /nonmembers when not supplied; fail loudly if either is empty.
  • Inversion (generate_inversion_attack): new agent tool (confidence / nes). Infers input_dim/input_shape from a /pool sample, reconstructs per class, reports per-class confidence + classes recovered.
  • Agent doc lists the four black-box classifier tools.

Bumps capability 1.12.0 -> 1.13.0 (immutable content requires a bump).

Tests

14 new unit tests (tests/test_attack_runner.py): pool/member-set derivation, fail-loud guards, rich-metric surfacing, transfer wiring, and the new inversion tool (dispatch registration, shape inference, per-class output, unknown-attack error).

Validation

  • pytest tests/test_attack_runner.py -> 152 passed, 1 pre-existing failure (TestGenerateMultimodalAttack::test_generates_compiling_script_with_image) confirmed failing on origin/main, unrelated to this change.
  • Pending: E2E via TUI against the ml-extraction-* targets to confirm extraction now queries the pool (non-zero fidelity), membership derives its sets, and inversion runs.

…st in the TUI

Via the TUI the agent ran extraction with an empty query pool -> 0 target
queries -> a bogus '0% clone' finding, while the SDK path (which fetches /pool)
worked. Membership had the same latent gap, and model inversion had no agent
tool at all.

- Extraction: derive the query pool from the target's sibling /pool when none is
  given, and fail loudly on an empty pool instead of reporting a 0-query clone.
  Surface the full result the SDK already computes (soft fidelity, KL divergence,
  per-class fidelity, fidelity-vs-budget curve, surrogate accuracy) to the agent,
  and actually run the transfer test (measure_transfer now defaults on, exposed
  as a tool param).
- Membership inference: derive member/non-member sets from the target's /members
  and /nonmembers endpoints when not supplied; fail loudly if either is empty.
- Model inversion: add generate_inversion_attack (confidence / nes) - it did not
  exist as an agent tool. Infers input_dim/input_shape from a /pool sample,
  reconstructs per class, and reports per-class confidence + classes recovered.

Bumps capability 1.12.0 -> 1.13.0. 14 new unit tests cover pool/member-set
derivation, fail-loud guards, metric surfacing, and the new inversion tool.
@rdheekonda
rdheekonda merged commit 8d13a55 into main Sep 3, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant