For DevSecAIOps · the method
How we solve it.Piece by piece, with provenance.
How AI risk becomes testable inside your pipeline — the gate set, what each gate checks, and the one gate proven end to end.
The exhibits
The working surfaces behind the answer.
DEVSECAIOPS · GATE FRAMEWORK · v0.2
Eight gates. Three lanes. Three lenses.
Every gate checks code, AI systems, and AI artifacts, for security, safety, and privacy.
Design
PLAN · before any code or model work begins
Pre-commit
WORKSTATION · before code enters the repo
Source
COMMIT · PR · on every push and pull request
Build
ARTIFACT · producing the deployable thing
Test
VERIFY · evals are to AI what unit tests are to code
A model can pass security red-teaming and still fail its fairness threshold. Each eval suite blocks independently.
Deploy
ADMISSION · into the runtime environment
Permission manifest compiled to enforcement. Kill-switch verified before go-live.
Human-in-the-loop checkpoints for consequential decisions; DPIA sign-off as an admission condition.
Retention and data-residency config verified; model card published (tri-lens: security, safety, privacy).
Runtime
CONTINUOUS · not a one-time checkpoint
Agent behavior monitored against the manifest; drift from declared scope is an incident. Shadow-AI sweep.
Harm monitoring with an incident taxonomy, a user feedback channel, periodic eval re-runs.
Personal information (PII) egress classification, DSAR readiness, retention enforcement as a continuous check.
Attest
MEASURE · closes the loop, feeds the next Design gate
Maturity per gate: exists → automated → blocking → evidence-mapped → measured
Piece 1
The gate set
Inputs
- Your existing pipeline stages — the gates attach where your checks already run
Mechanism
- Each tollgate is defined by the AI failure mode it catches (injection, tool misuse, model swap, data leak), the stage it guards, and the evidence it must produce to pass
Outputs
- A gate list you can wire into CI, each with a stated check and a pass condition
Provenance: The tollgate model at /library/tollgates; gate ids are stable and deep-linkable.
The deeper layer — named here, shown in a walkthrough
- The full gate acceptance criteria and thresholds per gate
Piece 2
What each gate checks
Inputs
- The AI-specific failure modes your code gates never see
Mechanism
- Prompt injection is tested with an attack corpus, not a lint rule; model swaps are caught by integrity checks; tool misuse by scope diffs — each check is CI-shaped: deterministic input, binary verdict
Outputs
- Checks your pipeline can run on every change, like everything else you ship
Provenance: Each gate names its check on the tollgates page.
The deeper layer — named here, shown in a walkthrough
- The corpus-construction rules for the injection gate (attack classes, held-out discipline, benign set)
Piece 3
The worked gate
Inputs
- A live app’s own guard and a fixed attack corpus
Mechanism
- Gate #4 (prompt-injection red-team as CI) run end to end: baseline the shipped guard, fit it, re-score on the same corpus with a held-out set — then wire the scorer as a pipeline gate
Outputs
- A measured before/after (modeled, fixed corpus), and a reference implementation you can reproduce
Provenance: The fitted-guard case study (StoryBond) — every number derived from lib/storybond-demo.ts and labeled with its evidence state.
The deeper layer — named here, shown in a walkthrough
- The fit moves that took the guard from baseline to full catch, step by step
Piece 4
The reskilling list
Inputs
- The Track 2 reskilling master list: 247 AICM controls, each attributed to one persona owner with a skill verb and a capability prompt
- Your saved diagnostic answers, where a run exists: the same answers Track 1 scored
Mechanism
- Controls that gate the pipeline, guard the interfaces, manage change and config, and ship with provenance resolve to DevSecAIOps as primary
- Track 2 is the diagnostic inverted: a control absent or breached in a function’s answers is re-read as the named skill this role acquires. No second scorer, no second instrument
Outputs
- 21 controls resolving to this role across 4 skill groups — a named skill list: direction and next rung, never a complete how-to
Provenance: tools/reskill-persona-map.json → lib/reskill-master.ts (owner sample-review approved 2026-07-17); gaps join in lib/reskill-deltas.ts.
The deeper layer — named here, shown in a walkthrough
- The per-control attribution table for this role and the judgment notes on boundary controls
Piece 5
The stack & instruments
Inputs
- 16 published instruments in three lanes — 14 testers, 3 corpora, 5 controls — validated at every build
- Python 3 stdlib-only tooling — nothing to install to reproduce a claim
Mechanism
- Tester-lane instruments ARE the pipeline gates: the jailbreak-resistance scorer that graded the worked example is the same instrument you wire into CI — deterministic input, binary verdict
- Lane integrity is machine-enforced: an instrument never grades its own lane, and nothing on this site says "tested" without a named instrument behind it
Outputs
- A named, reproducible instrument behind every tested claim you read here
Provenance: lib/tools-registry.ts, guarded by validate-tools in the prebuild chain; instrument detail renders in the /controls drill.
The deeper layer — named here, shown in a walkthrough
- The per-instrument wiring: which tool validates which skill, and which tool grades which instrument
- Acceptance thresholds per tester lane
- The gate wiring recipe per CI system
Canonical surfaces