AI System Measurement Framework

From Monitoring to Evidence: KPIs, Baselines and Statistical Defensibility

When it comes to the monitoring infrastructure, making that monitoring count as evidence is a different problem. Monitoring tells you the system is running. Evidence tells you it’s running correctly, consistently and in compliance with its governance commitments, and that someone not in the room could verify it. Three things create that difference: a defined baseline, a statistical framework for separating real deviation from noise and a retention trail that survives inspection.

Every system starts with a 90-day provisional baseline period at deployment. All eight KPIs are collected, but no alerts fire because no baseline exists yet. At day 90, the baseline is formally approved by the technical lead and entered into the AI Risk Register. It’s a primary ATO evidence artifact, not a configuration file.

Standard KPI Catalog

The standard measurement framework runs on eight KPIs, all drawn from existing logging infrastructure defined in Articles 4 and 5.

Confabulation rate: percentage of workflow runs producing flagged unverified outputs; rolling 30-day window; target <1%; alert on >0.5pp week-over-week increase or any single day >3× the rolling average. Maps to MS-2.5, MS-4.1.

Schema compliance rate: outputs passing schema validation per agent role; target ≥99% (99.9% for retrieval agents); alert if any role drops below 98% in 7 days or any role logs >5 failures in one hour. Maps to MS-2.5, SI-10.

HITL escalation rate: workflow runs triggering a HITL gate per workflow type; baseline from first 90 days; alert on >20% drop (possible bypass) or >50% increase (possible degradation). Maps to MS-2.6, PM-14.

HITL decision latency: time from gate trigger to reviewer decision at p50/p90/p99; recommended p90 <4 hr standard, <30 min urgent; alert on >25% SLA breach in any 7-day period. Maps to PM-14, IR-4.

Confidence score distribution: mean/SD/10th percentile per agent role over rolling 30 days; alert if mean drops >5pp below baseline or 10th percentile falls below the HITL escalation threshold. Maps to MS-2.5, CM-6.

Model behavioral drift index: composite weekly deviation score across output length, vocabulary diversity, schema compliance and confidence distribution; alert at >0.15 (investigate), re-certification trigger at >0.30. Maps to CM-6, MS-4.1.

Tool call anomaly rate: calls deviating from 90-day baseline usage patterns per agent role; target <0.1% steady state; alert on >0.5% in 24 hours or >3× normal frequency for any tool type in one hour. Maps to AC-3, AU-12, IR-4.

Sensitive data detection rate: documents where scrubbing pipeline detects at least one sensitive entity over rolling 30 days; alert on >30% drop (pipeline degradation) or >3× surge (new high-risk data source). Maps to SC-28, MP-6.

Baseline Governance

A baseline that can be changed without governance controls isn’t a baseline. It’s a moving target. Establishment requires technical lead sign-off and entry into the AI Risk Register. Updates require change advisory gate approval with documented rationale: what changed in the model, workflow or data source? Updates that can’t be explained by a system change are a governance finding. Historical baselines are retained permanently. Every KPI report shows current value, baseline value and deviation side by side.

Alert thresholds are starting points. Use rolling averages for high-variance KPIs rather than point-in-time comparisons, calculate confidence intervals and document the methodology. Cross-organizational benchmarks don’t yet exist at sufficient maturity; get the internal rigor right first.

Continuous Assurance and Monitoring

Monitoring without feedback loops generates information. It doesn’t govern anything. A system can drift toward a risk threshold for weeks with no governance response, because nobody connected the monitoring output to the governance inputs. Seven assurance domains need a complete circuit. For each: the signal source, how deviation is detected, how the program responds and what the finding feeds back into:

• Behavioral baseline deviation: weekly drift index per agent role; >0.15 triggers investigation within 5 business days; >0.30 initiates re-certification; findings update the risk register and trigger capability manifest review.
• Cross-agent consistency: for ensemble workflows, measure output consistency rate against baseline; drops require investigation; consistently wrong ensemble outputs trigger a red-team exercise.
• Tool call pattern anomaly: anomaly detection against 90-day baseline; >threshold sends CISO security alert within 4 hours; potential prompt injection activates AI Security Incident procedure (Article 4, Section 9).
• HITL bypass detection: every workflow reaching a trigger state validated against gate log; any bypass is a critical alert requiring workflow suspension and root cause documentation within 24 hours.
• Sensitive data detection anomaly: >30% drop below baseline triggers pipeline health check within 24 hours; >3× surge triggers data source review.
• Risk register currency: automated check for stale entries; entries >30 days past review date escalated to CAIO; new incident types require emergency register update within 5 business days.
• RMF profile currency: quarterly automated assessment of coverage state changes; updates triggered by system changes, incidents or regulatory developments under change advisory within 30 days.

The CISA 72-hour reporting pathway needs periodic verification, not just documentation. Verify portal credentials quarterly; maintain backup reporter designations; run an annual tabletop that specifically tests the external reporting step. Document results in the ATO evidence package. The clock starts at discovery, not when the organization determines it’s ready to file.

NIST AI RMF and 800–53 Mapping

NIST SP 800-53 Rev. 5 predates agentic AI architectures. That means some control mappings in this series are reasoned analogs rather than clean equivalents, and some interpretation is unavoidable. What isn’t defensible is presenting those analogs as canonical in formal ATO documentation. Each interpretive mapping needs narrative justification in the System Security Plan.

• Section 1 (Measurement Framework): advances MS-1.1, MS-2.5, MS-4.1 via AU-12, SI-10, CM-6. Function: MEASURE.
• Section 2 (Model Risk Management): advances MS-2.5, MS-4.1, MG-4.1, MG-4.2 via CM-3, CM-6, SA-10, IR-4. Functions: MEASURE, MANAGE.
• Section 3 (Data Governance): advances MP-3.4, MS-2.8–2.10 via SC-28, MP-6, AC-4, AU-9. Functions: MAP, MEASURE.
• Section 4 (HITL Maturity): advances MS-2.6, MG-2.2, MG-4.1, PM-14 via AC-3, AU-12, PM-14. Functions: MEASURE, MANAGE.
• Section 5 (Continuous Assurance): advances MG-2.2, MG-2.4, MG-4.1–4.2, MS-4.1 via AU-2, AU-12, IR-4, CA-7. Functions: MANAGE, MEASURE.
• Section 6 (Scalability): advances MG-2.1 (partial), MG-3.2, MG-4.2 via PM-9, SI-17, IR-5. Function: MANAGE.
• Section 7 (Assurance Evidence Model): advances GV-1.4, MS-2.7, MG-4.1, CA-2 via AU-9, AU-12, CA-2, IR-8. Functions: GOVERN, MEASURE, MANAGE.

Articles 1 through 7 close the majority of Framework Defined items across all four AI RMF functions. The two true Gaps from Article 1 (MP-2.3 and MG-2.1) remain out of scope. Section 6 moves the needle on MG-2.1 through the scalability governance guidance, but stops short of a full resource allocation framework.