AI Incident Response: The First Hour
TL;DR
When AI behavior causes harm or near-miss, treat it as an incident. Stabilize the system, preserve evidence, classify severity, notify stakeholders, and start corrective actions. Capture prompts, retrieved context, model/version, tool calls, approvals, and logs. Align with your AIMS and security IR processes.
Key Facts
Incidents include security, safety, compliance, or ethical harms tied to AI outputs.
The first hour prioritizes containment and evidence capture.
Logging must include prompts, context, outputs, tool calls, and approvals.
Reporting and CAPA are required under governance processes.
Post-incident reviews update controls and training.
Implementation Steps
Declare & page on-call → ticket.
Contain (disable features/tool calls) → change record.
Preserve prompts/context/logs → forensics bundle.
Classify severity & impact → SEV doc.
Notify stakeholders → comms record.
Triage root causes/hypotheses → triage notes.
Countermeasures (filters, gates) → hotfix record.
Monitor reoccurrence → metrics.
Document timeline → IR log.
CAPA & schedule review → actions, owners.
Glossary
- Incident
- Event causing or potentially causing harm to stakeholders or systems
- Containment
- Actions to prevent further harm or damage from an incident
- Forensics bundle
- Collection of evidence for incident investigation
- Severity
- Assessment of incident impact and urgency
- CAPA
- Corrective and Preventive Actions - systematic response to incidents
- Post-incident review
- Analysis of incident response to improve future handling
References
- [1] ISO 42001 AI Management Systems Standard https://www.iso.org/standard/78380.html
- [2] NIST AI Risk Management Framework https://www.nist.gov/itl/ai-risk-management-framework
Machine-readable Facts
[
{
"id": "f-incident",
"claim": "AI incidents include harms tied to AI outputs across security, safety, and compliance.",
"source": "https://www.nist.gov/itl/ai-risk-management-framework"
},
{
"id": "f-evidence",
"claim": "First-hour actions prioritize containment and evidence capture.",
"source": "https://www.iso.org/standard/78380.html"
},
{
"id": "f-capa",
"claim": "Governance requires corrective and preventive actions with owners and deadlines.",
"source": "https://www.iso.org/standard/78380.html"
}
]