This is a ready-to-use risk assessment for a generative-AI assistant used to draft and summarize inside deviation, CAPA, and investigation workflows. It uses a failure-mode-and-effects (FMEA) structure and concentrates on the failure modes a deterministic-software assessment never had to consider. Replace every <<FILL: ...>> placeholder with your own specifics and adjust the scoring to your own scales. This content is educational and general; adapt it and verify it before use.
Assessment header
| Field | Entry |
|---|---|
| System / assistant | <<FILL: assistant name / ID>> |
| Intended use | Drafting and summarizing in deviation, CAPA, and investigation workflows, with mandatory human review; no autonomous quality decision |
| Assessment ID | <<FILL: RA-ID>> |
| Assessors | <<FILL: QA, System Owner, Data Science, IT>> |
| Date | <<FILL: date>> |
| Risk method | FMEA (Severity x Occurrence x Detection), per <<FILL: SOP-ID for quality risk management>> and ICH Q9(R1) |
1. Methodology
Each failure mode is scored for Severity (impact on product quality, patient safety, or record integrity if it reaches a record undetected), Occurrence (how often the failure mode is expected to arise), and Detection (how likely the human-review control is to catch it before the record is finalized, where a high score means poor detectability). The Risk Priority Number (RPN) is Severity x Occurrence x Detection. The human-review step is treated as the primary detection control throughout, because the whole approach depends on it. Scoring the failure modes as if review did not exist would overstate residual risk; scoring them assuming perfect review would understate it, so Detection is scored on how well a realistic, trained reviewer catches each mode.
2. Scoring scales
Severity (1-5)
| Score | Meaning |
|---|---|
| 5 | An undetected error could directly affect product disposition, patient safety, or a regulatory submission |
| 4 | Could produce an inaccurate GxP record used in a quality decision |
| 3 | Could weaken an investigation or CAPA without an immediate product impact |
| 2 | Minor record-quality impact, correctable at next review |
| 1 | Negligible |
Occurrence (1-5)
| Score | Meaning |
|---|---|
| 5 | Expected in routine use without a specific control |
| 3 | Occasional |
| 1 | Rare |
Detection (1-5, higher = harder to detect)
| Score | Meaning |
|---|---|
| 5 | A trained reviewer would usually miss it (for example a fluent, plausible fabrication) |
| 3 | A trained reviewer catches it with defined checks |
| 1 | Obvious; almost always caught |
3. The assessment
| # | Failure mode | Effect | Cause | S | O | D | RPN | Mitigation | Residual S/O/D | Residual RPN |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Confabulation: a fluent, confident, factually wrong statement in a draft | Inaccurate GxP record if adopted; investigation misdirected | Model draws on training data instead of supplied facts | 4 | 4 | 5 | 80 | Grounding to supplied facts; fact-by-fact verification in review (WI step 2); disclosure flag | 4 / 2 / 2 | 16 |
| 2 | Miscounted or invented number in a summary | Inaccurate trend record; false or missed signal | Language models are unreliable at counting/arithmetic over large inputs | 4 | 4 | 4 | 64 | Deterministic computation of counts; verify every number against source (WI step 3) | 4 / 2 / 2 | 16 |
| 3 | Premature cause in a deviation description | Investigation anchored on a wrong cause before it starts | Model speculates when not constrained to observations | 3 | 3 | 3 | 27 | Prompt constrains to observations; reviewer checks for prejudged cause (WI step 4) | 3 / 1 / 2 | 6 |
| 4 | Generic reflex CAPA (blanket retraining, “update the SOP”) not tracing to the cause | Ineffective CAPA; recurrence; repeat becomes its own finding | Model defaults to plausible generic actions | 4 | 4 | 3 | 48 | Reviewer rejects any action not traced to the confirmed root cause (WI step 4) | 4 / 2 / 2 | 16 |
| 5 | Model’s suggested cause accepted as the answer | Root cause effectively decided by the tool | Automation bias amplified by fluent output | 5 | 3 | 4 | 60 | Human evidence-weighing mandatory; cause attributed to named investigators; RCA never delegated | 5 / 1 / 2 | 10 |
| 6 | Fabricated “similar past deviation” citation | Investigation relies on a comparison that does not exist | Model invents a plausible reference | 3 | 3 | 4 | 36 | Every cited historical record confirmed to exist and be relevant before use | 3 / 1 / 2 | 6 |
| 7 | Confidential GxP content entered into a public AI tool | Data leaves control; confidentiality and integrity event | Staff use an unsanctioned tool for convenience | 4 | 3 | 4 | 48 | Sanctioned contained tool only; trained prohibition; monitoring/DLP where available | 4 / 1 / 3 | 12 |
| 8 | Silent truncation of a long input | Summary misses records; a real signal is buried | Input exceeds the tool’s context window | 3 | 3 | 4 | 36 | Reviewer confirms coverage; chunking with reconciliation; completeness check (WI step 5) | 3 / 2 / 2 | 12 |
| 9 | Vendor model change alters behavior in production | Validated state lapses silently; output quality shifts | Vendor updates the base model with no version bump on your side | 4 | 3 | 4 | 48 | Version pinning where available; vendor change treated as change control; re-confirm guardrails | 4 / 2 / 2 | 16 |
| 10 | Rubber-stamp review | The control the approach depends on is not actually performed | Reviewer approves polished drafts without engaging | 5 | 3 | 4 | 60 | Meaningful-review WI with per-step acceptance; edit capture; QA monitors edit-rate as a signal | 5 / 2 / 2 | 20 |
| 11 | Undisclosed AI use discovered in inspection | Finding for hidden, unassessed AI in a quality process | No disclosure flag or procedure | 4 | 2 | 3 | 24 | Mandatory AI-assistance flag; procedure defines permitted use; transparency in the record | 4 / 1 / 2 | 8 |
| 12 | No record of generation conditions | Basis of a draft cannot be reconstructed | Model version and prompt not captured | 2 | 3 | 3 | 18 | Capture model version and prompt/template version on the disclosure log where feasible | 2 / 1 / 2 | 4 |
4. Risk acceptance
Set an action threshold appropriate to your scales (for example, any residual RPN above <<FILL: threshold>>, or any residual Severity of 5 with Occurrence above 1, requires a further control or management sign-off). The two residual risks that stay highest in this assessment are rubber-stamp review (#10) and model-decided root cause (#5), both because their severity is inherent and the mitigation is human discipline rather than a code control. Manage them with the meaningful-review work instruction, reviewer training on these exact failure modes, and QA monitoring of the draft edit-rate as a leading indicator that review has gone shallow.
5. Residual risk statement
With grounding, mandatory fact-and-number verification, a trained meaningful-review control, a sanctioned contained tool, disclosure flagging, and change control over vendor model updates in place, the residual risk of the assistant in its stated intended use is <<FILL: acceptable / acceptable with monitoring>>. The approach is defensible because the assistant only drafts and summarizes, a named human owns every decision and record, and the highest residual risks are actively monitored rather than assumed away.
6. Approval
| Role | Name | Signature | Date |
|---|---|---|---|
| Assessment author | <<FILL>> | ||
| Quality Assurance | <<FILL>> | ||
| System Owner | <<FILL>> |
Common inspection findings this assessment addresses
- No documented risk assessment for an AI tool in use in quality workflows.
- A risk assessment that treats the assistant like deterministic software and never names confabulation, miscounting, or generic-CAPA failure modes.
- Mitigations listed but not tied to a specific, verifiable control (a work-instruction step, a disclosure field, a change-control trigger).
- No leading indicator (such as edit-rate monitoring) for the review control the whole approach depends on.
How to adapt this assessment
- Replace the scoring scales with your own quality-risk-management scales and re-score to them.
- Add or remove failure modes for the specific workflows and tool you deploy.
- Tie each mitigation to your real work instruction, disclosure log, and change-control procedure.
- Set your action threshold and route residual risks above it for management acceptance.
- Re-run the assessment on a vendor model change or an intended-use change.