This is a ready-to-use release-readiness checklist. Run it as the go/no-go gate before a generative AI system produces or drives any GxP record. It pulls the qualification method into a single signed decision, so a reviewer and QA can see at a glance whether every load-bearing control is in place. Replace each <<FILL: ...>> placeholder with your own specifics. A filled specimen follows. Mark each item Pass, Fail, or N/A with evidence; any Fail on a required item blocks release.
Document control header
| Field | Entry |
|---|---|
| Checklist title | Generative AI Qualification Release Readiness |
| System / use case | <<FILL: name>> |
| Version qualified | <<FILL: model + prompt + retrieval config version>> |
| Reviewer | <<FILL: role>> |
| Date | <<FILL: date>> |
1. Intended use and risk pattern
| # | Item | Pass/Fail/NA | Evidence |
|---|---|---|---|
| 1.1 | An intended-use statement names the generative output, the action it triggers, and the accountable role | <<FILL>> | |
| 1.2 | The use case is assigned to a generative risk pattern (drafting with human authorship, retrieval QA, structured extraction, or autonomous action), with an ICH Q9(R1) rationale | <<FILL>> | |
| 1.3 | If the output drives action with no per-output human review, the highest-risk controls are in place or release is refused | <<FILL>> |
2. Acceptance criteria
| # | Item | Pass/Fail/NA | Evidence |
|---|---|---|---|
| 2.1 | More than one metric is defined (groundedness, hallucination rate, completeness, consistency, abstention, accuracy where checkable), not a single accuracy number | <<FILL>> | |
| 2.2 | Each metric has a numeric threshold justified against the consequence of error | <<FILL>> | |
| 2.3 | The thresholds were recorded in the URS before any evaluation was run | <<FILL>> |
3. Golden dataset
| # | Item | Pass/Fail/NA | Evidence |
|---|---|---|---|
| 3.1 | The dataset is representative and includes edge, unanswerable, and adversarial cases | <<FILL>> | |
| 3.2 | Expected behavior was set by qualified SMEs under a labeling SOP with recorded inter-rater agreement | <<FILL>> | |
| 3.3 | The set is sized to support the thresholds with a stated confidence interval | <<FILL>> | |
| 3.4 | Development and locked-evaluation splits are separated; the locked split was not used for tuning | <<FILL>> | |
| 3.5 | The dataset is versioned and hashed | <<FILL>> |
4. Scoring and any LLM judge
| # | Item | Pass/Fail/NA | Evidence |
|---|---|---|---|
| 4.1 | The scoring method for each metric is defined | <<FILL>> | |
| 4.2 | Any LLM judge is qualified: measured agreement with human experts above a stated threshold, per category | <<FILL>> | |
| 4.3 | The human-scoring rubric and rater qualifications are on file | <<FILL>> |
5. Prompt and version control
| # | Item | Pass/Fail/NA | Evidence |
|---|---|---|---|
| 5.1 | Prompt, retrieval config, guardrails, model version, and sampling parameters are under version control | <<FILL>> | |
| 5.2 | The model version is pinned, not a “latest” alias | <<FILL>> | |
| 5.3 | Each qualification result references the exact configuration version it was produced under | <<FILL>> | |
| 5.4 | GxP outputs are traceable to the configuration that generated them | <<FILL>> |
6. Guardrails
| # | Item | Pass/Fail/NA | Evidence |
|---|---|---|---|
| 6.1 | Each guardrail is specified as a requirement and verified with scripted test cases, including negative and adversarial inputs | <<FILL>> | |
| 6.2 | Outputs are grounded in and traceable to retrieved sources | <<FILL>> | |
| 6.3 | Structured outputs are schema-validated; injection and out-of-scope inputs are filtered | <<FILL>> | |
| 6.4 | A human review gate exists for any output that becomes or drives a GxP record | <<FILL>> |
7. Production monitoring
| # | Item | Pass/Fail/NA | Evidence |
|---|---|---|---|
| 7.1 | A monitoring plan is live from day one with defined triggers (cadence, override-rate threshold, input drift, vendor version change) | <<FILL>> | |
| 7.2 | Re-evaluation on the locked golden dataset is scheduled and triggered by any configuration change | <<FILL>> | |
| 7.3 | A defined response exists when a monitor trips (who is notified, pause or route to fuller review, how it is investigated) | <<FILL>> |
8. Records and integrity
| # | Item | Pass/Fail/NA | Evidence |
|---|---|---|---|
| 8.1 | Any output that becomes part of a GxP record is treated under Part 11 and ALCOA+, not as informal | <<FILL>> | |
| 8.2 | Any embedded or vendor generative feature that touches a GxP process is inventoried and assessed, not hidden | <<FILL>> |
Disposition
| Field | Entry |
|---|---|
| Any required item Failed? | Yes / No (Yes blocks release) |
| Open items with justification | <<FILL: list or none>> |
| Decision | Release / Do not release / Release with restriction |
| Reviewer signature and date | <<FILL>> |
| QA approval signature and date | <<FILL>> |
References
21 CFR Part 11 and ALCOA+ for record integrity. ICH Q9(R1) (Quality Risk Management) for risk-based sizing of the qualification. FDA guidance on Computer Software Assurance for Production and Quality Management System Software (current version). GAMP 5 (second edition) risk-based principles; described in original wording, not reproduced.
Confirm the current version of each reference before you rely on it.
Filled specimen (extract)
Illustrative; replace with your own. Retrieval-QA assistant answering specification questions.
| # | Item | Result | Evidence |
|---|---|---|---|
| 2.1 | Multiple metrics defined | Pass | URS-AI-07: groundedness, hallucination, completeness, consistency, abstention, numeric accuracy |
| 3.4 | Splits separated, locked split not tuned | Pass | Dataset SPEC-AI-03: 120 dev / 280 locked, hashes on file |
| 4.2 | LLM judge qualified | Pass (partial) | Judge protocol PRO-AI-11: qualified for groundedness/completeness on answerable and unanswerable items; adversarial items human-scored |
| 6.4 | Human review gate present | Pass | Answer is reference only; controlled document remains the record; no disposition authorized by the assistant |
| 7.1 | Monitoring live from day one | Pass | Monitoring plan MON-AI-05: weekly override rate, monthly re-eval, model-version watch |
Disposition in the specimen: no required item Failed, the judge limitation on adversarial items is documented with human-scoring fallback, so the decision was Release with the monitoring plan active. The value of the checklist is that it forces the one question that sinks weak generative deployments to be answered before go-live: is there a real human gate, a locked golden dataset, a pinned model, and a live monitor, or just a single accuracy number.
Common inspection findings this checklist prevents
- A generative system released on a single accuracy number with no golden dataset behind it.
- A “latest” model alias in production, so the running system is not the qualified one.
- No monitoring, so a silent vendor model change degrades the system unnoticed.
- An embedded vendor generative feature writing into records, never assessed as a generative system at all.
How to adapt this checklist
- Set the system, qualified version, and reviewer in the header.
- Mark items that genuinely do not apply as N/A with a reason; do not leave them blank.
- Treat any Fail on a required item as a release blocker, and record any release-with-restriction decision and its basis.
- Run the checklist again after any configuration change that triggers re-evaluation, not only at first release.
- Confirm every regulation in the references against the current published version before you rely on it. This checklist is educational guidance for adapting your own qualification and release process; it is not legal, regulatory, or professional advice, and it does not itself certify or guarantee compliance with any regulation or guidance it references.