This is a ready-to-use periodic operational record for a pharmacovigilance AI model already in production. A safety model degrades quietly: new products launch, reporting channels shift, language evolves, and none of it changes the model’s code, so the validated state has to be demonstrated on a schedule rather than assumed from release day. This record ties together the four signals that matter most for a PV model: labeled-sample recall and precision on the safety-relevant class, the rejection-sampling false-negative result for a screening model, the MedDRA version status for a coding model, and the reviewer override rate as an automation-bias check. Replace every <<FILL: ...>> placeholder with your own specifics and route it through your normal document control. A worked filled specimen follows the template. This content is educational reference, not legal or regulatory advice.
Document control header
| Field | Entry |
|---|---|
| Document title | Periodic AI Model Performance and Rejection-Sampling Review |
| Document number | <<FILL: RPT-ID, e.g. RPT-PV-AI-2026-07>> |
| Model / system and version | <<FILL: MODEL NAME + version>> |
| Risk pattern | <<FILL: A / B>> |
| Review period | <<FILL: from>> to <<FILL: to>> |
| Author | <<FILL: name / role>> |
1. Purpose and period activity
State plainly what was reviewed this period: the labeled-sample performance check, the rejection-sampling cycle (if the model is Pattern B), the MedDRA version status (if the model codes), the override-rate trend, and any triggered investigation. <<FILL: two to four sentences on the period>>
2. Labeled-sample performance versus the release baseline
| Metric | Release baseline | Alert threshold | Observed this period | Met threshold | Note |
|---|---|---|---|---|---|
| Recall, safety-relevant class | <<FILL>> | <<FILL>> | <<FILL>> | Yes / No | <<FILL>> |
| Precision, safety-relevant class | <<FILL>> | <<FILL>> | <<FILL>> | Yes / No | <<FILL>> |
| Confidence calibration | <<FILL>> | <<FILL>> | <<FILL>> | Yes / No | <<FILL>> |
| Sample size and source | <<FILL: count and how drawn>> | ||||
| Labeler(s) and qualification | <<FILL>> |
3. Rejection-sampling outcome (Pattern B models only)
| Field | Entry |
|---|---|
| Items rejected by the model this period | <<FILL>> |
| Sample size drawn for re-review | <<FILL>> |
| Sampling reviewer(s) | <<FILL>> |
| True misses identified | <<FILL: count and reference(s)>> |
| Measured false-negative rate | <<FILL>> |
| Threshold | <<FILL>> |
| Met threshold | Yes / No |
| Each true miss: routing and outcome | <<FILL: case/article reference, routing date, reporting-clock impact assessed>> |
4. MedDRA version status (coding models only)
| Field | Entry |
|---|---|
| MedDRA version pinned at last validation | <<FILL>> |
| Current MedDRA version in production use | <<FILL>> |
| Version change since last record | Yes / No |
| If Yes, impact assessment reference | <<FILL>> |
| Coded output correctly version-stamped for the period | <<FILL: percentage or "100%">> |
5. Reviewer override rate and automation-bias check
| Field | Entry |
|---|---|
| Total model outputs reviewed this period | <<FILL>> |
| Overrides | <<FILL>> |
| Override rate | <<FILL>> |
| Trend versus prior period | <<FILL: rising / stable / falling>> |
| Sustained near-zero override pattern investigated | <<FILL: Yes/No, and outcome>> |
| Sustained near-total acceptance pattern investigated | <<FILL: Yes/No, and outcome>> |
6. Input-distribution and negation-set check
| Field | Entry |
|---|---|
| Input-distribution drift vs training profile | <<FILL: distance measure and limit>> |
| Breach (Y/N) | <<FILL>> |
| Negation/uncertainty fixed set re-run this period | <<FILL: Yes/No>> |
| Negation/uncertainty outcome | <<FILL: pass/fail, count>> |
7. Triggers and response this period
| Trigger | Fired (Y/N) | Response taken | Reference |
|---|---|---|---|
| Labeled-sample recall below spec | <<FILL>> | <<FILL>> | <<FILL>> |
| Rejection-sampling false-negative rate above threshold | <<FILL>> | <<FILL>> | <<FILL>> |
| Input-distribution or calibration drift | <<FILL>> | <<FILL>> | <<FILL>> |
| Override-rate anomaly (either direction) | <<FILL>> | <<FILL>> | <<FILL>> |
8. Conclusion and disposition
- Model remains in its validated state; no action beyond routine monitoring.
- Degradation observed within tolerance; monitoring tightened, cause noted.
- Performance or rejection-sampling result below spec; model paused or routed to fuller human review; investigation opened.
| Field | Entry |
|---|---|
| Overall disposition and basis | <<FILL>> |
| Author (name, signature, date) | <<FILL>> |
| QA review (name, signature, date) | <<FILL>> |
| QPPV or delegate informed (if a trigger fired) | <<FILL: Yes/No, date>> |
Acceptance criteria
- Every applicable section (2 through 6) is completed for the period, with sample sizes and sources stated, not just headline numbers.
- Every triggered response in section 7 is recorded with a reference, not only observed.
- A rejection-sampling true miss results in the affected item being routed and its reporting-clock impact assessed, every time.
- The record is signed by the author and reviewed by QA, and retained per
<<FILL: retention period>>.
References
ICH Q9(R1), Quality Risk Management, for the risk basis of thresholds and cadence. 21 CFR Part 11 and EU GMP Annex 11, for this record as a controlled electronic GxP record. FDA and EMA, “Guiding Principles of Good AI Practice in Drug Development” (published jointly 14 January 2026), for lifecycle monitoring expectations. The model’s validation protocol and release baseline that define the specification these signals are compared against.
Confirm the current version of each reference before use.
Revision history
| Version | Date | Author | Summary of change |
|---|---|---|---|
<<FILL: 1.0>> | <<FILL: date>> | <<FILL: author>> | Initial issue. |
Approvals
| Role | Name | Signature | Date |
|---|---|---|---|
| Author | <<FILL>> | ||
| Approver (QA) | <<FILL>> |
Filled specimen
A completed monthly record for an example literature-screening model (Pattern B). Illustrative only.
Model: literature-screen v3.0. Risk pattern: B. Review period: 01 July 2026 to 31 July 2026. Author: T. Okafor.
Period activity: Labeled-sample recall held above spec. Rejection sampling found one true miss out of a 150-item sample, within threshold but investigated for cause. No MedDRA dependency (this model does not code). Override rate stable. No input-distribution breach.
| Metric | Release baseline | Alert threshold | Observed | Met | Note |
|---|---|---|---|---|---|
| Recall, reportable-article class | 0.93 | below 0.90 | 0.92 | Yes | Stable |
| Precision, reportable-article class | 0.71 | below 0.60 | 0.74 | Yes | Improved slightly |
Rejection sampling: 4,120 articles screened out this period; sample of 150 drawn; independent reviewer found 1 true miss (a case report in a regional-language journal describing a suspected interaction). False-negative rate 0.7%, threshold 3%, met. The missed article was routed to literature review the same day; no case was ultimately reportable from it, but the near-miss traced to a language the screening corpus underweights, and a corpus-expansion action was opened, tracked outside this record under change control.
Override rate: 210 flagged articles reviewed, 14 overridden, rate 6.7%, stable versus the prior three months (range 5 to 8 percent). No automation-bias pattern found.
Disposition: Model remains in its validated state. Degradation observed within tolerance (the language-coverage gap); monitoring tightened by adding two regional-language journals to the sampling frame for the next two cycles. Author T. Okafor, signed 05 August 2026. QA review R. Odusanya, signed 06 August 2026.
Reading it: the model passed every numeric threshold this period, and the record still surfaces a real gap, the underweighted regional-language coverage, because the rejection sample looked past the aggregate pass/fail into the one miss it found. That is the difference between a record that confirms a number and one that actually functions as ongoing surveillance.
Common inspection findings this record prevents
- A production PV AI model with no periodic performance evidence, so the validated state is an assertion from go-live day.
- A rejection-sampling program that runs but whose misses are never traced to a reporting-clock or signal impact assessment.
- An override rate tracked with no defined response when it moves, so drift is visible in the data but never acted on.
- A MedDRA version change that occurred with no documented impact assessment on the coding model’s output.
- Records that state only the headline pass/fail and omit sample size, so the number cannot be judged for statistical weight.
How to adapt this record
- Set your model, thresholds, and cadence from your validation protocol and risk assessment, not generic defaults.
- Drop sections that do not apply (MedDRA version status for a non-coding model, rejection sampling for a Pattern A model) rather than leaving them blank; state “not applicable” and why.
- Point the trigger references to your real investigation and change-control procedures.
- For a generative or narrative-drafting model, add a factual-consistency or unsupported-claim rate row to section 2.
- File each completed record with the system’s periodic review evidence, and route any true miss immediately rather than waiting for the record to close.