Log: Quality Event Classification Calibration Session Record

This is a ready-to-use log for a quality event classification calibration session, the exercise the triage and classification article describes: reviewers independently classify the same set of historical events, blind to each other and to the original tier, so drift between reviewers becomes visible and fixable instead of staying invisible until an inspector or a mis-tiered event finds it first. Replace every <<FILL: ...>> placeholder. A filled specimen follows.

Session record

Independent classification matrix

List every event used, its original recorded tier, and each reviewer’s independent tier. Use the risk-ranking worksheet as the scoring method where the event is not obvious, so the session also tests whether reviewers apply that tool consistently, not only whether they reach the same label.

Agreement definitions. Exact means every reviewer independently landed on the same tier as the original record. Adjacent means a reviewer landed one tier away (Major instead of Critical, or Minor instead of Major); worth a note, not a full divergence. Divergent means a reviewer’s tier was two levels away from the original (Critical read as Minor, or the reverse), which is the failure mode that matters most because it is the one that changes investigation depth, escalation, and disposition outright.

Agreement scoring

A session with no divergence recorded at all, across every reviewer and every event, is treated as a facilitation defect (reviewers likely were not genuinely blind or compared notes before submitting), not as a clean result, unless the facilitator documents specifically why none occurred.

Divergence analysis

For every adjacent or divergent result, trace the cause and the resulting action. Do this before the group is told the “right” answer, so the reasoning is captured while reviewers can still articulate why they scored as they did.

Reconciliation

Sign-off

Cumulative trend log (append one row per session)

Acceptance criteria

A calibration program is defensible when a defined cadence exists and sessions actually occur on it; the events used are real, span all three tiers, and are presented blind; every session produces the classification matrix, the three agreement rates, and a divergence analysis; every divergence traces to a stated cause and closes with a recorded action; and the cumulative trend log shows exact-tier agreement tracked session over session, not a single session filed and forgotten. See the tier distribution and timeliness trend log for the companion signal, the mix of tiers assigned to real events over the same period.

Retention

Retain calibration session records per <<FILL: retention period>>, per the records retention schedule, separate from the individual event classification records themselves.

References

ICH Q9(R1), Quality Risk Management (2023 revision), explicitly pushes to reduce subjectivity in risk assessment; calibration is how that is operationalized rather than asserted. ICH Q10, Pharmaceutical Quality System, continual improvement of the quality system, which a classification calibration program feeds. 21 CFR 211.192 (investigation of discrepancies and failures), the basis for expecting a consistent, defensible severity judgment across reviewers and shifts.

Confirm the current version of each reference before issue.

Filled specimen

The following is a worked calibration session using six historical events. Company, event details, and names are illustrative; replace them with your own.

Session record

Independent classification matrix

Agreement scoring

Divergence analysis

Reconciliation

Sign-off

Cumulative trend log

This session shows the program working as intended: no reviewer scored an event two tiers away from the original, agreement is trending up from the Q1 baseline, and each miss traced to a specific, fixable gap in the anchored examples rather than a reviewer being simply wrong.

Common inspection findings this log prevents

• A calibration program described in the SOP with no session records to show it actually runs.
• Sessions that report “good agreement” with no matrix, no rates, and no way to verify the claim.
• Divergent scoring found in a session and never traced to a specific, fixable cause in the criteria or the anchored examples.
• A calibration session used to quietly re-open and change a closed event’s classification without its own documented, evidence-based re-classification.
• No trend across sessions, so a slow decline in agreement is invisible until an inspector asks for the history.

How to adapt this log

1. Set your document number, owner, and cadence, and align the trigger list to your own program.
2. Choose your event set from real, closed events, not hypotheticals; spanning all three tiers is what makes the session diagnostic rather than reassuring.
3. Keep reviewers genuinely blind to each other and to the original tier; a facilitation shortcut here is the single most common way a calibration session becomes theater.
4. Set your own bar for what counts as an acceptable exact-agreement rate and revisit it as the program matures; a new program’s baseline is not the standard a mature program should be held to indefinitely.
5. Pair this log with the tier distribution and timeliness trend log in the periodic quality management review; reviewer agreement and tier-mix drift are different failure modes and neither substitutes for the other.