This is a ready-to-use risk assessment. Replace every <<FILL: ...>> placeholder with your own specifics. A worked filled specimen follows so you can see how a completed assessment reads. Verify each cited regulation against the current source before you rely on it.
Document control header
| Field | Entry |
|---|---|
| Assessment title | AI Training Data Bias and Representativeness Risk Assessment |
| Document number | <<FILL: RA-ID, e.g. RA-AI-027>> |
| Version | <<FILL: version>> |
| Effective date | <<FILL: date>> |
| Model / dataset assessed | <<FILL: model name and dataset version>> |
| Assessment owner | <<FILL: role, e.g. Data Science Lead>> |
1. Purpose and scope
This assessment identifies and scores sources of bias in a candidate training dataset that could cause an AI/ML model to perform unevenly across the population it will face in production, and it records the decision to remediate or explicitly scope out any subgroup the data under-represents. It applies to the dataset named above, at the point captured by the dataset version in the header, and is re-run whenever the dataset is materially re-extracted or the model is retrained on new data.
2. Methodology
This assessment scores each identified bias source on likelihood of affecting model behavior and severity of consequence if it does, following a risk-based approach consistent with ICH Q9 (Quality Risk Management). It is not a full FMEA; it is scoped specifically to data bias and representativeness, feeding into the broader AI risk assessment for the model as a whole (see AI risk assessment for GxP).
Likelihood scale
| Score | Definition |
|---|---|
| 1, Low | The gap is small and the affected population is a routine, well-understood case |
| 2, Medium | The gap is moderate, or affects a case the model will encounter regularly |
| 3, High | The gap is large, or affects a case central to the model’s intended use |
Severity scale
| Score | Definition |
|---|---|
| 1, Low | An error on the affected subgroup has minimal consequence and would be caught by routine review |
| 2, Medium | An error on the affected subgroup could affect a quality decision but with a compensating control in place |
| 3, High | An error on the affected subgroup could affect product quality, patient safety, or a release decision with no independent check |
Risk score = Likelihood x Severity. Score 1-2 is Low, 3-4 is Medium, 6-9 is High. A High score requires remediation or an explicit, QA-approved scope exclusion before the model proceeds to build or retrain.
3. Bias sources assessed
Assess each of the following categories against the actual dataset; not every category will apply to every model.
- Selection bias. The data captures only some of the population the model will face: one site, one product, one instrument, one time period, or one operating condition.
- Class imbalance. The outcome of interest is rare in the data, so naive accuracy is meaningless and the model can learn to default toward the majority class.
- Historical bias. The data reflects past practice, including past inconsistent categorization or past errors, which the model will reproduce faithfully rather than correct.
- Measurement bias. Different instruments, methods, or analysts produce systematically different readings that the model mistakes for a real difference in the underlying process or population.
- Label bias. Ground-truth labels were assigned inconsistently across labelers, time periods, or sites, so the “truth” the model learns is itself unreliable in places.
4. Assessment table
| # | Bias source | Category | Description of the gap found | Likelihood | Severity | Risk score / level | Mitigation | Residual risk |
|---|---|---|---|---|---|---|---|---|
| 1 | <<FILL>> | <<FILL: selection / imbalance / historical / measurement / label>> | <<FILL>> | <<FILL: 1-3>> | <<FILL: 1-3>> | <<FILL>> | <<FILL: remediate or scope out>> | <<FILL>> |
| 2 | <<FILL>> | <<FILL>> | <<FILL>> | <<FILL>> | <<FILL>> | <<FILL>> | <<FILL>> | <<FILL>> |
| 3 | <<FILL>> | <<FILL>> | <<FILL>> | <<FILL>> | <<FILL>> | <<FILL>> | <<FILL>> | <<FILL>> |
Add rows as needed. Every High-risk row must resolve to either a completed mitigation with re-measured residual risk, or a documented, QA-approved scope exclusion; neither may be left open at sign-off.
5. Subgroup-sliced evaluation requirement
Aggregate performance metrics hide subgroup failure. For every subgroup identified as a bias source above with Medium or High risk, the model’s performance must be reported sliced by that subgroup, not only in aggregate, as part of the model’s validation evidence (not this assessment). Record here which subgroups require sliced reporting and the metric to be used for each.
| Subgroup requiring sliced evaluation | Metric | Minimum acceptable performance | Reference |
|---|---|---|---|
<<FILL>> | <<FILL: e.g. recall on critical class>> | <<FILL>> | <<FILL: validation protocol ID>> |
6. Remediate versus scope-out decisions
For each subgroup where full remediation (collecting more representative data) is not immediately feasible, document the explicit decision to exclude it from the intended use rather than deploy the model against it silently.
| Subgroup | Remediate or scope out | Rationale | Where the exclusion is stated in the intended-use documentation |
|---|---|---|---|
<<FILL>> | <<FILL>> | <<FILL>> | <<FILL: reference to intended-use statement or SOP>> |
7. Acceptance criteria
- Every applicable bias category (selection, imbalance, historical, measurement, label) has been assessed, even where the finding is “not applicable, with rationale.”
- Every identified bias source is scored for likelihood and severity using the scales in section 2.
- Every High-risk finding has either a completed mitigation with re-measured residual risk, or a documented, QA-approved scope exclusion.
- Subgroups requiring sliced evaluation are named with a defined metric and threshold, feeding the model validation plan.
- The assessment is re-run when the dataset is materially re-extracted or the model is retrained on new data.
8. References
ICH Q9, Quality Risk Management, for the risk-based methodology this assessment follows. FDA, Data Integrity and Compliance With Drug CGMP, Questions and Answers (Guidance for Industry, December 2018), for the completeness and representativeness expectations underlying section 3. GAMP 5 Second Edition (ISPE, 2022), for the risk-based lifecycle context (reference by title, do not reproduce clauses).
Confirm the current version and clause numbers of each reference before issue.
9. Revision history
| Version | Date | Author | Summary of change |
|---|---|---|---|
<<FILL: 1.0>> | <<FILL: date>> | <<FILL: author>> | Initial issue. |
10. Approvals
| Role | Name | Signature | Date |
|---|---|---|---|
| Author (Data Science) | <<FILL>> | ||
| Reviewer (Process SME) | <<FILL>> | ||
| Approver (Quality Assurance) | <<FILL>> |
Filled specimen
The following shows the assessment completed for an illustrative automated visual-inspection model that screens injectable vials for cosmetic and critical defects. Numbers and findings are illustrative; replace them with your own.
| # | Bias source | Category | Description of the gap found | Likelihood | Severity | Risk score / level | Mitigation | Residual risk |
|---|---|---|---|---|---|---|---|---|
| 1 | Line 2 under-represented | Selection | Training images 88% from Line 1, 12% from Line 2; Line 2 has a different camera angle and lighting rig | 3 | 3 | 9, High | Collected 4,000 additional Line 2 images over 6 weeks; re-balanced to 55/45 | 2, Medium, re-assess after next production quarter |
| 2 | Critical defect class rarity | Class imbalance | Critical (fill-level) defects are 0.4% of labeled images | 3 | 3 | 9, High | Metrics shifted from accuracy to recall on the critical class, with a fixed minimum recall gate; targeted oversampling of critical-defect images during training | 4, Medium, recall gate carried into validation protocol |
| 3 | Labeler drift over 2 years | Label bias | Labeling guidance for “cosmetic, non-rejectable” tightened in year two; older labels may be inconsistent with the current standard | 2 | 2 | 4, Medium | Re-labeled a stratified sample of year-one images under the current guidance; agreement with original labels 81%, acceptable for cosmetic-tier only | 2, Low |
Subgroup-sliced evaluation requirement: Line 2 and the critical-defect class both require sliced reporting in the validation protocol, with a minimum recall of 98% on the critical class specifically, not just in aggregate.
Remediate versus scope-out: Line 2 was remediated (data collected) rather than scoped out, because Line 2 represents a third of production volume and excluding it from intended use would leave a material gap in inspection coverage. This is recorded in the model’s intended-use statement, IU-VISION-2026-04, as fully in scope following remediation.
Common inspection findings this risk assessment prevents
- A model is deployed with no documented assessment of whether its training data represents the population it inspects or predicts against.
- A known under-representation is discovered only after a production failure, rather than assessed and disposed of before build.
- Class imbalance is present but performance is reported only as aggregate accuracy, hiding a model that never catches the rare, high-consequence case.
- A subgroup was silently excluded from effective coverage with no documented scope statement, so users believe the model covers a population it does not.
- Bias findings exist on paper but no subgroup-sliced evaluation ever appears in the validation record, so the finding was never actually verified as resolved.
How to adapt this risk assessment
- Set your document number, owner, and the model or dataset it assesses in the header.
- Adjust the likelihood and severity scales in section 2 to match your organization’s existing risk-scoring convention if one exists, rather than running two incompatible scales in parallel.
- Add or remove bias categories in section 3 based on the model’s actual data sources; a text-classification model over free-text QMS narratives will weight label bias more heavily than a sensor-driven model will.
- Feed the subgroup list from section 5 directly into the model’s validation protocol so sliced evaluation is not left as an afterthought.
- Confirm every regulation in section 8 against the current published version before issue.