Managing the GxP Machine Learning Lifecycle: Drift, Retraining, and Continuous Monitoring

Validating a machine learning model and getting it into production is the easy part. The hard part starts the day after go-live, when the data the model sees in the real world begins to diverge from the data it was trained on, when a vendor pushes a new model version, and when someone asks the uncomfortable question: how do you know the model still works?

Traditional computerized system validation assumes deterministic software. Same input, same output, forever, until a documented change. Machine learning breaks that assumption in two ways. First, the model is a statistical artifact whose correctness is probabilistic, not absolute. Second, the world it operates in moves, so a model that was correct at qualification can quietly degrade without a single line of code changing. This article covers the lifecycle controls that close that gap: drift detection, the locked versus continuously learning decision, retraining under change control, and the continuous monitoring program that holds it all together.

This is the post-deployment companion to the build-and-qualify work covered in validating AI GxP systems and building AI tools for GxP. Read those first if you have not validated an ML system before.

Why ML needs a lifecycle program, not just a validation

The regulatory basis

No single regulation says “machine learning.” The obligations are inherited from existing GxP frameworks, and you have to map them onto the ML problem.

• 21 CFR Part 11 (1997) and EU Annex 11 (2011) apply to any computerized system that creates, modifies, or maintains GxP electronic records. An ML model that decides whether a batch passes visual inspection, flags a deviation, or classifies an adverse event is producing GxP records and falls squarely inside these. Annex 11 clause 1 requires risk management across the full lifecycle, clause 10 requires change control, and clause 11 requires periodic evaluation that the system remains in a validated state. That periodic evaluation clause is the legal hook for continuous monitoring of a model.
• GAMP 5 Second Edition (ISPE, 2022) is the framework used in practice. The Second Edition explicitly addresses AI and ML in a dedicated appendix (Appendix D11, Artificial Intelligence and Machine Learning) and frames the problem as a lifecycle with ongoing performance monitoring, not a one-time test campaign. It introduces the idea that systems with adaptive or statistical behavior need controls proportionate to that behavior.
• FDA guidance “Computer Software Assurance for Production and Quality Management System Software” shifts the emphasis from documentation volume to critical thinking and risk. The 2022 draft was titled ”…Quality System Software,” first finalized on 24 September 2025 as ”…Production and Quality System Software,” then superseded by the version issued 3 February 2026, which added “Management” to the title to align with the Quality Management System Regulation (21 CFR Part 820). For ML, CSA is useful because it pushes you to spend assurance effort where the model can actually hurt a patient or a product, and to treat ongoing monitoring as part of the assurance story rather than a separate activity. See computer software assurance FDA.
• FDA guidance “Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions” (final since December 2024, with the current version issued 18 August 2025) is written for medical devices, but it is the clearest regulatory thinking that exists on how to pre-authorize model changes. The Predetermined Change Control Plan (PCCP) concept, a pre-agreed envelope of allowed modifications plus the methods and acceptance criteria for making them, is directly transferable to GxP manufacturing and lab models even when no device submission is involved.
• ICH Q9(R1) Quality Risk Management (2023) governs how you assess and document the risk that drives all of the above.
• EU GMP Annex 22, “Artificial Intelligence” (draft, 2025) is the first GMP annex written specifically for AI in pharmaceutical manufacturing. The European Commission released the draft for public consultation on 7 July 2025; the consultation closed on 7 October 2025, and EMA convened a further multistakeholder workshop on 30 June and 1 July 2026 to work through generative-AI control options before finalizing the text, which is now expected no earlier than 2027. Verify its status and wording against the published source before you rely on it, because it is still a draft. As drafted, it takes a clear position that maps onto this article: static, deterministic AI/ML models can be used in critical GMP applications when they are qualified and kept under lifecycle control, while dynamic (continuously learning) models and generative AI are limited to non-critical uses with documented human oversight. That is the regulators putting the locked-by-default principle into a GMP text. A companion draft revision of Annex 11 (Computerised Systems) went out for consultation in the same period and reinforces data governance and periodic evaluation for systems that now include AI; treat both as direction-of-travel, not yet binding.
• FDA draft guidance “Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products” (draft, 7 January 2025) is the drug and biologic counterpart to the device PCCP guidance, and a better audience fit for most readers here. It introduces a risk-based credibility-assessment framework keyed to the model’s context of use, and it explicitly covers the manufacturing and postmarketing phases where a model runs continuously and can drift. Where the article below talks about proving a model stays credible over its operating life, this is the guidance that frames it for drugs and biologics rather than devices.

The risk rationale is simple. A degraded model does not crash. It keeps returning confident answers that are increasingly wrong. A false-negative on a defect classifier ships bad product. A drifted adverse-event triage model under-reports a safety signal. Because the failure mode is silent, the control has to be active and continuous. You cannot inspect your way to confidence after the fact.

What “lifecycle” means here

The ML lifecycle in a GxP context has five phases that repeat: data and problem framing, build and train, validate and qualify, deploy, and operate. The operate phase is where this article lives, and it contains four mandatory elements:

1. Continuous performance monitoring against pre-defined acceptance criteria.
2. Drift detection (data drift, concept drift, and prediction drift).
3. A retraining decision and execution process under change control.
4. Periodic review and re-validation triggers.

How this compares to a traditional CSV/CSA lifecycle

Anyone coming from traditional computerized system validation asks the same question first: is this just CSV with extra steps, or something genuinely different. The stage names look similar. Three of them behave differently once you look closely.

Rows one, four, and six are where ML genuinely departs from CSV. Everywhere else, treat it as CSV applied to a probabilistic subject, and you will not be far wrong.

Locked versus continuously learning models

This is the first and most consequential design decision, and inspectors increasingly ask about it directly.

The two model types

A locked model has fixed parameters after training. It produces the same output for the same input every time. You train it, freeze the weights, validate that frozen artifact, and deploy it. Any change to the weights is a new version that goes through change control. The overwhelming majority of GxP ML systems should be locked, and almost all defensible ones are.

A continuously learning model (also called adaptive or online learning) updates its parameters automatically as new data arrives in production. Its behavior at noon is not guaranteed to equal its behavior at 9am. This is genuinely difficult to validate because the validated state is a moving target, and most quality systems are not built to control software that changes itself.

The FDA’s own framing for AI-enabled devices uses “locked” versus “adaptive” algorithms, and the PCCP guidance, final since December 2024 and current in its 18 August 2025 version, exists precisely because adaptive algorithms cannot be handled by traditional one-time clearance. The same logic applies inside a GxP facility. The draft EU GMP Annex 22 lands in the same place from the manufacturing side: as written, it allows static, deterministic models in critical applications and pushes continuously learning models and generative AI out to non-critical uses with human oversight. When two regulators reach the same conclusion from opposite directions, the design default is settled. Lock the model unless you can write down a risk-justified reason not to.

How to decide

The decision is a risk-based one, documented in your validation plan and risk assessment.

Decision sequence:

1. Define the GxP impact of an erroneous output via your risk assessment. If a wrong output can affect product quality, patient safety, or data integrity, default to locked.
2. If you believe continuous learning is genuinely needed, ask whether scheduled retraining of a locked model on a defined cadence solves the same problem. Almost always, it does. Quarterly retrain-and-revalidate of a locked model gives you adaptivity without giving up control.
3. Only if scheduled retraining cannot keep up with the real drift rate, and the risk is acceptable, consider continuous learning, and only with a PCCP-style pre-authorized change envelope plus automated guardrails (champion or challenger comparison, automatic rollback, bounded performance gates).

Acceptance criteria for the decision

Good looks like: a documented, risk-justified statement of model type in the validation plan; for locked models, an explicit rule that any weight change is a versioned, change-controlled event; for continuous-learning models, a pre-approved change protocol, defined performance boundaries the model may never cross, and an automatic rollback to the last validated state when it does.

Common mistake

The most common finding pattern in this area is a model that is described in documentation as “locked” but whose vendor silently updates it behind a cloud API. You validated version 3.1. The vendor is now serving 3.4 and never told you. If your model runs as a SaaS or cloud inference endpoint, you have not actually locked anything unless your supplier agreement and your monitoring catch version changes. See cloud SaaS validation and software supplier assessment CSA.

Drift: the three kinds and how to detect each

Drift is the reason a locked, validated model degrades without any change to its code. You must monitor for it, and you must know which kind you are dealing with because the response differs.

Data drift (covariate shift)

The distribution of the input features changes, even though the relationship between inputs and the correct answer has not. Example: a vision model for particulate inspection was trained on vials filled on line A. Production shifts to line B with a slightly different lighting geometry. The pixel-intensity distribution moves. The model may still be right, but you cannot assume it without checking.

How to detect:

• Compare the live feature distribution against the training (reference) distribution on a rolling window.
• Common statistical tests: the Population Stability Index (PSI), Kolmogorov-Smirnov test for continuous features, and chi-square for categorical features. Kullback-Leibler divergence and Jensen-Shannon distance are also used.
• A widely used PSI rule of thumb: below 0.1 means no meaningful shift, 0.1 to 0.25 means moderate shift worth investigating, above 0.25 means significant shift requiring action. Set your own thresholds and justify them; do not treat the rule of thumb as a validated limit.

Concept drift

The relationship between inputs and the correct output changes. The inputs may look the same, but the right answer is now different. Example: a model that predicts which deviations are likely to become CAPAs was trained before a process change. After the change, the same deviation signature now means something different. This is the most dangerous kind because input monitoring alone will not catch it.

How to detect:

• You need ground truth, which usually arrives with a delay. Track the model’s accuracy, precision, recall, or whichever metric matters, against confirmed outcomes over time.
• Where labels are slow, use proxy signals: a rising rate of human overrides of the model’s recommendation, a rising rate of downstream rework, or a shift in the model’s own confidence distribution.

Prediction drift (label drift)

The distribution of the model’s outputs changes. If a defect classifier that historically flags 2% of units suddenly flags 8%, something moved, and it is either a real process problem (good, the model is working) or the model has drifted (bad). Prediction drift is cheap to monitor because it needs no ground truth, so it is often your earliest warning, but it is ambiguous and must be triaged.

A worked drift-monitoring table

A monitored visual-inspection classifier, reviewed monthly:

This single table tells the story: inputs are mostly stable, but recall has dropped below the validated floor and humans are overriding the model three times more than at qualification. This is a concept-drift signal that triggers the retraining decision process.

Setting a defensible drift threshold, not a rule of thumb

The PSI bands above, below 0.1 is fine, 0.1 to 0.25 is a warning, above 0.25 calls for action, are an industry rule of thumb, not a statistical result about your data. An inspector who asks why 0.25 and not 0.15 deserves a better answer than “that is what everyone uses.” A threshold derived from your own stable-period history holds up better, and getting there takes four steps.

1. Pick a stable reference period. Take a stretch of time after go-live that your monitoring and override data already show was well-behaved: no known process change, no flagged drift, ideally several months of data. This is your null case, the process was fine, and you want to know how much PSI moves around even when nothing is actually wrong.
2. Compute PSI between successive windows inside that stable period. Split the stable period into windows the size you intend to monitor at, weekly or monthly, and compute each window’s PSI against the training reference distribution, or against the prior window, matching however you plan to run the live comparison. The result is a distribution of PSI values that occurred with nothing wrong at all.
3. Set the warn and act thresholds from that null distribution, not from general practice. A common approach sets the warn threshold near the 90th to 95th percentile of the null PSI values and the act threshold higher, for example the 99th percentile, or the null mean plus three standard deviations, whichever fits your risk tolerance and sample size. A naturally noisy feature, some raw sensor readings behave this way, earns a wider band than a feature that barely moves under normal operation. That is the point: the threshold reflects the feature’s own behavior, not a number borrowed from a slide deck.
4. Document the derivation, not just the result. Record the stable period used, the number of windows, the resulting null distribution’s shape (a simple percentile table is enough), and the rule chosen to turn a percentile into a threshold. That derivation is what turns 0.18 from an arbitrary number into a defended one.

None of this retires the industry rule of thumb; it gives you something better for the feature whose natural variability does not match it. A model with ten monitored features rarely has one threshold that fits all ten, and treating 0.25 as a fixed rule for every feature is itself a way to miss real drift on a quiet feature while chasing false alarms on a noisy one.

Acceptance criteria

Good looks like: every model has a documented reference dataset, a defined set of drift metrics, numeric thresholds with a documented rationale, a monitoring cadence proportionate to risk, and defined actions for warn and act states. The thresholds are set before monitoring starts, not reverse-engineered to make the current data pass.

Common mistakes

• Monitoring only input drift and assuming the model is fine. Concept drift sails straight through input-only monitoring.
• No reference dataset retained, so there is nothing to compare against. The training data snapshot must be version-controlled and kept. See data lifecycle and metadata.
• Thresholds that were never justified, so when a regulator asks “why 0.25?”, there is no answer.

Continuous performance monitoring

Drift detection tells you the world has moved. Performance monitoring tells you whether that movement has actually broken the model. You need both.

What to monitor

Monitor at three levels:

1. Model quality. The task-appropriate metrics: accuracy, precision, recall, F1, area under the ROC curve, mean absolute error, calibration. Pick the metric that maps to the GxP harm. For a defect classifier the cost of a false negative (shipping a defect) usually dwarfs a false positive, so recall and the false-negative rate are the headline metrics, not raw accuracy.
2. Operational health. Latency, error rate, input data completeness, missing or out-of-range features, throughput. A model fed garbage produces garbage; input validation at inference time is part of monitoring.
3. Human-in-the-loop signals. Override rate, time-to-override, and the agreement rate between the model and the qualified human reviewer. Rising disagreement is one of the earliest and most honest signals you have.

How to build the monitoring program

1. Define the reference performance from the validation/qualification dataset. These become your acceptance floors.
2. Set the monitoring frequency by risk. A high-impact model used in batch disposition might be reviewed monthly with automated daily alerting; a low-impact triage assist might be quarterly.
3. Automate data collection. Log every prediction, the input features (or a hash plus reference), the model version, the timestamp, and where available the eventual ground truth. This log is a GxP record and must meet ALCOA+ principles. See ALCOA+ in detail and audit trail design and review.
4. Define alert thresholds (warn and act) and route them to a named owner.
5. Produce a periodic monitoring report that QA reviews and approves. This report is the documented evidence under Annex 11 clause 11 that the system remains in a validated state.

A sample monitoring report structure

• Header: model name, version, period covered, model owner, reviewer.
• Performance summary table: each metric, validated floor, current value, status.
• Drift summary: the drift table above.
• Events: any threshold breaches, alerts raised, investigations opened.
• Volume: number of predictions, number overridden, input data quality issues.
• Conclusion: model remains validated / model requires retraining / model suspended.
• Signatures: owner and QA, dated.

Acceptance criteria

Good looks like: a signed periodic monitoring report exists for every in-scope model, every reporting period, with no gaps; breaches are linked to deviations or change controls; and someone can pull the report in an inspection within minutes and show an unbroken chain of “the model still meets its validated performance.”

Common mistakes

• The model was validated and then nobody looked at it again. This is the single most common ML inspection gap. Validation evidence dated at go-live, nothing since.
• Monitoring exists in a data-science notebook that no one in quality has seen, with no signed report and no QA review. Monitoring that is not reviewed and approved does not count.
• Logging predictions without the model version, so when performance drops you cannot tell which version produced what. Always log the version.

Large language models and generative AI: what changes in the operate phase

Everything so far assumes a model that returns a class label, a score, or a number, the kind of model a defect classifier or a deviation-triage model represents. A large language model or another generative system produces open-ended text, and that changes what drift and monitoring mean once it is live. Qualifying LLMs and generative AI for GxP use covers how to validate a generative system before go-live, including the golden dataset, scoring method, and guardrail design. This section covers what is different once the system is running and needs the same continuous discipline the rest of this article describes.

Hallucination is not drift

A classifier that has drifted was probably fine yesterday and is degrading today because the world moved. A large language model can produce a fluent, wrong, entirely fabricated answer on day one, on a completely ordinary input, with no drift involved at all. Hallucination is a standing property of the technology, not a signal that something changed. A drift dashboard built around PSI and KS tests on structured features tells you nothing about whether today’s batch of generated summaries invented a lot number that does not exist. You need a second, parallel monitoring track built around content correctness rather than distribution shift: a defined grounding check, does every factual claim in the output trace to a passage in the retrieved source material or the input rather than being unsupported, a citation accuracy check where the tool cites specific documents, and a sampled human accuracy review scored against a rubric, the same way you would score a human-written summary. Track the rate of ungrounded claims over time the way you would track a defect rate; a rising rate is a real signal even though no input distribution moved at all.

Prompt and model version pinning

A locked classifier is defined by its frozen weights. A generative system is defined by a combination of the model version, the prompt or system instruction, the retrieval corpus if one is used, and the generation parameters, and any one of those four can change independently of the others. Vendor-hosted large language models are usually not versioned the way an on-premise model is: the same model name can point at different underlying weights over time unless you pin an exact, dated version string, and a growing number of vendors deprecate a specific pinned version on a fixed timeline whether or not you are ready. Log the exact version string the API actually returns with every call, not the name a developer typed into a configuration file, and treat a change to any of the four elements as a change requiring the same assessment discipline as a retrain. The Record: AI prompt, model version, and retrieval corpus change control gives the per-change record for exactly this problem, and the Log: SaaS vendor release notes review covers the parallel discipline for a vendor’s functional releases; neither substitutes for actively checking the served model version against your locked reference, because a silent model swap under an unchanged name produces no release note at all.

Retrieval freshness as its own drift dimension

A retrieval-augmented system is only as current as its retrieval corpus. If the corpus is a snapshot of controlled documents, SOPs, specifications, batch records, taken at a point in time, then the moment a source document changes and the corpus is not refreshed, the system starts answering from a stale source with complete confidence. This is a drift-like failure, but it will not show up in any input-feature PSI calculation, because the input question distribution has not moved at all; what moved is the ground truth the system is retrieving against. Monitor corpus currency directly: the age of the oldest and newest documents in the index relative to the source system of record, a defined maximum staleness before the corpus must refresh, and a spot check that a recently changed source document is actually retrievable and cited correctly within the refresh window.

Why classical drift statistics do not transfer cleanly

Population Stability Index and Kolmogorov-Smirnov compare two distributions of a defined numeric or categorical feature. Free text has no single defined feature to compare in that sense. Programs monitoring generative systems in production typically substitute a handful of proxy signals instead: embedding-space distance between a rolling sample of outputs and a reference sample of validated-period outputs, as a coarse semantic-drift proxy; the refusal, or “I cannot answer,” rate, where a rising rate can mean the input population is moving into territory the guardrails were not tuned for; output length and structure statistics, where a schema-constrained generative tool suddenly failing more schema checks is an early, cheap signal; and the human-edit distance between what the tool proposed and what the reviewer finally accepted, which behaves much like the override rate in the classifier world. None of these is as clean as PSI on a numeric feature, and that is a genuine, honest limitation of monitoring generative systems, not a gap in your program, as long as the proxies are named and someone is actually watching them.

Explainability and interpretability for high-impact models

A monitoring report can show a model still meets its acceptance criteria in aggregate while nobody in the room can say why it made a specific call. For a model whose output can affect patient safety, product disposition, or a reported safety signal, that gap is itself a finding waiting to happen: the FDA’s risk-based credibility framework for AI in drug and biologic regulatory contexts and GAMP 5’s treatment of AI and ML both point the same direction, the higher the consequence of a wrong output, the more the organization needs to understand not just that the model is usually right, but why it reached a given answer on a given input.

What interpretability buys you that accuracy alone does not

Two models can post identical aggregate accuracy on your validation set and behave completely differently underneath. One relies on the clinically meaningful features a subject-matter expert would expect. The other has found a spurious correlate in the training data, a batch number, a date field, an artifact of how the training set was assembled, that predicts the label without meaning anything. Only interpretability work surfaces the second case before it fails in production on data where the spurious correlate no longer holds.

Two families of method, described plainly

Model-intrinsic interpretability comes from choosing a model type whose internal logic a person can read directly, such as a decision tree of bounded depth, a scorecard-style linear model, or a rule list. You trade some predictive power for a model an SME can walk through line by line, and for many GxP use cases, especially where the input feature set is already well understood and modest in size, that trade is worth making deliberately rather than by default reaching for the most complex model available.

Post-hoc explanation methods instead treat a more complex model as fixed and explain individual predictions after the fact. Two widely used families are worth naming. SHAP, built on the game-theoretic idea of Shapley values, attributes each feature’s contribution to a specific prediction in a way that stays mathematically consistent across features and adds up to the model’s actual output for that case. LIME instead builds a small, simple, interpretable model that approximates the complex model’s behavior only in the local neighborhood of the one prediction being explained, trading global fidelity for a fast, case-specific answer. Both produce a ranked list of which inputs pushed a specific prediction up or down, which is exactly the artifact an investigator wants when asking why the model flagged, or failed to flag, one particular record.

Where to apply it, and where not to bother

Interpretability work costs effort, and applying it everywhere is not a good use of that effort. Reserve routine, per-decision explanation output for the risk tier where a wrong output has real consequence, tied to the same risk classification that drives the locked-versus-continuous decision described earlier and the linked risk assessment: AI/ML system. For a low-risk assistive tool, a documented understanding of the model’s general behavior at validation time, plus the human review step, is proportionate; generating a per-case explanation for every routine output spends assurance effort where the risk does not justify it, the same CSA thinking this article already applies to monitoring depth.

Common mistakes

• Choosing the most complex available model for a use case the simpler, inherently readable model would have handled just as well, and paying the interpretability cost for no accuracy gain.
• Treating a feature-importance chart generated once at validation as a permanent description of the model’s behavior, when what the model actually relies on can shift as the input population drifts, the same way its accuracy can.
• Producing an explanation output that nobody, including the reviewer relying on it, has been trained to read or challenge.
• Assuming interpretability output is itself evidence of correctness. A consistent, well-attributed explanation for a wrong answer is still a wrong answer; explainability output supplements the accuracy evidence, it does not replace it.

Retraining under change control

When monitoring says the model has degraded, or when scheduled retraining comes due, you retrain. In a GxP environment retraining is a change to a validated system, and it goes through change control for validated systems. The trick is doing this without it taking three months every time.

The regulatory basis

Annex 11 clause 10 requires change control for computerized systems. GAMP 5 Second Edition requires changes to be assessed for impact and validated proportionately. For models, retraining changes the validated artifact (the weights), so by definition it needs assessment, testing against acceptance criteria, and approval before the new version goes live. The FDA PCCP guidance, final since December 2024 and current in the version issued 18 August 2025, gives you the mechanism to pre-authorize a defined class of these changes so each one does not require a fresh full validation.

What triggers retraining

Document your triggers explicitly:

• Performance breach: a monitored metric falls below its validated floor.
• Drift breach: a drift metric crosses the act threshold.
• Scheduled cadence: a planned retrain (for example quarterly) regardless of measured drift, to absorb slow accumulating shift.
• Data or process change: a known upstream change (new instrument, new product, process change) that you predict will move the input distribution.
• Defect found in the model: a discovered failure mode.

The retraining decision path

Before you start retraining, route the trigger through one decision: does this change fall inside the pre-authorized envelope, or not. That single fork decides whether you run a fast pre-approved protocol or a full change control. The path is the same shape every time.

The retraining procedure, step by step

1. Open a change control referencing the trigger (the monitoring report, the deviation, or the schedule).
2. Assess impact and risk. What does the change affect, what could go wrong, what level of testing is proportionate.
3. Curate the retraining dataset. This is where most retraining goes wrong. The new training data must be representative, correctly labeled, free of leakage, and version-controlled. Document the dataset version, its provenance, and how labels were assigned and quality-checked. Garbage labels produce a worse model that passes superficial checks. See data criticality and data risk.
4. Retrain using the documented, version-controlled training pipeline. The pipeline itself should be qualified so the process is reproducible.
5. Validate the candidate (challenger) against acceptance criteria on a held-out test set that the model never saw in training. The new model must meet or exceed the validated performance floors, and must not regress on any safety-critical metric. Compare against the current production model (the champion).
6. Check for regressions and fairness/bias where relevant, and confirm the candidate has not improved overall accuracy while quietly getting worse on a critical subgroup or defect type.
7. QA review and approval of the validation evidence.
8. Controlled deployment with the new version number, updated configuration, and an updated entry in the system inventory. Retain the previous version so you can roll back.
9. Heightened monitoring for a defined post-deployment period to confirm the new version behaves in production as it did in test.
10. Close the change control with all evidence attached.

Champion or challenger and the held-out set

The champion is the model currently in production. The challenger is the candidate. You never promote a challenger on training performance; you promote it only after it beats or matches the champion on an independent held-out test set under your pre-defined acceptance criteria. Keeping the test set strictly separate from training data is the discipline that prevents you fooling yourself, and data leakage between the two is one of the classic ways an ML “improvement” turns out to be illusory.

Predetermined Change Control Plan

A PCCP, borrowing the FDA device concept set out in the December 2024 final guidance and carried into its current 18 August 2025 version, lets you pre-approve a bounded class of changes. It documents three things:

• The types of changes you are pre-authorizing (for example, retraining on new production data of the same type, with no change to inputs, architecture, or intended use).
• The methods (the protocol) you will follow each time: the data curation rules, the training pipeline, the test protocol.
• The acceptance criteria the new version must meet, and the impact assessment confirming the change stays inside the envelope.

With an approved PCCP, a routine quarterly retrain executes against the pre-approved protocol and acceptance criteria without re-litigating the whole validation each time. A change outside the envelope (new input feature, new model architecture, new intended use) falls out of the PCCP and gets full change control. This is how you get fast, controlled retraining instead of slow uncontrolled retraining.

A worked retraining acceptance table

Challenger v3.2 evaluated against champion v3.1, held-out test set, defect classifier:

Even though the challenger improved overall recall and cut false negatives in half, it regressed on the cosmetic-defect subgroup below the floor. This challenger is rejected and sent back. The held-out subgroup check caught a regression that aggregate accuracy hid. This is exactly the kind of finding the process is designed to surface.

Roles and responsibilities

Common mistakes and inspection findings

• Retraining done informally by data science with no change control, no impact assessment, and no QA approval. A “we just refreshed the model” answer in an inspection is a finding waiting to be written.
• No held-out test set, or the test set leaked into training, so the validation evidence is meaningless.
• Acceptance criteria set after seeing the results so that the new model passes. Criteria must be pre-defined and approved.
• Old versions deleted, so there is no rollback path and no ability to reproduce a historical decision.
• The same person trains and validates with no independence. Separation of duties matters here as much as anywhere in GxP.

Model decommissioning and retirement

Every model eventually stops running: the process it supported is retired, a better model replaces it, a vendor relationship ends, or a decision is made that its risk no longer justifies keeping it live. Retirement is the least controlled stage of the entire lifecycle in practice, because nothing is generating new alerts to force attention onto it. A model that quietly stops being called is not the same thing as a model that has been decommissioned. The endpoint can still exist, the training data can still be sitting in someone’s storage with no retention plan, and the AI system inventory can still show it as live months after anyone last used it.

What decommissioning actually requires

Retirement is a change, and it goes through the same change control discipline as a retrain. Beyond disabling the endpoint and removing access, three things are easy to get wrong:

• Record retention for the decisions the model already made. The GxP records this model produced while it was live, a batch disposition it fed, a deviation tier it suggested, do not stop needing to be reconstructible just because the model that produced them is gone. Retain the model version or versions that were live during the retention period, the training and reference datasets those versions relied on, and enough of the validation and monitoring history to let a future investigator understand how a specific historical output was reached. Match the retention period to the schedule for the GxP records the model influenced, not a shorter default IT-asset period; a model that fed batch disposition decisions inherits that record class’s retention period, which typically runs far longer than a routine software asset’s.
• A defined fallback for whatever the model was doing. If nothing replaces the model’s function, the process needs an explicit fallback, a manual step, an existing non-AI control, so retiring the model does not leave a silent gap in a control that was relied upon.
• Closure on the vendor side, for a vendor-hosted model. Confirm what happens to your data and any site-specific tuning artifacts at the vendor under your quality or technical agreement’s data-return and deletion terms, and get that confirmation in writing rather than assuming the ended relationship is enough on its own.

Acceptance criteria

Good looks like: the decommissioning decision and its rationale are recorded and approved; the model’s final version, training data, and monitoring history are archived per the applicable GxP records retention schedule, not a generic IT default; the live endpoint is confirmed disabled by a direct check, not assumed from a configuration change; any vendor-side data return or deletion is confirmed in writing; and the AI system inventory is updated to show the retired state with a closing date. The Checklist: AI system decommissioning walks through this in full, with a filled example.

Common mistake

A model retired informally, by simply no longer calling it, while its endpoint stays live, its access grants stay open, and the inventory still lists it as active. Months later an integration nobody remembered still calls it, or an audit finds a live credential to a system everyone thought was gone.

Putting it together: the operating model

A defensible ML operating model in GxP looks like this, end to end:

1. The model is locked by default and version-controlled. The reference (training) dataset is retained and versioned.
2. Every prediction is logged as a GxP record with version, timestamp, inputs, output, and eventual ground truth where available.
3. Automated drift and performance monitoring runs continuously and alerts a named owner on warn and act thresholds.
4. A signed periodic monitoring report is reviewed by QA every period, providing the Annex 11 clause 11 evidence that the system remains validated.
5. A PCCP pre-authorizes routine retraining; everything outside the envelope gets full change control.
6. Retraining runs champion or challenger against pre-defined acceptance criteria on an independent held-out set, with independent validation and QA approval, controlled deployment, heightened post-deploy monitoring, and a retained rollback version.
7. A periodic review (annually or risk-based) revisits the whole thing: are the thresholds still right, has the intended use changed, is the supplier still qualified.
8. Retirement is planned, not discovered: when the model’s working life ends, it goes through the same decommissioning discipline as any other GxP computerized system, with its records and training data retained on the schedule the GxP records it produced actually require.

The single sentence that ties it to the regulations: the validated state of an ML system is not a point in time, it is a controlled, monitored, and continuously re-evidenced condition, and your job is to prove it has never lapsed.

Interview-ready questions and how to answer them

“How is validating a machine learning system different from validating normal software?” Normal software is deterministic and its correctness is binary, so you test it once and it stays correct until a documented change. An ML model is statistical, so its correctness is probabilistic, and it can degrade in production without any code change because the input data or the underlying relationships shift. That is why ML needs continuous performance monitoring, drift detection, and a retraining process under change control, not just a one-time validation. The validated state has to be re-evidenced over time.

“What is the difference between a locked and a continuously learning model, and which would you use?” A locked model has frozen parameters and gives the same output for the same input until a change-controlled version update. A continuously learning model updates itself in production. I default to locked for anything with GxP impact because the validated state stays stable and reproducible, and I use scheduled retrain-and-revalidate of a locked model to get adaptivity under control. Continuous learning is hard to validate because the validated state moves, and I would only consider it with a pre-approved change envelope and automatic rollback guardrails, citing the FDA’s locked-versus-adaptive framing.

“Walk me through how you detect drift.” Three kinds. Data drift is a shift in the input distribution, detected by comparing live data to the reference dataset using tests like Population Stability Index, Kolmogorov-Smirnov, or chi-square. Concept drift is a change in the input-to-output relationship, detected through ground-truth accuracy over time and proxies like rising human-override rates. Prediction drift is a shift in the output distribution, cheap to monitor and often the earliest warning but ambiguous. I monitor all three because input-only monitoring misses concept drift, which is the most dangerous kind.

“A vendor hosts your model as a cloud API. What is your concern?” That the vendor silently updates the model and breaks my locked-and-validated assumption. I need the supplier agreement to require version-change notification, I need monitoring that detects behavior changes, and I need to log the served model version with every prediction so I know which version produced which record. Without that, I have not actually locked anything.

“How do you retrain a model in a GxP environment?” Under change control. Trigger from monitoring, a deviation, or a schedule. Open the change, assess impact, curate and version the training data, retrain on a qualified pipeline, validate the challenger against pre-defined acceptance criteria on an independent held-out set, compare to the production champion including safety-critical subgroups, get independent validation and QA approval, deploy with a new version while retaining the old for rollback, then monitor with heightened scrutiny. A PCCP lets routine retrains run against a pre-approved protocol so each one is not a full revalidation.

“What is a Predetermined Change Control Plan?” From the 2024 FDA AI device guidance, it pre-authorizes a bounded class of model changes by documenting the types of changes allowed, the methods used to make them, and the acceptance criteria they must meet. It lets you make controlled, pre-agreed changes like routine retraining without a full new validation each time, while anything outside the envelope still gets full change control.

“What is EU GMP Annex 22 and why does it matter here?” It is the first GMP annex written specifically for AI in pharmaceutical manufacturing, released as a draft for consultation on 7 July 2025 with the consultation closing on 7 October 2025, so it is direction-of-travel rather than binding yet. As drafted it allows static, deterministic models in critical GMP applications under qualification and lifecycle control, and restricts continuously learning models and generative AI to non-critical uses with documented human oversight. It matters because it is the manufacturing-side regulator reaching the same locked-by-default conclusion the FDA reached for devices, so it strengthens the case for a locked model with controlled, periodic retraining. I would check its final status before quoting it.

“How is monitoring a large language model different from monitoring a classifier?” A classifier’s failure mode is drift, a gradual mismatch between what it was trained on and what it now sees, and it is caught with distribution tests like PSI or Kolmogorov-Smirnov. A large language model can hallucinate a fluent, wrong answer on an ordinary input with no drift involved at all, so I run a second monitoring track built around content correctness: a grounding check that every factual claim traces to the source material, a citation accuracy check, and a sampled human accuracy review. I also treat the model version, the prompt, the retrieval corpus, and the generation parameters as four separate things that can each change independently, and I pin and log all four.

“How would you set a drift threshold you could defend to an inspector, rather than just quoting the usual PSI bands?” I would take a stable period after go-live where monitoring already shows nothing was wrong, compute PSI between successive windows across that period, and use the resulting distribution as my null case, how much PSI moves around when nothing is actually happening. I would set the warn threshold near its 90th to 95th percentile and the act threshold higher, maybe the 99th percentile, and document the stable period, the windows, and the rule used. That gives every feature its own defensible threshold instead of a single borrowed number applied everywhere.

“When would you want an explainable model instead of the most accurate one?” When the consequence of a wrong output is high enough that someone needs to see why the model reached a specific answer, not just that it is usually right in aggregate. For a use case with a modest, well-understood feature set, I would rather use an inherently readable model, a shallow decision tree or a scorecard, and give up a little accuracy, than reach for a complex model and bolt on a post-hoc explanation method like SHAP or LIME afterward. For a low-risk assistive tool, documented validation-time understanding plus the human review step is proportionate, and I would not spend explainability effort where the risk does not call for it.

“A model is being retired. What are you making sure happens?” That retirement runs through change control like any other change, not as an informal switch-off. I confirm the final model version, the training data, and the monitoring history are archived against the retention period the GxP records this model produced actually require, not a shorter IT-asset default. I confirm the live endpoint is actually disabled, not assumed disabled. If it is a vendor-hosted model, I get written confirmation of data return or deletion. And I update the AI system inventory so it stops looking live the day it actually stops being live.

“What evidence would you show an inspector that the model is still valid?” A continuous, gap-free series of signed periodic monitoring reports showing performance against the validated floors, the drift monitoring data, any breaches linked to deviations or change controls, the change-control records for every retrain with their acceptance evidence, and the retained version history. That chain demonstrates the Annex 11 clause 11 requirement that the system remains in a validated state.

“What is the most common failure you would expect to find?” Validation evidence dated at go-live and nothing since. The model was validated once and then nobody monitored it, so there is no evidence it still works, and quite possibly it has drifted. Closely behind that: informal retraining by data science with no change control.

Practical tips

• Set drift and performance thresholds before you start monitoring, and write down the rationale. Reverse-engineered thresholds are an instant credibility loss in an inspection.
• Keep the reference dataset versioned and immutable. If you cannot reproduce what the model was trained and validated against, you cannot defend anything downstream.
• Log the model version with every single prediction. It is the cheapest control you can add and the one whose absence hurts most.
• Treat the human-override rate as a leading indicator. It moves before your ground-truth metrics catch up and it is honest.
• Qualify the training pipeline, not just the model. Reproducible retraining depends on a controlled pipeline.
• Keep independence between whoever trains and whoever validates the challenger.
• For cloud-hosted models, get version-change notification into the supplier agreement and assume nothing.
• Do not over-engineer a low-risk assistive model. CSA thinking applies: spend the assurance effort where a wrong output can actually harm product or patient.
• For a generative system, derive your drift proxies (grounding rate, refusal rate, human-edit distance) before go-live, the same way you derive PSI thresholds for a classifier. Discovering you have nothing to watch is worse after launch than before it.
• Do not let a model’s endpoint outlive its last real decision. Put a decommissioning date on the calendar the day a replacement goes live, not the day someone notices the old one is still running.

Related reading

• Validating AI GxP systems
• Building AI tools for GxP
• Qualifying LLMs and generative AI for GxP use
• AI model credibility and trust
• Computer software assurance FDA
• GAMP 5 CSV framework
• Change control for validated systems
• Cloud SaaS validation
• Software supplier assessment CSA
• Quality risk management
• Data criticality and data risk
• 21 CFR Part 11 and EU Annex 11
• Statistics in quality: Cpk and control charts