Statistics in QA: Sampling, Process Capability, Control Charts, and OOT

Quality work runs on inference. You almost never measure every unit, every batch, every data point, you measure a sample and decide something about the whole. That decision is statistical, with or without the math written out. The practitioner who treats sampling as “grab a few and look” is making the same statistical claim as the one who cites an acceptance number from a published plan, just without knowing the producer’s and consumer’s risk they accepted.

This article walks through the four tools the work demands: how many units to inspect (attribute sampling), whether the process can hold the spec (capability indices), whether the process is behaving consistently over time (control charts), and how to catch a result that is drifting before it goes out of specification (out-of-trend detection). They connect: the same statistics that justify a sampling plan during release support the ongoing process monitoring that the FDA calls Stage 3, continued process verification. The same logic applies to a drug substance batch, a finished device lot, a packaging operation, or an environmental monitoring program. The modality changes; the statistics do not.

Why statistics is a regulatory expectation, not an optional add-on

The 2011 FDA process validation guidance, Process Validation: General Principles and Practices, reframed validation as a lifecycle: Stage 1 process design, Stage 2 process qualification, Stage 3 continued process verification. It leans on statistical thinking throughout. It states plainly that a manufacturer should understand sources of variation, and that “the number of samples should be adequate to provide sufficient statistical confidence of quality both within a batch and between batches.” That is a statistical standard of evidence, not a count you pick by feel.

ICH Q10, Pharmaceutical Quality System, expects ongoing monitoring of process and product performance. ICH Q9(R1), Quality Risk Management, expects risk decisions to be informed by data. Beyond those, ISO 9001:2015 (clause 9.1.1) calls for evaluating quality management system performance using suitable methods, including statistical techniques where appropriate, and ISO 13485:2016 (clause 8.4, Analysis of data) requires documented procedures that include appropriate statistical techniques. FDA 21 CFR 211.110 ties in directly for drug products: it requires control procedures to monitor output and validate performance of the processes that may cause variability in in-process material and the drug product. As a supporting note for combination products and software in a medical context, 21 CFR 820.250 (Statistical techniques), now folded into the Quality Management System Regulation through harmonization with ISO 13485, requires sampling plans to rest on a valid statistical rationale and to be written down. None of these prescribe a specific formula. They require that you have a defensible method and apply it.

So the inspector’s question is rarely “what is your Cpk?” It is “how did you decide that number was acceptable, and what do you do when it moves?” Knowing the tools is the easy part. Knowing the assumptions behind them, and where they break, is the actual skill. The rest of this article is organized around that distinction: each tool, what it tells you, what it assumes, and what an auditor checks.

A short orientation before the detail. Two ideas thread through everything:

• Common cause versus special cause. Common-cause (random, inherent) variation is the process humming along. Special-cause (assignable) variation is an external disturbance: a wrong raw material lot, a worn tool, an operator change, an instrument out of calibration. Almost every tool here exists to tell these two apart, because the correct response is opposite. You leave common cause alone and reduce it by redesign; you hunt down and eliminate special cause.
• Voice of the process versus voice of the customer. The process has a natural spread (its voice). The customer or registration sets the specification (their voice). Capability is the comparison of the two. Control is whether the process voice is stable over time. They are independent: a process can be stable and incapable, or capable and unstable.

Attribute sampling: ANSI/ASQ Z1.4 and the risks you are buying

Attribute sampling is for pass/fail characteristics, a unit conforms or it does not (correct label, intact seal, present component, legible expiry, no visible particulate). The dominant scheme is ANSI/ASQ Z1.4 (historically MIL-STD-105E, and equivalent to ISO 2859-1). It is a system of sampling plans indexed by lot size and an Acceptable Quality Limit (AQL).

The mechanics, stripped down

1. Pick an inspection level. General levels I, II, III trade sample size against discrimination; Level II is the default. Special levels (S-1 through S-4) give very small samples for costly or destructive tests.
2. Find the sample-size code letter from the lot size and inspection level in the tables.
3. Pick the AQL. This is the quality level the plan is designed to accept most of the time, not a target defect rate and not a guarantee.
4. Read the accept (Ac) and reject (Re) numbers for that code letter and AQL from the single, double, or multiple sampling table.

A worked example: a lot of 5,000 units, General Inspection Level II, gives code letter L, which is a sample size of 200. At an AQL of 0.65%, the single sampling plan accepts on 3 or fewer defects (Ac=3) and rejects on 4 or more (Re=4). If you found 3 defective units in your 200, the lot is accepted; the 4th defective rejects it.

The risks you are actually buying

The trap is reading AQL as “we accept 0.65% defective.” What the plan actually fixes is two risks:

The full behavior lives in the Operating Characteristic (OC) curve, the probability of acceptance plotted as a function of the lot’s true defect rate. Two plans with the same AQL but different sample sizes have different OC curves and very different consumer protection. A small sample at a loose AQL can accept genuinely bad lots a meaningful fraction of the time.

Here is the point that wins the interview. The single number that protects the patient is not the AQL, it is the Rejectable Quality Level (RQL), also called the Lot Tolerance Percent Defective (LTPD): the defect rate the plan is designed to reject most of the time. When the characteristic is patient-critical, AQL-based plans are usually the wrong tool. You want a zero-acceptance (c=0) plan, or attribute plans built to a specified LTPD, so you are designing around consumer’s risk rather than producer convenience. Squeglia’s c=0 plans are the common reference for this. A c=0 plan rejects on the first defect found, which is the only logically defensible stance for a critical defect where any nonconformance is a safety issue.

Switching rules are part of the standard

Z1.4 also defines switching rules, normal, tightened, reduced inspection, that are part of the standard, not optional flavor. The standard’s own thresholds for moving between states are spelled out in the document itself; the logic, stripped down, is a feedback loop:

• Start at normal inspection.
• A run of rejections in recent lots moves you to tightened inspection.
• A sustained run of accepted lots under tightened returns you to normal.
• Reduced inspection (smaller samples) becomes available only once a longer history of accepted lots and steady, qualifying production conditions are met, with approval by the responsible authority.
• A rejection while under tightened inspection stops sampling under the plan until the process is fixed.

Consult ANSI/ASQ Z1.4 for the exact numeric switching criteria. Inspectors notice when a site uses the AQL tables but ignores the switching rules; it signals the plan was selected for a number, not operated as a system. A documented finding pattern is a site sitting on tightened-triggering history while still sampling at normal level.

Where attribute sampling breaks

Z1.4 assumes attributes (count of defectives), random sampling, and lots of comparable history. It says nothing about which units you pull. A plan executed by sampling the easiest-to-reach units (the top tray, the front of the tote) is invalid regardless of the Ac/Re numbers; representativeness is an assumption you have to engineer, often with a randomization scheme or a stratified pull across the lot. For variables data, a measured dimension or an assay value, attribute sampling throws away information. There you move to capability indices and, for sampling by measurement, ANSI/ASQ Z1.9 (ISO 3951), which achieves the same protection with smaller samples by using the measured value and an assumed normal distribution.

One more practical decision: single versus double versus multiple sampling. Double and multiple plans can reduce average sample size when lots are clearly good or clearly bad, at the cost of a more complex procedure on the floor. For a hard-to-execute destructive test, fewer average samples is real money; for a fast visual check, the simplicity of a single plan usually wins.

Roles and responsibilities for sampling

Process capability: Cp, Cpk, Pp, Ppk

Capability indices answer a single question: can this process reliably produce output inside the specification limits? They compare the voice of the process (its natural spread) to the voice of the customer (the spec window).

Define the lower and upper specification limits LSL and USL. The four standard indices:

Cp, potential capability, ignores centering:

Cp = (USL - LSL) / (6 sigma)

Cpk, actual capability, accounts for how off-center the mean is:

Cpk = min[ (USL - mu) / (3 sigma), (mu - LSL) / (3 sigma) ]

Pp and Ppk, the same two formulas, but using overall standard deviation rather than the within-subgroup estimate.

Worked capability example

A fill operation has spec limits LSL = 9.80 mL and USL = 10.20 mL. Across 25 subgroups of 5 vials each, you estimate the mean mu = 10.04 mL, within-subgroup sigma = 0.040 mL, and overall sigma = 0.060 mL.

• Cp = (10.20 - 9.80) / (6 x 0.040) = 0.40 / 0.24 = 1.67. The spread could fit the window comfortably.
• Cpk = min[ (10.20 - 10.04)/(3 x 0.040), (10.04 - 9.80)/(3 x 0.040) ] = min[ 1.33, 2.00 ] = 1.33. The process is off-center toward the upper limit, so actual capability is worse than potential.
• Ppk uses overall sigma: min[ (10.20 - 10.04)/(3 x 0.060), (10.04 - 9.80)/(3 x 0.060) ] = min[ 0.89, 1.33 ] = 0.89.

Read the story those three numbers tell: Cp says the inherent spread is fine (1.67), Cpk says centering costs you (1.33), and Ppk well below Cpk (0.89) says the process is wandering between subgroups. Instant-to-instant the process is capable; over time it is not delivering it, and the recentering plus a hunt for the between-subgroup source (shift, raw material lot, setpoint creep) is the action.

The Cp/Pp distinction, said precisely

This is the part people get wrong, so be exact about it:

• Cp and Cpk use the within-subgroup standard deviation, typically estimated from the average range (R-bar/d2) or pooled subgroup variation. This is “short-term” capability, what the process could do if only common-cause variation were present. It implicitly assumes the process is in statistical control.
• Pp and Ppk use the overall standard deviation computed from all individual values. This is “long-term” performance and captures shift and drift between subgroups.

A large gap between Cpk and Ppk is a signal: the within-subgroup spread is tight but the process is wandering between subgroups (tool wear, shift changes, lot-to-lot raw material variation). Cpk says “capable instant to instant,” Ppk says “not actually delivering that over time.” Reporting Cpk alone on a drifting process flatters the result. A convention worth knowing: many statisticians reserve Cp/Cpk for the in-control short-term picture and report Pp/Ppk as the honest long-term performance, then quote the ratio as a stability indicator.

There is also the one-sided case. A microbial count, an impurity, or an endotoxin level has only an upper limit. Then only the relevant half of the Cpk formula applies (often written Cpu), and Cp is undefined because there is no two-sided window.

Cp, Cpk, Pp, and Ppk at a glance

A practical reading order for an interview or an inspection: start with Ppk, because it answers what the process actually delivered. Bring in Cpk to diagnose whether a shortfall is a centering problem, fixable by adjusting the setpoint, or a real spread problem, fixable only by reducing variation. Treat the Cp-to-Cpk gap and the Cpk-to-Ppk gap as two separate diagnostic signals rather than folding them into one score.

Decision tree: which capability index applies

What the numbers mean

Assuming a normal, in-control process:

A note on the famous “1.5 sigma shift.” The Six Sigma program assumes a long-term process can drift up to 1.5 sigma from center, which is why a “6 sigma” process is quoted at 3.4 defects per million rather than the 0.002 a perfectly centered 6 sigma process implies. That is a long-term-versus-short-term accounting convention, not a law of nature; know it exists so you can answer the question, but do not treat 3.4 DPMO as a regulatory target.

The assumptions that invalidate the index

1. Normality. Cpk’s defect estimates assume a normal distribution. Many real quality characteristics are not normal: particle counts, impurities near zero, dissolution bounded at a ceiling, anything with a hard physical floor. Compute Cpk on skewed data and the implied PPM is fiction. Either transform the data (for example, Box-Cox), use a non-normal capability method that fits the actual distribution (a fitted Weibull or lognormal), or use percentile-based indices that read off the 0.135 and 99.865 percentiles of the fitted curve. Run a normality test (Anderson-Darling, Shapiro-Wilk) and, more usefully, look at the histogram and probability plot before you trust the number.
2. Statistical control. Capability is only meaningful for a stable process. If the process is out of control, sigma is not estimating a stable spread, it is a mix of common and special causes, and the index has no predictive value. Always confirm control with a chart before computing capability. This is the single most common analytical error in capability studies.
3. Adequate, representative data. A handful of consecutive units underestimates true variation. Capability studies generally want enough subgroups to see between-subgroup variation, commonly 25 or more subgroups, collected across the real sources of variation (shifts, operators, lots, seasons).
4. Measurement system adequacy. If your gauge variation is a large fraction of the total, your sigma is partly the measurement system, not the process. A measurement systems analysis (Gauge R&R) should precede a serious capability claim, and the instruments behind it need their own qualification and calibration program (see calibration and metrology program); a rule of thumb flags a measurement system consuming more than 10 percent of tolerance as marginal.

Inspectors and auditors look for the linkage: a capability claim should trace to a stability assessment, a normality check, a measurement-system check, and a defined sample plan. A lone Cpk on a slide with no control chart and no normality statement is a finding waiting to happen.

Handling non-normal data

Many quality characteristics have a hard physical boundary, an impurity level near zero, a particle count, a leak rate, and pile up against that boundary instead of spreading out symmetrically. Running the standard Cpk formula, which assumes a normal bell curve, on that kind of data produces a defect-rate estimate that does not match reality, usually understating the real tail risk on the side pressed against the boundary.

Three responses, roughly in the order labs try them:

1. Test first, then decide. Run a normality test (Anderson-Darling is more sensitive to tail behavior than Shapiro-Wilk, and both beat eyeballing a histogram) and look at a probability plot. A statistically significant departure from normality on a small sample is not automatically fatal to the analysis; look at the shape, not only the p-value.
2. Transform the data, then apply the normal formula to the transformed values. A Box-Cox transformation (Box and Cox, 1964) finds a power, conventionally called lambda, that makes the transformed data closer to normal: lambda = 0 is the natural log, lambda = 0.5 is a square root, lambda = 1 leaves the data unchanged. Fit lambda from the data, transform the raw values and the specification limits together, compute Cpk on the transformed scale, and report the transformation used alongside the result. A transformation that only marginally improves normality, or that needs re-fitting every batch, is a sign the distribution needs the next option instead.
3. Fit a non-normal distribution and read off percentile-based indices. Where the data follows a known shape (Weibull for time-to-failure and particle-size data, lognormal for concentrations and potency ratios), fit that distribution and read the 0.135 and 99.865 percentiles directly from the fitted curve instead of assuming a normal distribution. A percentile-based capability index substitutes those fitted percentiles for the plus-or-minus-3-sigma points in the ordinary Cpk formula. ISO 22514-2 (which replaced ISO 21747, the earlier standard some older references and software still cite by that number) sets out this approach across several distribution families. It is the most defensible option when the distribution shape is well understood and stable, and the least defensible when the fit itself is shaky, since a poorly fit distribution just relocates the same error into a different formula.

Whichever route is used, say so. “Cpk = 1.41” with no mention of a normality check, a transformation, or a distribution fit is not wrong on its face, but it invites the one question that exposes an unsupported number: what did the data actually look like, and did anyone check?

Short-run and low-volume processes

A cell therapy product manufactured a handful of lots a year, a clinical-supply batch, or a low-volume specialty product rarely accumulates the 25-plus subgroups a standard capability study wants. Three adaptations are common, each with a real tradeoff:

• Label limits provisional and say so. A Cpk computed on 6 to 10 batches is not wrong to report, but it is unstable; the range of plausible values behind a small-sample Cpk is wide even when the point estimate looks good. State the batch count next to the index, and treat a provisional Cpk as a screening number, not a release criterion, until more data accumulates.
• Pool across similar characteristics with a target-based transform. Where several related characteristics, several fill volumes across product strengths, several dimensions on a family of similar parts, each have too few points on their own, coding each observation as the result minus the target, divided by half the tolerance, puts them on a common scale and lets one pooled capability estimate borrow strength across the family. This only holds up where there is a real reason to believe the underlying variation mechanism is shared across the pooled characteristics; pooling a stable step with an unstable one just hides the unstable one inside the average.
• Use the Taguchi capability index, Cpm, when hitting the target matters as much as staying inside the limits. Cpm = (USL - LSL) / (6 x square root of [sigma squared + (mu - target) squared]). Unlike Cpk, Cpm penalizes a process that sits inside spec but off the nominal target, which matters for a characteristic like a potency assay, where drifting toward either edge, even while still passing, erodes the safety margin.

None of these substitute for data. They are ways to be honest about what a small data set can and cannot support, and an inspector who sees a provisional label with a stated rationale reads that very differently from a Cpk of 1.90 quoted from four lots with no caveat at all.

Shewhart control charts: separating signal from noise

A control chart is the tool that decides whether variation is common cause (inherent, random, the process humming along) or special cause (an assignable, external disturbance). Walter Shewhart’s insight, from the 1920s, still anchors the discipline: react to every wiggle and you “tamper,” adding variation; ignore real signals and you ship drift. Control limits are the decision boundary.

Control limits are not specification limits. Specs come from the customer or registration; control limits come from the process data itself, conventionally the mean plus or minus 3 sigma of the plotted statistic. A process can be in perfect statistical control and still fail spec (capable of nothing useful), or be capable but out of control (capable on average, unpredictable). You need both views, and you never draw spec limits on a control chart, because doing so invites operators to chase the chart toward spec, which is the tampering Shewhart warned against.

Choosing the chart

The chart depends on the data type:

In batch pharma and biologics manufacturing the I-MR chart dominates, because you usually get one result per batch, not subgroups. That choice has consequences: I-MR is slower to detect small sustained shifts than subgrouped charts and is sensitive to non-normality. For detecting small, persistent drifts, EWMA (exponentially weighted moving average) or CUSUM (cumulative sum) charts are markedly more powerful and are the right escalation when a half-sigma drift matters, for example a slowly degrading chromatography column or a creeping fill setpoint. EWMA and CUSUM trade the fast detection of a large shift for far better detection of a small one; pick based on which failure mode hurts.

Decision tree: which chart applies

How control limits are actually set

The procedure is specific and inspectors ask about it:

1. Collect a baseline of data from a period believed to be in control, commonly 20 to 30 subgroups (or individual points for I-MR).
2. Compute the centerline (the mean) and the process sigma from within-subgroup variation. For I-MR, sigma is estimated from the average moving range divided by 1.128, not from the standard deviation of all points, because the moving-range estimate is insensitive to the very drift you are trying to detect.
3. Set limits at centerline plus and minus 3 sigma. Apply the chart constants (A2, D3, D4 for X-bar/R; the moving-range factors for I-MR) rather than computing 3 sigma by hand.
4. Remove identified special-cause points with a documented assignable cause, then recompute, so the baseline limits reflect a stable process.
5. Lock the limits and define a recalculation trigger: limits are recalculated only on a deliberate, justified basis (a validated process change, a defined data interval), never quietly each time new data arrives. Auto-recalculating limits every point is a classic way to make a drifting process look stable.

Worked example: a p-chart across lots of varying size

Attribute data rarely arrives in equal-sized lots, and that changes the arithmetic. A visual-inspection line records rejects across eight lots in a week, sample sizes varying with the lots actually run:

Center line, p-bar = total rejects / total inspected = 55 / 1,560 = 0.0353 (3.53 percent).

Because a p-chart’s control limits depend on each lot’s own sample size, the limit is not one flat line; it is recalculated per lot: UCL for lot i = p-bar + 3 x square root of [p-bar x (1 - p-bar) / n(i)]. For lot L6 (n = 160): UCL = 0.0353 + 3 x square root of (0.0353 x 0.9647 / 160) = 0.0353 + 0.0438 = 0.0791. Lot L6’s rate of 0.0875 sits above its own 0.0791 limit; every other lot’s rate sits below its own limit (the smallest lot, L2, has the widest limit at roughly 0.0805, and the largest, L5, has the tightest at roughly 0.0703). The lower control limit works out at or near zero for every lot here, because the reject rate is low relative to the sample sizes; a p-chart’s lower limit only becomes informative at a low defect rate with a large sample.

The point worth taking from this arithmetic: if the eight lots had been judged against one limit calculated at the average sample size, roughly 195 units, that shared limit would land close to 0.0753, and L6 at 0.0875 would still flag, but a smaller excursion on a smaller lot might not have, simply because it borrowed a limit too generous for its actual sample size. Where sample sizes vary substantially, either compute the limit per lot as shown, or use a standardized (z-score) p-chart, where each point is plotted as (p(i) - p-bar) divided by the same square-root term, so every lot is judged on one common scale regardless of its size. The same size-dependent-limit logic carries from a c-chart to a u-chart, and from an np-chart to a p-chart, whenever the count of inspection opportunities is not constant across periods; environmental monitoring programs run into this constantly when the number of plates or the sample volume varies by location. See environmental monitoring program design.

Western Electric / Nelson rules

The control limit alone (a point beyond 3 sigma) is only one of several signals. The Western Electric rules (from the 1956 WECO Statistical Quality Control Handbook), refined by Nelson, define patterns that indicate a process is no longer behaving randomly even when no single point has breached 3 sigma. The chart is divided into zones measured in sigma distance from the centerline (the outer band near the limits, a middle band, and an inner band) on each side. The patterns these rule sets recognize, described in general terms rather than reproduced verbatim, fall into a few families:

1. A single point beyond the 3 sigma limit: a gross shift or measurement error.
2. A short cluster of points crowding the outer band on one side: an emerging shift.
3. A run of points sitting beyond the inner band on one side: a sustained smaller shift.
4. A long run of consecutive points all on one side of the centerline: a mean shift the limits have not caught yet.

The fuller rule sets also recognize a steady trend (a run of points consistently rising or falling), stratification (a long run hugging the centerline, often a sign of incorrectly computed limits or mixed data streams), and oscillation (a long alternating up-down pattern, signalling over-adjustment or two interleaved streams). The precise point counts and sigma multipliers for each pattern are set out in the Western Electric handbook and in Nelson’s published rules; apply the specific values from those sources.

A practical caution: every rule you add raises the false-alarm rate. A single point on a stable process trips the 3 sigma rule about 0.27 percent of the time (roughly 1 in 370 points). With all eight Nelson rules running, the chance of a false signal on a stable process climbs well above that. Apply rules deliberately. For routine release monitoring, the 3 sigma rule plus a long-run-on-one-side rule and a trend rule covers most real signals without drowning the team in false positives. Reserve the full battery for investigations where sensitivity outweighs false-alarm cost. State which rules you apply in the SOP, so the response is consistent and the rule set is auditable.

Worked example: a chart catching drift before it breaches the limit

A purification step’s yield is tracked one result per batch, I-MR chart. The first ten batches establish the baseline:

Center line (mean of the 10 baseline results) = 91.90. Average moving range = (1.1+1.3+1.2+1.1+1.3+1.5+1.6+1.2+0.8) / 9 = 11.1 / 9 = 1.233. UCL = 91.90 + (2.66 x 1.233) = 91.90 + 3.28 = 95.18. LCL = 91.90 - 3.28 = 88.62. No Nelson violations in the baseline; limits are locked.

The next ten batches:

Batch 19 is the ninth consecutive point above the center line. That trips the Western Electric long-run rule at batch 19, well inside the control limits, and it should have opened an assessment right there: the process is drifting upward, one Nelson rule at a time, long before any single point looks alarming. Instead, in this scenario, nothing was done, and batch 20 breaches the UCL outright, a Rule 1 violation that confirms what the run rule already flagged nine batches earlier. The lesson is not that Rule 1 failed; it worked exactly as designed. The lesson is that a chart running on Rule 1 alone gave up nine batches of lead time the run rule was already offering for free. An investigation opened at batch 19 has a much better chance of finding a fixable cause, a reagent lot change, a resin nearing its qualified cycle count, a recalibrated in-line sensor, before a batch actually clears the limit and forces a disposition decision on top of the investigation.

Rational subgrouping

The most under-appreciated control-chart concept is rational subgrouping, organizing data so that within-subgroup variation captures only common cause, and between-subgroup variation exposes the special causes you care about. If you build a subgroup from units spanning a shift change, you bury the shift effect inside the within-subgroup noise and the chart goes blind to it. Conversely, sampling five consecutive units off one head of a multi-head filler tells you about that head’s repeatability but hides head-to-head differences. Subgroup composition is a design decision, and getting it wrong makes a technically correct chart useless. The question to ask: “what source of variation do I most want this chart to catch, and have I arranged the subgroups so it shows up between subgroups rather than hiding within them?”

What “out of control” obligates you to do

A control chart is only worth keeping if a signal triggers action. A defensible SOP states: on a confirmed special-cause signal, the process owner initiates an assessment, the event is logged (often as a deviation if product is affected), root cause is pursued, and the corrective action is verified by the chart returning to control. A chart that has been red for six months with no investigation is worse than no chart, because it documents that you saw the signal and did nothing.

Out-of-trend (OOT): catching drift before it becomes OOS

An out-of-specification (OOS) result fails a registered acceptance criterion. It is a discrete event with a defined investigative path; the FDA guidance Investigating Out-of-Specification (OOS) Test Results for Pharmaceutical Production (originally 2006, Revision 1 May 2022) is the reference, and the laboratory side is covered in detail in our OOS investigation process article. An out-of-trend (OOT) result is different: it is within specification but inconsistent with the expected pattern from historical data. OOT is the early-warning layer. By the time a result is OOS, you have a confirmed problem; OOT is the chance to act while everything still passes.

OOT matters most in stability programs, where you track a product across its shelf life and a single timepoint can look fine against spec while the trend line is clearly walking toward the limit before expiry. It also applies to release results trending across batches and to environmental monitoring counts creeping upward. See stability programs (ICH Q1A-Q1E) for the program context and out-of-trend investigations for the investigation mechanics.

The three statistical approaches

There is no single regulatory formula for OOT, but the industry converged on three complementary statistical approaches, most influentially described in a 2003 PhRMA stability working-group paper in Pharmaceutical Technology:

1. By-time-point (control-chart-style): compare a new result at a given timepoint against the distribution of historical results at that same timepoint. A result beyond a control limit (mean plus or minus 3 sigma of comparable historical data) is OOT for that point.
2. By-time-point regression / slope: fit the historical stability trend and flag a result that deviates from the predicted value by more than a defined tolerance (for example, a prediction interval). This catches a single aberrant point against an otherwise good trend.
3. Slope control / regression-control: evaluate whether the slope of an individual batch’s degradation differs from the historical slope distribution, catching a batch degrading faster than the population even if no single point is alarming yet.

Worked OOT example

A small-molecule assay has a release range of 95.0 to 105.0 percent and a stability trend that declines slowly. From 12 historical batches the 18-month assay values average 98.5 percent with a standard deviation of 0.5 percent. A new batch reads 96.8 percent at 18 months.

• Against spec: 96.8 is well inside 95.0 to 105.0, so it is not OOS.
• By-time-point limit: mean minus 3 sigma = 98.5 - 1.5 = 97.0 percent. The result of 96.8 is below 97.0, so it is OOT for that timepoint.
• Action: 96.8 at 18 months, declining, projects below 95.0 before the 24-month expiry. This is exactly the case OOT exists to catch: passing today, failing before end of shelf life. It triggers an assessment of the trend and the affected batch, not an automatic OOS lab investigation, though it can escalate to one.

What an OOT program must define

A workable OOT program defines, in an SOP:

• The limit-setting basis: how many historical batches or lots establish the baseline, and the minimum data before limits are considered valid. A common rule is a minimum of around 6 to 10 batches before by-time-point limits mean anything; fewer, and the limits are noise.
• The statistical method(s) applied and the alert/action thresholds.
• The distinction from OOS: OOT triggers an assessment, not automatically a full OOS lab investigation, though it can escalate.
• The action on a confirmed OOT: assessment, potential investigation, impact on the affected lot and on the trend, and any communication to stability or release decisions.
• The cadence: OOT is evaluated when each result is generated, not retrospectively at annual product review.

The most common failure mode is setting OOT limits from too few batches, so the “limits” are just noise, and then either drowning in false alarms or quietly widening the limits until they never fire. Both defeat the purpose. The other classic gap is having an OOT procedure on paper that no one actually runs until the annual product review, at which point the drift has been visible for months. A documented inspection pattern is exactly this: a stability trend walking toward the limit for four timepoints, flagged only when the lot finally went OOS, with the OOT SOP never invoked.

How it ties together: continued process verification (Stage 3)

Stage 3 of the FDA lifecycle is where these tools stop being separate exercises and become one monitoring system. The aim is ongoing assurance that the process stays in the validated state of control during routine production. The same expectation lives in the EU GMP Annex 15 concept of ongoing process verification, and the program-level treatment is in our continued process verification (CPV) article.

A concrete Stage 3 loop:

1. Identify the parameters and attributes to monitor: critical quality attributes and critical process parameters established in Stage 1, informed by risk assessment (ICH Q9, see quality risk management).
2. Collect data per a defined sampling plan: the same statistical-confidence standard the validation guidance demands, executed with random, representative sampling.
3. Plot on control charts (I-MR for batch results, with EWMA/CUSUM where small drifts matter) and apply a defined, deliberate set of run rules.
4. Compute capability periodically (Ppk for the as-delivered long-term picture), having first confirmed statistical control, because capability on an out-of-control process is meaningless.
5. Run OOT detection on the parameters where drift toward a limit is the failure mode, especially stability.
6. Feed signals into the quality system: a control-chart special cause or a confirmed OOT becomes a deviation or change control input, closing the loop back to ICH Q10’s continual improvement expectation, and rolling up into the annual product review (APR/PQR).

The intellectual honesty test for the whole chain is this: each statistic should be traceable to a stated assumption (normality, statistical control, representative sampling, an adequate measurement system) and a documented rationale for its limits. A capability index without a stability check, a sampling plan without its OC-curve/consumer-risk justification, a control chart with limits set on three data points, or an OOT procedure that never actually runs, each is a place where the math is decorative rather than load-bearing.

Roles across the Stage 3 loop

Statistical significance versus practical significance

A large enough data set makes almost any real difference detectable, and a control chart accumulating points over years of continued process verification is exactly that kind of data set. A Nelson trend rule can fire on a genuine, reproducible drift of a fraction of a percent, a shift that is statistically real and, at the same time, has no meaningful effect on product quality, patient safety, or the process’s ability to meet specification. Treating every statistically confirmed signal as equally important either drowns a quality system in low-value investigations, or, worse, trains reviewers to wave off small signals as noise until that habit gets applied to one that mattered.

The corrective is to define practical significance in advance, in the same units as the alert and action limits already in use: how large a shift, sustained for how long, would actually change the risk to the patient or the product, given the margin the process has to its specification. A process running at a Ppk of 2.5 has enormous room, and a statistically real quarter-percent drift there is genuinely low value to chase immediately. The same drift on a process running at a Ppk of 1.1 is a different conversation, because there is almost no margin left to spend. Practical significance is not a fixed percentage figure; it is read off the capability margin the process actually has.

The reverse trap matters just as much. A real, practically important shift, an emerging contamination source, an aging tool, can sit below the detection threshold of a small data set for a long time simply because too few points have accumulated to trip a rule, especially on a low-volume product running only a handful of batches a year. The absence of a statistical signal is not evidence the process is fine; it can just as easily be evidence that there was not yet enough data to see it. Stating the detection power a chart actually has, given its real sample size and sampling frequency, is part of an honest answer to “is this process under control,” not an afterthought to it.

Common mistakes and inspection-finding patterns

• AQL read as an accept rate. Treating “AQL 0.65%” as a promise that lots are 0.65% defective, with no RQL or OC-curve justification for critical defects. Finding: critical defect sampled to a loose AQL with no consumer-risk basis.
• Switching rules ignored. Using Z1.4 tables but never moving to tightened inspection despite a rejection history that requires it.
• Capability on an unstable or non-normal process. A Cpk quoted with no control chart and no normality statement, with an implied PPM that the data do not support.
• Cpk reported, Ppk hidden. Reporting only the flattering short-term index on a process that drifts between subgroups.
• Spec limits drawn on a control chart. Mixing the voice of the customer onto a process-voice chart, inviting tampering toward spec.
• Control limits auto-recalculated every point. Limits that quietly track a drifting process so it never signals.
• Run rules undefined or applied inconsistently. Different analysts calling different patterns, or the full Nelson battery on routine charts drowning the team in false alarms.
• Charts red with no action. Documented signals with no investigation, which is worse than no chart.
• OOT limits from too few batches, or an OOT SOP that only runs at annual review. Drift visible for months, caught only when it finally goes OOS.
• Non-representative sampling. A statistically correct plan executed by pulling the easiest units, invalidating the result regardless of the Ac/Re numbers.
• Non-normal data run through the standard Cpk formula with no check. A skewed or boundary-pinned distribution (particle counts, low-level impurities) produces a defect-rate estimate that does not match the real tail risk, with no normality test, transformation, or distribution fit ever performed.
• A provisional, small-batch Cpk presented as an established capability claim. Four or six batches quoted as “Cpk = 1.8” with no batch count shown and no provisional label, on a process nowhere near the subgroup count the procedure calls established.
• A statistically confirmed trend chased with the same urgency regardless of the margin the process actually has. No stated practical-significance threshold, so a trivial drift on a process running at Ppk 2.5 consumes the same investigation resource as a real drift on a process running at Ppk 1.1.

What an inspector actually probes

• Sampling: “Show me how this n was derived.” They want a statistical justification, not “it’s what we’ve always done.” For critical attributes they expect to see consumer’s risk addressed, often via c=0, and they expect the switching rules to be operated.
• Capability: “Was the process in control when you computed this? Is the data normal? How good is the measurement system?” An unsupported Cpk invites a follow-up that the site usually cannot answer.
• Control charts: “How were the limits established, how often are they recalculated, and what happens when a rule fires?” They look for evidence that signals trigger action, not just a chart that has been red for six months.
• OOT: “How is OOT different from OOS here, and what is your action when an OOT confirms?” They want a real, run-on-time procedure, not an annual-review afterthought.

Interview questions and how to answer them

Q: What is the difference between Cp and Cpk? Cp measures potential capability, the ratio of the spec width to the process spread (6 sigma), and ignores where the process is centered. Cpk adds centering by taking the minimum of the distance from the mean to each spec limit over 3 sigma. Cp equals Cpk only when the process is perfectly centered; Cpk is always less than or equal to Cp.

Q: When do you use Ppk instead of Cpk? Cpk uses within-subgroup (short-term) sigma and assumes statistical control; Ppk uses the overall (long-term) sigma from all individual values and captures drift between subgroups. Use Ppk to report actual delivered performance over time, and compare the two: a large Cpk-to-Ppk gap means the process is wandering between subgroups even though each subgroup is tight.

Q: A point is inside the control limits but a long run of recent points are all above the centerline. Is the process in control? No. That trips the Western Electric long-run-on-one-side rule, a sustained mean shift the 3 sigma limit has not yet caught. It is a special-cause signal and should trigger an assessment even though no single point breached 3 sigma.

Q: Why can’t I just use specification limits as my control limits? They answer different questions. Specs come from the customer or registration and describe acceptable product; control limits come from the process data and describe expected process behavior. A process can be in control and still fail spec, or be capable and out of control. Drawing specs on a control chart encourages operators to adjust toward spec, which adds variation (tampering).

Q: What is the difference between OOS and OOT? OOS fails a registered acceptance criterion and triggers a defined investigation path. OOT is within spec but inconsistent with the historical trend; it is an early-warning signal that triggers an assessment before a result actually fails, most importantly in stability where a passing result can be projected to fail before expiry.

Q: What is an AQL and what does it not tell you? AQL is the quality level a sampling plan is designed to accept most of the time; it sets producer’s risk. It is not a guarantee of outgoing quality and not a promise that lots are at or below that defect rate. The consumer’s protection comes from the RQL/LTPD and the OC curve, which is why critical defects often use c=0 plans designed around consumer’s risk.

Q: Before you compute capability, what do you check? Statistical control (with a chart), normality of the distribution, that the data are representative and span the real sources of variation, and that the measurement system is adequate (Gauge R&R). Capability computed without those checks can be confidently wrong.

Q: Your capability data fails a normality test. What do you do? Do not report a standard Cpk on it as though the formula still applies. Either transform the data (a Box-Cox transformation is the common choice) and compute Cpk on the transformed scale, or fit the actual distribution shape (Weibull, lognormal) and use a percentile-based capability index that reads the 0.135 and 99.865 percentiles off the fitted curve instead of assuming a normal bell curve. State which approach was used.

Q: You only have six batches. Can you claim Cpk 1.5? You can report the number, but not as an established capability claim. State the batch count alongside the index, label it provisional, and treat it as a screening result rather than a release criterion until the data set reaches the subgroup count your procedure calls established, commonly 25 or more.

Q: A control chart signal is statistically real but tiny. Do you have to investigate it with the same urgency as a large shift? Not automatically. Define practical significance in advance, against the margin the process has to specification: a small drift on a process running at Ppk 2.5 is low value to chase immediately, while the same drift on a process running at Ppk 1.1 deserves real attention, because there is almost no margin left. The trigger for opening an assessment stays the same; the urgency and depth of the response scales with the margin.

None of this requires a statistician’s depth on day one. It requires understanding what each tool assumes, what it actually tells you, and where it lies to you if you skip the assumption check. That is the statistical literacy the role demands, and it is mostly about being honest with yourself about what a sample of the whole can, and cannot, prove. For the broader process context, see process validation lifecycle and quality metrics and KPIs; for the clinical-trial counterpart of these methods, see trial design statistics (ICH E8/E9).

References (cited by name)

• FDA, Guidance for Industry: Process Validation, General Principles and Practices (2011): lifecycle model, statistical-confidence standard for sampling.
• FDA, Guidance for Industry: Investigating Out-of-Specification (OOS) Test Results for Pharmaceutical Production (originally 2006, Revision 1 May 2022): OOS investigation framework (contrast with OOT).
• ICH Q9(R1) (Quality Risk Management), ICH Q10 (Pharmaceutical Quality System).
• EU GMP Annex 15 (Qualification and Validation): ongoing process verification.
• ANSI/ASQ Z1.4 (attribute sampling; ISO 2859-1 equivalent) and ANSI/ASQ Z1.9 (variables sampling; ISO 3951 equivalent).
• 21 CFR 211.110 (Sampling and testing of in-process materials and drug products): control procedures to monitor output and validate process performance.
• 21 CFR 820.250 (Statistical techniques), now folded into the Quality Management System Regulation via harmonization with ISO 13485:2016: supporting hook for combination products and software in a medical context.
• Squeglia, Zero Acceptance Number Sampling Plans: c=0 consumer-risk-based plans.
• Western Electric, Statistical Quality Control Handbook (1956); L. S. Nelson, control-chart run rules (Journal of Quality Technology).
• ISO 9001:2015 (clause 9.1.1) and ISO 13485:2016: requirements for statistical techniques in the quality management system.
• PhRMA CMC Statistics and Stability Expert Teams, out-of-trend detection methods for stability data (Pharmaceutical Technology, 2003).
• ISO 22514-2, Statistical methods in process management, Capability and performance, Part 2: Process capability and performance of time-dependent process models: capability methods for non-normal and time-dependent processes; replaced the earlier ISO 21747:2006, which is withdrawn but still referenced by that number in some legacy tooling and literature.
• Box, G. E. P. and Cox, D. R., “An Analysis of Transformations,” Journal of the Royal Statistical Society, Series B (1964): origin of the Box-Cox transformation.
• Taguchi, G.: origin of the Cpm capability index for deviation from target.