SOP: Independent Statistical Programming and TLF Validation

This is a ready-to-use SOP for validating the tables, listings, figures, and analysis datasets produced by clinical statistical programming. Replace every <<FILL: ...>> placeholder with your own specifics, set your document numbers and dates, and route it through document control. A worked filled specimen follows. This is general educational content to adapt to your organization and study, not regulatory advice; confirm each cited guideline against the current source.

Document control header

1. Purpose

To define how <<FILL: COMPANY / FUNCTION>> validates analysis datasets (ADaM) and tables, listings, and figures (TLFs) so that every reported number reflects the statistical analysis plan (SAP), can be reproduced, and can be traced to its source. The objective is to catch analysis and presentation defects before delivery to the clinical study report or submission.

2. Scope

This procedure applies to all ADaM datasets and TLFs that support a clinical study report, an interim analysis, or a regulatory submission. It covers SDTM-to-ADaM derivation, ADaM-to-TLF generation, and figures. It does not cover SDTM creation or database lock, which are governed by <<FILL: SOP-ID for data management>>, nor the writing of the SAP, governed by <<FILL: SOP-ID for the SAP>>.

3. Responsibilities

4. Definitions

• Independent double programming: a second programmer writes a fully separate program from the specification without viewing the production code, and the two outputs are compared programmatically.
• Automated compare: a utility (for example PROC COMPARE in SAS, or an equivalent in R) that reports every difference between two datasets or outputs.
• Validation tier: the risk-based level of validation assigned to an output before production (see section 5.1).
• Discrepancy: any difference the compare reports, or any output that does not match its approved shell.

5. Procedure

5.1 Assign the validation tier before production

Assign each output a tier based on the consequence of an error, and record it before programming. Do not lower a tier after a difference is found.

5.2 Fix the specification first

The SAP, the table shells, and the ADaM specification must be approved and version-controlled before production programming begins, and before unblinding where timing matters. Programming against a moving specification is out of control.

5.3 Program independently

1. Assign two programmers to Tier 1 and Tier 2 outputs: one production, one validation. Record both assignments.
2. Both program from the approved specification. The validation programmer does not open the production code and, ideally, does not reuse the same template.
3. Validate ADaM at two levels: conformance (run the CDISC conformance checker; resolve every error; justify every retained warning; reconcile the Define-XML to the data) and correctness (independently reproduce the high-risk derivations: baseline, change from baseline, treatment-emergent flag, and every population and analysis flag).

5.4 Compare programmatically

1. Run the automated compare of the two datasets or outputs.
2. A pass requires zero unequal values, equal observation counts, and matching key variable attributes. A compare that matches “except for rounding” is not a pass; it means the SAP rounding rule was implemented two ways.
3. Save the compare log as evidence.

5.5 Review the program logs

1. Review both the production and validation program logs, not just the outputs.
2. A pass requires no unresolved errors, no unexpected warnings, and no problem notes (for example an unintended many-to-many merge, an uninitialized variable, or missing values generated by coercion).
3. Fix or document a rationale for every flagged message. Retain the log with the output as a record.

5.6 Reconcile discrepancies

1. Investigate each discrepancy until you know which program was correct and why.
2. Correct the program that had the defect and rerun the compare.
3. Record the discrepancy, the root cause, and the resolution in the discrepancy log.

5.7 Confirm cross-output consistency and shells

1. Confirm subject counts and denominators reconcile across related outputs (demographics, disposition, population counts).
2. Confirm each output matches its approved shell in structure, titles, footnotes, and decimal precision, and that footnotes still describe the current analysis after any SAP amendment.
3. Reconcile figures to their companion tables (numbers at risk, censoring, medians, axes).

5.8 Sign off

1. A reviewer who is neither of the two programmers confirms the compares are clean, the logs are reviewed, the discrepancy log is closed, cross-output consistency holds, and outputs match the shells.
2. That reviewer signs the validation record. The signer is a third party, not either programmer.

6. Acceptance criteria

An output is validated when: it was assigned a tier before production; the production and validation programs were written from the approved specification without code sharing (Tier 1 and 2); the automated compare shows zero unequal values, equal observation counts, and matching attributes; the program logs are reviewed and retained with no unresolved errors, warnings, or problem notes; every discrepancy has a documented root cause and resolution; the output matches its shell; subject counts reconcile across related outputs; and a third party has signed the validation record.

7. Records generated

• Validation tier assignment for each output
• Automated compare logs
• Program logs (production and validation), retained
• Conformance report and Define-XML reconciliation (for ADaM)
• Discrepancy log
• Signed validation record

8. References

ICH E6(R2), Good Clinical Practice, sponsor quality management and computerized-system expectations. ICH E9, Statistical Principles for Clinical Trials (pre-specified analysis). FDA Study Data Technical Conformance Guide (CDISC-conformant standardized data). CDISC ADaM and Define-XML standards (referenced by title; describe in original wording).

Confirm the current version of each reference before issue.

9. Revision history

10. Approvals

Filled specimen

The following shows a completed validation record for one Tier 1 output, so you can see the level of detail expected. Names and numbers are illustrative.

In this example the second program found a denominator defect that would have understated the placebo rate and inflated the treatment difference in the primary endpoint. The compare flagged it, reconciliation traced it to a visit-window handling error against the SAP, the production program was corrected, and the log review separately caught an uninitialized variable. Finding to reconciliation to corrected rerun to third-party sign-off is exactly the sequence an inspector expects to see.

Common inspection findings this SOP prevents

• The primary endpoint table produced and “reviewed” but never independently programmed.
• Validation that was not actually independent (validator saw the production code, or both shared a defective template).
• A delivered output that cannot be matched to the program version and dataset version that produced it.
• Program logs not reviewed, so a silent merge or coercion defect shipped behind a clean-looking table.
• Discrepancies closed with no documented root cause.
• Subject counts that disagree across tables because population subsets were applied inconsistently.

How to adapt this SOP

1. Set your document number, owner, and effective date, and point the cross-references in section 2 to your real data-management and SAP procedures.
2. Adjust the tier examples in section 5.1 to your therapeutic area and output set, keeping the primary and key safety outputs in Tier 1.
3. Name the compare utility and the conformance checker your organization uses.
4. Decide and state your standard for a clean log in your language of choice (SAS and R differ), and reference the log-review checklist you use.
5. Confirm every reference in section 8 against the current source before issue.