Log: Vendor-Hosted AI Model Version Pin Verification

This is a ready-to-use log for a vendor-hosted or API-delivered AI or machine learning model that your organization has validated as locked. It exists because of a gap the rest of the AI documentation set does not close: Record: AI prompt, model version, and retrieval corpus change control documents a change once you know one happened, and Log: SaaS vendor release notes review documents review of a vendor’s published release notes, but a served model swapped in behind an unchanged name, endpoint, or alias produces no release note and no change ticket for either document to catch. This log is the routine, dated check that actually looks for that swap, whether or not the vendor said anything, and that tracks the vendor’s own deprecation timeline for the version you locked. Replace every <<FILL: ...>> placeholder, run it on the cadence your risk assessment sets, and route any mismatch into the change control record above. A worked filled specimen follows the template.

Document control header

1. Purpose

A locked, validated model is only actually locked if you can show, on a recurring basis, that what the vendor is serving today is what you validated. Two failure modes make that assumption fail silently. First, a vendor updates the weights behind a stable model name or alias with no version-string change and no announcement, because the vendor does not consider a quality improvement to their own product a customer-facing event. Second, a vendor deprecates the specific pinned version you validated on a fixed timeline, and if nobody is watching for the notice, the pin expires with no plan in place. This log gives one place to show, entry by entry, that someone actually checked, rather than assuming the model was still what the validation report says it is.

2. Scope

Run this log for any AI or machine learning model where the model itself, not just the surrounding application code, is a vendor-hosted or third-party-hosted service and is the validated, locked artifact: a commercial large language model API, a hosted classification or embedding endpoint, or a machine learning feature embedded in a SaaS platform where the underlying model is the vendor’s, not a model you trained and deployed yourself. It does not cover a model you host and control end to end, where your own change control governs every version change directly, nor does it replace the per-change record raised once a mismatch here is confirmed.

3. Field table

4. Verification methods

Use whichever of these your vendor and model actually support; most programs use more than one.

• Exact version string from API metadata. Many vendor APIs return a specific, dated model identifier in the response metadata, distinct from the alias or family name used in your configuration (for example, an alias like “current” or “latest” may resolve to a different specific version over time). Capture and compare the exact string returned, not the alias you configured.
• Canary-prompt or canary-input comparison. For a model with no exposed version string, or as a second check even when one is exposed, maintain a small, fixed set of inputs and their expected outputs or output hashes captured at validation. Re-run the set on the verification cadence and compare. A vendor can hold an alias’s name constant while quietly changing what it points to; a canary set catches a behavior change a version string does not reveal, and it is also the only method that would catch a version-string change that happened without any behavior change at all, which tells you the swap itself is a fact regardless of whether it mattered yet.
• Vendor status page, changelog, or support confirmation. Where the vendor publishes a model changelog, deprecation calendar, or status page, check it directly rather than waiting for an email that may not arrive; subscribe to the vendor’s notification channel where one exists, and record here that the channel was actually checked.

5. Escalation rule

A mismatch found here is not itself a deviation; it becomes one if it is not acted on. The finding, promptly investigated and routed into the change record, is the control working as intended.

6. Instructions

1. Set the verification cadence in the header from your risk assessment; a high-impact model warrants weekly checks, a lower-tier assistive tool may run monthly, but never less often than your monitoring report’s own reporting period, because this check is a precondition for that report’s “the model is still what we validated” statement being true.
2. Build the canary set before the first check, not after a mismatch is suspected. Draw it from validation-time cases, keep it small enough to run cheaply and often, and store its expected outputs or hashes alongside the model card.
3. Run the check on schedule regardless of whether anything seems wrong; the entries with nothing to report are exactly what proves the discipline is routine and not reactive.
4. On any mismatch or deprecation notice, complete the escalation reference field the same day the action is raised, not retroactively when the change record closes.
5. Bring this log to periodic review and reconcile its entry count against the cadence; a gap in the log is treated the same as a gap in a monitoring report.
6. Retain with the model’s validation file for the same period as the validation and monitoring records for the system.

7. Acceptance criteria

• Every scheduled check produced an entry, with no unexplained gap in the cadence.
• Every entry states the method used and, where the API exposes one, the exact version string returned, not the configured alias.
• A model with no exposed version identifier and no canary set is flagged as a gap, not silently accepted as unverifiable.
• Every mismatch or deprecation notice has a recorded action and, where applicable, an escalation reference to the change control record.
• A deprecation date on file has a planning item opened with enough lead time to validate a replacement before the sunset date arrives.

8. References

21 CFR Part 11 and EU GMP Annex 11 (computerised systems), for the electronic record controls applied to this log and for the periodic evaluation expectation that a locked system remains in its validated state. ICH Q9(R1), Quality Risk Management, for setting the verification cadence proportionate to the model’s risk tier. GAMP 5 Second Edition (ISPE, 2022), for the supplier and lifecycle-management expectations this log operationalizes for a vendor-hosted model, described here in original wording. FDA guidance, “Computer Software Assurance for Production and Quality Management System Software” (current version issued 3 February 2026), for the risk-based assurance thinking behind setting the check cadence to the model’s actual risk rather than a fixed default. EU Artificial Intelligence Act, Regulation (EU) 2024/1689, to the extent the vendor’s underlying model falls within its general-purpose AI model provisions; confirm current applicability and any provider disclosure obligations before relying on them.

Confirm the current version and status of each reference before issue. This is general guidance to adapt to your own quality system, not legal or regulatory advice.

9. Revision history

10. Approvals

Filled specimen

The following shows four weeks of entries for an illustrative vendor-hosted deviation-triage assistant built on a commercial large language model API, so you can see the level of detail expected. The vendor, model, and results are invented for illustration.

In this example, three of four checks came back clean, and the one canary divergence was investigated the same week rather than waved off, closing with a documented, reasoned conclusion instead of silence. When the vendor’s deprecation notice arrived in week four, it went straight into a tracked planning item with a four-month runway to validate a replacement version, rather than becoming a surprise on day 119.

Common inspection findings this log prevents

• A model documented as “locked” that has never actually had its served version checked against the validation report since go-live.
• A vendor’s silent weight update behind an unchanged model name or alias, discovered only when production behavior visibly degrades, months after it could have been caught by a routine check.
• A vendor deprecation notice that arrived by email, was never read by anyone accountable for the model, and the pinned version stopped working with no transition plan in place.
• A model with no exposed version identifier and no canary set, so “we cannot tell if it changed” is treated as an acceptable answer rather than a documented gap.
• A canary or version-string mismatch found informally by an engineer, discussed verbally, and never routed into a change record, so no evidence exists that it was assessed.

How to adapt this log

1. Set the locked reference version, the vendor, and the verification cadence in the header from your risk assessment and validation report.
2. Build the canary set from real validation-time cases before the first check, and store expected outputs or hashes with the model card so a future reviewer can confirm the comparison basis.
3. Point the escalation row directly at your own change control record for AI model changes, and confirm the tier classification logic in that record covers “vendor-forced, discovered via this log” as a trigger, not only “vendor-forced, discovered via a notice.”
4. If your vendor exposes no version identifier and offers no changelog, say so explicitly in section 4 and rely on the canary set as the sole method, rather than leaving the row blank.
5. Bring this log, not just the vendor’s own version history, to periodic review, and reconcile the logged cadence against the calendar to confirm no cycle was skipped.