Independent and not affiliated with the FDA, MHRA, ISPE, PDA, or any agency. Get the appgoutham@madhadi.com
madhadi.comData Integrity & GxP Quality
Browse all topics -> Articles Templates & Procedures Learning paths GlossaryScenariosToolsRegulatory ReferencesLearning PathsTopics About Start here
Advanced CSV / CSA

Security Event Logging, Monitoring, and Review for GxP Systems

How to capture, store, alert on, and independently review the system-level security events that protect a GxP record, treating the infrastructure log layer as a data integrity control in its own right.

Most data integrity programs put enormous effort into the application audit trail and almost none into the layer underneath it. That is a mistake, and inspectors have started to notice. The application audit trail tells you that user jdoe changed an assay result from 98.2 to 99.1. It does not tell you that two minutes earlier an administrator logged into the database directly, ran an UPDATE statement against the results table, and disabled the audit trail for ninety seconds while doing it. The application has no idea that happened. The thing that knows is the system log: the operating system security log, the database server log, the application server log, the directory service log. Those logs are a separate data integrity control, and treating them as one is what separates a mature program from a paper one.

This article covers the security event logging layer end to end: which events actually matter for integrity, how to get them off the host where the threat lives and into a tamper-evident store, how to alert on the handful that warrant a same-day response, and how QA reviews administrator activity without IT marking its own homework. If you are coming from the application side, read audit trail design and review first, then come back here, because the central idea of this article is that the system log is a distinct control that the application audit trail cannot replace. Access control mechanics are covered in CSV cybersecurity and access control; this article assumes those controls exist and asks the next question: how do you prove they are working and catch it when they fail.


The system log as a data integrity control

Why the two layers are not the same thing

The application audit trail and the system security log answer different questions, and a defensible program needs both.

QuestionAnswered by
What did a user change inside the application?Application audit trail
Who logged into the application, and when?Application audit trail (usually)
Who logged into the operating system or database hosting the application?System security log
Was the application audit trail turned off, and by whom?System log and application config log, not the application data trail
Did anyone edit the database directly, bypassing the application?Database server log only
Did the system clock move?Operating system log
Did a privileged account get created or have its rights changed?Directory service and OS security log

The pattern that should worry you: every question that an attacker or a careless administrator would most want unanswered is answered only by the system log, never by the application trail. A user who wants to falsify a result and cover the tracks does not do it through the application, because the application records it faithfully. They do it underneath, where the application is blind. If the only integrity control you have is the application audit trail, you have built a very good lock on the front door and left the back wall open.

Why it is required (regulation and risk)

The regulatory hook is less explicit than it is for the application audit trail, which is part of why this layer gets neglected, but it is firmly there. 21 CFR Part 11.10(d) requires that closed systems limit system access to authorized individuals, and 11.10(g) requires authority checks to ensure only authorized individuals can use the system, electronically sign records, access the operation, or alter a record. You cannot demonstrate that access was limited to authorized individuals unless you log access attempts and review them. 11.10(e) requires secure, computer-generated, time-stamped audit trails; if a privileged user can disable that trail, the act of disabling it has to be recorded somewhere the same user cannot reach, which is the system log on an off-host store.

Persons who use closed systems to create, modify, maintain, or transmit electronic records shall employ procedures and controls designed to ensure the authenticity, integrity, and, when appropriate, the confidentiality of electronic records. (21 CFR 11.10, opening clause)

“Authenticity” and “integrity” are the operative words. An audit trail you cannot prove was running, on a host whose administrator access you do not monitor, supports neither.

In the European Union, EU GMP Annex 11 addresses security directly. Its access-control clause requires a computerized system to be locked down to people who are genuinely authorized to be in it, through whatever mix of physical and electronic barriers actually fits the system: a badge reader on the server room, a hardware token, a password tied to a named account, a fingerprint or other biometric step, none of them mandatory in isolation, all of them acceptable building blocks in a defensible design. A related provision in the same clause goes further than keeping the wrong people out and requires the system itself to record, for every touch a record receives, who was behind it and precisely when, regardless of what form that touch took: a fresh entry, a change to something already there, a sign-off, or an outright removal. The MHRA “GXP Data Integrity Guidance and Definitions” (March 2018) and PIC/S PI 041 both treat the segregation of system administration from the people who own the data as a core integrity expectation, and you cannot enforce or evidence that segregation without logging what administrators do. The risk rationale is simple: the highest-privilege accounts on a GxP system are the ones with the means and the access to falsify records invisibly, so they are the accounts whose activity most needs an independent record.


Which events to capture

The trap here is to either log nothing useful (default OS settings rarely capture what matters) or to log everything and drown. The discipline is to capture the integrity-relevant events completely and to be deliberate about the rest. The list below is the set that matters for a GxP system, regardless of whether the system is a LIMS, a chromatography data system, an MES, a historian, or the database and servers underneath them. Every event runs the same short gate:

A security event occursLogin, configuration change, account change, clock change, direct database access, log action.
→
Integrity-relevant?Could it hide, enable, or evidence a change to a GxP record? If not, log it lightly with a documented rationale.
→
Capture in full, off-hostWho, what, and when, forwarded to the tamper-evident store in near real time.
→
Time-sensitive?Would waiting for the periodic review let harm land? If yes, alert now. If not, route it to the independent review.

The integrity-relevant event set

EventWhy it is integrity-relevantWhere it is logged
Audit trail disabled, paused, or its configuration changedDirect attack on the primary integrity controlApplication config log and OS/DB log
System or instrument clock changeBackdating a late entry to look contemporaneousOS security log
Privileged or administrator login (interactive and remote)The accounts with means to falsify records invisiblyOS, DB, and directory logs
Direct database access outside the applicationBypasses the application audit trail entirelyDB server log
Failed-login spikes against an accountBrute force, credential stuffing, or a shared-account fishing attemptOS, application, directory logs
Account creation, deletion, enable, or disableUnauthorized access provisioning; orphaned accountsDirectory and application logs
Privilege or group-membership changeQuiet escalation to administrator or to a data-editing roleDirectory and application logs
Security log cleared or rotated off-scheduleDestroying the record of the aboveOS security log
Logging service stoppedGoing dark before doing somethingOS log
Changes to the logging or forwarding configurationRedirecting or muting the evidence streamOS and forwarder logs
Backup deletion or restore from backupA restore can silently roll a record back to an earlier stateBackup system and DB logs
Account lockout and lockout resetLockout reset by an admin can mask a brute-force attemptOS and directory logs
NTP source changed, unreachable, or time service stoppedA host that has lost its time reference can drift silently, or be pointed at a source nobody controlsOS time-service log
Clock offset exceeds the documented drift toleranceDrift this large means the timestamp on every record written during the window is no longer trustworthyNTP client/monitoring log

If you capture only these and capture them completely, you have the events an inspector will ask about and the events an investigation will need. The first three rows, audit-trail-disabled, clock change, and privileged login, are the ones to build first if you have to phase the work.

A clock change caught in the moment it happens is only half the picture, because a clock can also go wrong by doing nothing: a time server stops answering, a firewall rule quietly blocks the poll, and the host’s clock drifts for weeks before anyone notices, the exact “NTP configured once, never monitored” failure named in time stamps, NTP synchronization, and time-zone control. The architecture and governance of the NTP hierarchy, stratum design, traceability to a reference source, virtualization and container caveats, belong in that article and the paired time synchronization and clock control SOP. This article’s job is narrower and specific to the logging layer: capture when a host’s time source changes or disappears, capture when measured offset crosses the tolerance that SOP defines, and route both into the same alerting and review path as every other integrity-relevant event. A trustworthy timestamp is not a one-time qualification result; it is a clock whose ongoing health is itself a monitored security event.

What you can be lighter on

Routine application reads, successful standard-user logins to a low-risk system, and verbose debug output are not integrity-relevant in themselves and can be logged at a lower retention or excluded with a documented rationale. The judgment is risk-based: tie the depth of logging to what the system does. A system that takes raw analytical data and feeds batch release decisions earns the full set; a read-only reporting front end that touches no source record can carry less. Write the rationale down in the system’s data integrity assessment so the scoping decision is defensible rather than accidental.

Tiering the depth by system risk

“Risk-based” is easy to say and hard to apply consistently across a portfolio of forty systems unless it is written down as a tier definition, the same discipline audit trail design and review applies to the application layer. Use the system’s own data criticality rating, already assigned for validation and audit-trail-review scoping, as the input, so the security-logging tier and the audit-trail tier are set from the same answer rather than two separate judgment calls that can drift apart.

TierTypical systemMinimum captureOff-host forwardingAlerting
HighCDS, LIMS, MES, or historian feeding release or disposition decisionsFull integrity-relevant event set, all twelve rowsNear real time, every hostAll six real-time alert types enabled
MediumDocument management, training records, in-process monitoring that does not gate releaseAudit-trail-disabled, privileged login, account changes, clock events; direct-DB-access logging if the system has a database tierNear real time or scheduled hourlyAudit-trail-disabled and privileged-login-spike at minimum
LowRead-only reporting layers, reference or lookup data with no write path to a GxP recordPrivileged login and account changes onlyDaily batch acceptableNone required; covered at periodic review only

A system does not get to pick its own tier by self-report. The tier comes from the data criticality assessment the system owner already completed, cross-checked by QA, and recorded in the system inventory alongside the audit-trail tier so an inspector sees one consistent risk rating driving both controls, not two independent and possibly contradictory ones. Re-tier a system whenever its role changes, for example when a reporting layer that was read-only gains a write-back feature; the logging scope that was correct on day one is not automatically still correct after the system’s function changed under it.

How to turn capture on (step by step)

  1. Start from the integrity-relevant event set above and map each row to the specific log source on this system: which OS audit policy subcategory, which database audit specification, which application security setting, which directory audit policy.
  2. Enable the corresponding audit policy on each host. On Windows this is Advanced Audit Policy (Logon/Logoff, Account Management, Account Logon, Policy Change, System, and object access on the audit-trail tables and config files). On Linux this is the audit daemon (auditd) with rules watching the relevant binaries, config files, and the audit configuration itself. On the database it is native auditing scoped to logins, DDL, direct DML on GxP tables, and audit-config changes.
  3. Set object-level auditing on the artifacts that hold or control the integrity record: the audit trail table or files, the application configuration that enables them, the logging configuration, and any credential store.
  4. Configure forwarding off the host immediately (see the next section) so that the moment events start being captured they also start leaving the host.
  5. Qualify the capture: in OQ or a focused configuration verification, perform each integrity-relevant event in a controlled way and confirm it appears in the log with the right detail. This is the test that proves the logging works, and it is the test most often skipped.

Acceptance criteria for capture. Each event in the integrity-relevant set produces a log entry containing who (a unique, attributable identity, never a shared account), what (the specific action and target), and when (a trustworthy timestamp). The capture survives a host reboot and is enabled by a controlled configuration that is itself under change control. A test that deliberately triggers each event during qualification finds the matching log entry every time.


Off-host, tamper-evident storage

Why local logs are not enough

Here is the failure mode the whole storage design exists to defeat. An administrator on the host has, by definition, the rights to edit or clear the logs on that host. If the only copy of the security log lives on the same machine as the activity it records, the person most able to do something they should not is also the person most able to erase the evidence. A local log is a control that the controlled party administers, which is no control at all for the highest-risk events.

The fix is to get the log off the host the moment it is written, into a store that the host’s administrators cannot reach with the same credentials. This is sometimes called a SIEM (Security Information and Event Management) platform, sometimes a central log server, sometimes a log archive. The name matters less than three properties.

The three properties an off-host store must have

PropertyWhat it meansHow it is achieved
Separation of dutiesThe people who administer the GxP hosts cannot administer or delete from the log storeDifferent administrator group, ideally a different team (security or a neutral IT function)
Tamper evidenceAny alteration or deletion of stored logs is itself detectableWrite-once or append-only storage, cryptographic hashing or chaining, immutability windows
Near-real-time forwardingEvents leave the host fast enough that a local wipe does not also lose the central copyStreaming forwarder (syslog over TLS, agent-based forwarding) rather than nightly batch pulls

Tamper evidence does not require exotic technology. Append-only or write-once-read-many (WORM) storage, an object store with an immutability lock for the retention period, or periodic hashing of log batches with the hashes stored separately all qualify. The standard you are meeting is the same one the application audit trail meets under Part 11.10(e): record changes shall not obscure previously recorded information, applied now to the log store itself.

How to set it up (step by step)

  1. Stand up a log store outside the administrative control of the GxP system owners. If IT runs the GxP servers, the log store should be owned by security or by a separate IT function, with access that the server administrators do not hold.
  2. Configure each GxP host to forward the integrity-relevant events over an encrypted channel (TLS-protected syslog or an authenticated agent) in near real time, not as a scheduled batch.
  3. Enable immutability on the store: WORM, an object-lock retention policy, or hash-chaining so that any deletion or edit of an archived event is detectable.
  4. Set retention to at least the GxP record retention period for the records the system supports, and confirm the store enforces it. Security logs that prove the integrity of a batch record need to outlive the batch record.
  5. Monitor the forwarding path itself. A host that stops forwarding is a host that has gone dark, intentionally or not, and that gap is exactly when something would be hidden. Alert on forwarding silence.
  6. Qualify the store and the path as infrastructure: include log forwarding and retention in infrastructure qualification, with a test that a forwarded event arrives intact and that an attempted deletion from the store is prevented or detected.

Verifying the tamper-evidence still holds

Qualifying the store at installation proves it was tamper-evident on one day. An object-lock policy can lapse at a storage-tier migration, a hash-chaining job can silently stop running, a WORM retention setting can get reset during a platform upgrade, and none of those failures look like anything until the day someone actually needs the evidence and finds it was quietly degradable the whole time. Treat the control the same way you treat the capture itself: periodically re-prove it, not just qualify it once.

  1. On a defined schedule (at minimum annually, more often for a high-tier system), attempt a controlled deletion or alteration of a test event already committed to the store, in a non-production copy or a designated test partition if the platform supports one.
  2. Confirm the attempt is either blocked outright or produces a detectable integrity failure (a hash mismatch, a broken chain, a logged denial), never a silent success.
  3. Independently recompute the stored hash or chain value for a sample of archived events and confirm it matches the value recorded at ingestion.
  4. Confirm the object-lock or WORM retention setting is still active and still set to a period at or beyond the records it protects; a setting can be correct at go-live and quietly shortened later by someone with storage-administrator rights who has nothing to do with the GxP system.
  5. Document the test, the result, and the reviewer in the same record used for infrastructure periodic review, and raise a deviation immediately if any step fails; a degraded tamper-evidence control is itself an integrity finding, not a maintenance ticket.

Acceptance criteria for storage. Security events arrive at the off-host store within minutes of occurring; the GxP host administrators cannot delete or alter events in the store using their host credentials; any tampering with stored events is detectable, and that detectability is periodically re-tested rather than assumed from the original qualification; retention meets or exceeds the supported records’ retention; and a forwarding outage raises an alert rather than passing silently.


When the system is SaaS, and you do not own the host

Everything above assumes you can reach the operating system and the database, set an audit policy, and forward the logs yourself. For a growing share of GxP systems, a cloud-hosted LIMS, an eQMS, a validated document system, that assumption is false. You do not administer the host, you cannot enable native database auditing, and the machine where the data actually lives belongs to the vendor. FDA’s Computer Software Assurance guidance, in its 3 February 2026 version, added explicit treatment of cloud models (software, platform, and infrastructure as a service) and expanded what it expects around vendor oversight, precisely because so much GxP software now runs this way. The logging problem does not disappear in the cloud; it moves across a contract line, and you have to work it through the vendor instead of through a host you control. The validation of these systems is covered in cloud and SaaS validation; this section is the security-logging cut of the same shared-responsibility question.

What splits, and who owns which log

The shared-responsibility model divides the stack. As a rule of thumb, the higher up the stack the service sits, the more of the logging the vendor holds and the less you can set yourself.

LayerTypical owner in SaaSSecurity logs you can usually get
Physical and network infrastructureVendorLittle directly; rely on the vendor’s certifications and reports
Operating system and database hostVendorVendor-held; visible to you only if the vendor exposes an admin-access or audit feed
Application (your tenant)SharedThe application audit trail and an admin-activity or security-event log, if the product provides one and you turn it on
Identity and accessOften youAuthentication, single sign-on, and privileged-access logs from your own identity provider
Tenant configurationYouWho changed a tenant setting, a role, or an integration, from the application’s config log

The events you can see and control shift toward identity and configuration; the events at the host and database layer, the exact layer where the on-premises version of this article told you the invisible falsification happens, are now behind the vendor’s operational boundary. That is the risk to name in the system’s data integrity assessment: a vendor engineer with production database access could, in principle, touch your data in a way your tenant logs never show, the same structural gap as a local administrator editing a local log, relocated to the supplier.

What to demand before you sign, and monitor after

You close that gap through the agreement and the supplier assessment, not through a host you cannot reach. What to require:

  1. A tamper-evident audit log that covers privileged and vendor-staff access to your tenant’s data, not only end-user actions inside the application.
  2. Export or streaming of security events to your own store or SIEM (sometimes called log egress or an audit-log API), so your independent review draws from a copy you hold, not only from a console the vendor controls.
  3. Retention of those logs that meets or exceeds the retention of the records the system supports, contractually, since you cannot enforce it on their storage yourself.
  4. Prompt notification of security events and confirmed incidents affecting your data, with a defined timeline, written into the quality or technical agreement.
  5. Evidence that the vendor segregates its own administrative duties and independently reviews its privileged staff’s access to customer data, ideally shown through an independent audit report against a recognized control framework, plus a right to audit.
  6. Clarity on which security responsibilities are yours (identity, roles, tenant configuration, the events those generate) so nothing falls in the gap because each side assumed the other had it.

The SaaS security event logging vendor assessment checklist turns this list into a document you actually complete and keep, at onboarding and again at each periodic vendor review, rather than a set of intentions that lived only in the negotiation.

After go-live the review does not stop, it retargets. You reconcile the identity-provider and tenant-configuration logs you do control, ingest whatever tenant audit feed the vendor exposes, and monitor the egress itself: a SaaS audit feed that goes silent is the cloud version of a host that stopped forwarding, and it deserves the same alert. Where the vendor holds a log you cannot get, the honest entry in the assessment is that the control is contractual and monitored through vendor oversight, not technical and enforced by you. Naming that plainly is far stronger than pretending a tenant audit trail sees the whole host. The wider vendor-governance discipline is in software supplier assessment and supplier and vendor qualification.

Acceptance criteria for the SaaS case. The data integrity assessment names which security logs are yours, which are the vendor’s, and which residual events neither side can see; the agreement commits the vendor to a tamper-evident privileged-access log, event export or streaming, retention meeting the records it supports, and timely incident notification; your independent review draws from an exported copy you hold plus your own identity and configuration logs; egress silence raises an alert; and the vendor’s segregation of duties and independent review of its own staff are evidenced, not assumed.


Alerting: the events that warrant a same-day response

Most security events are reviewed periodically. A small subset cannot wait for a periodic review because by the time the monthly review runs, the damage is done and the window to investigate has closed. Alerting is the control that handles those.

The error to avoid is alert fatigue: wire up a hundred alerts, get five hundred a day, and the reviewer learns to ignore all of them, including the real one. The discipline is to alert on the few events that are both high-signal and time-sensitive, and to send everything else to the periodic review instead.

What to alert on in real time

AlertWhy it cannot waitTypical response
Audit trail disabled or paused on a production GxP systemThe integrity control is off right now; any data created in that window is suspectImmediate investigation; quarantine affected records
System clock moved backward outside a known maintenance windowActive backdating risk against a contemporaneous recordImmediate investigation; identify records timestamped in the affected window
Logging service stopped or forwarding silent on a production hostThe system has gone dark; you are blind until it is fixedRestore logging; investigate the gap
Failed-login spike above threshold against a privileged accountBrute force or credential attack in progressLock the account; investigate source
New account added to an administrator or data-editing groupUnauthorized privilege escalationVerify against an approved access request; revoke if none
Security log cleared on a production hostDirect destruction of evidenceImmediate investigation; rely on the off-host copy

Tune the failed-login threshold to the system. Three failures might be normal for a forgetful user; thirty failures in two minutes against a service account is an attack. Set the threshold from observed baseline behavior, document it, and revisit it, because a threshold that fires constantly trains people to ignore it and a threshold that never fires is not actually a control.

What does not need an alert

Single failed logins, routine successful logins, scheduled maintenance windows that were approved in advance, and the ordinary day-to-day administration that the periodic review will catch. Route these to the review queue, not to someone’s phone at 2 a.m. The credibility of the alerting channel depends on it being quiet most of the time.

Acceptance criteria for alerting. Each real-time alert reaches a named, available responder by a defined route; thresholds are documented and based on baseline behavior; a test alert (a deliberately triggered audit-trail-disable in a non-production copy, for example) reaches the responder within the target time; and a documented procedure defines the response, the escalation, and the record of action for each alert type. False-positive rate is low enough that responders still act on alerts.


QA-independent review of administrator activity

Why independence is the whole point

Periodic review of the security log is where the system layer earns its keep, and the single most important attribute of that review is independence. The review has to be done by, or formally on behalf of, someone who is not the administrator whose activity is under review. IT reviewing its own administrators is the audit equivalent of marking your own homework: the conflict is structural, not a matter of anyone’s good faith.

This is the direct analogue of the rule for the application audit trail: the people who generate the records should not be the sole reviewers of the trail of those records. For the system log the records are administrator actions, so quality assurance, or a security function reporting outside the line being reviewed, owns the review. The reviewer does not need to be able to run a database; they need to be able to read the log, recognize what is normal, and challenge what is not.

What the review covers

  1. Every privileged login on the production GxP systems in the period: was each one performed by an authorized administrator, for a documented reason, ideally tied to an approved change or incident ticket.
  2. Every audit-trail configuration change, every clock change, and every logging-service stop: each explainable from a contemporaneous record (a planned-maintenance entry, a change record, a documented NTP resync), with anything unexplained raised as a deviation.
  3. Every account creation, deletion, and privilege change: reconciled against approved access requests, with any change lacking an approval treated as a finding. Pair this with the periodic access review described in CSV cybersecurity and access control.
  4. Every direct database access outside the application: each justified by a documented, approved task (an emergency data fix under change control, for example), with unjustified direct access treated as a serious integrity finding.
  5. Failed-login patterns: confirm that spikes were investigated and closed.

How to run the review (step by step)

  1. Set a frequency that matches risk. High-risk production systems that feed release decisions earn a monthly cadence; lower-risk systems can run quarterly. Write the frequency and rationale into the procedure.
  2. Pull the period’s integrity-relevant events from the off-host store, not from the host (reviewing the host’s local copy reintroduces the tampering risk the store exists to remove).
  3. Work the categories above. For each event, reconcile to an authorizing record. The review is fundamentally a reconciliation: every privileged action should map to an approved reason.
  4. Document what was reviewed, the period, the source, the reviewer, the date, the events examined, the exceptions found, and the disposition of each. A review with no record of scope is not evidence that the review happened.
  5. Route exceptions into the quality system as deviations or investigations under change control and the CAPA process. An exception with no follow-through is worse than no review, because it shows you saw the problem and did nothing.

Acceptance criteria for review. The reviewer is demonstrably independent of the administrators being reviewed; the review draws from the tamper-evident off-host store; every privileged action in the period reconciles to an authorizing record or is raised as a deviation; the review is documented with scope, source, reviewer, and disposition of each exception; and exceptions are tracked to closure in the quality system. The frequency is defined, risk-based, and met.

Metrics that show the layer is working

A control you cannot measure is a control you cannot defend. When an inspector or an internal auditor asks how you know the logging layer is doing its job, a handful of numbers, trended over time, is a far stronger answer than “we review the logs.” Track a small set and act on the trend, not the single reading.

MetricWhat it tells youA healthy pattern
Forwarding uptime per hostWhether events are actually leaving the host, or a host has gone darkAt or near 100%; any sustained gap investigated as a blind window
Time to detect a triggered test eventHow fast the alert path really fires, measured with a deliberate test eventWithin the alert’s defined target time, every test
Privileged actions reconciled to an authorizing recordWhether every high-privilege action maps to an approved reasonRising toward 100%; the unreconciled remainder becomes deviations
Alert false-positive rateWhether the alert channel is credible enough that responders still actLow and stable; a rising rate means thresholds need tuning before people tune out
Review completed on scheduleWhether the independent review is actually happening at its defined cadenceEvery review on time; a slip is itself a finding
Exceptions closed within targetWhether findings are tracked to closure, not just noticedClosures keeping pace with openings, no aging backlog

The two that matter most to an inspector are the reconciliation rate, because it shows privileged activity is genuinely accounted for, and the time-to-detect from a real test, because it proves the alerting is not just wired but works. A program that can show both, trended across quarters, is demonstrating control rather than asserting it.


A worked example

A quality control laboratory runs a chromatography data system whose results feed batch release. The application audit trail is fully configured at the application level, Tier 3, prior values captured, review documented. By the application’s own evidence, the system is clean.

The monthly security log review, run by QA from the central log store, surfaces a privileged login to the database server at 23:14 on a Tuesday by the account svc_dbadmin. The application audit trail shows nothing at that time, because nothing happened in the application. The database log, however, shows three things in sequence: a direct connection to the results database, an UPDATE statement against the chromatography results table, and a disconnect, all within four minutes. The OS log shows that ninety seconds before the UPDATE, the application’s audit-trail logging service was stopped, then restarted afterward.

None of this is visible from the application. The application audit trail has no entry for the changed result because the change was made underneath it, with the trail temporarily off. The only reason it surfaced at all is that the system log captured the privileged login, the direct DML, and the logging-service stop, forwarded all three off-host before anything local could be cleaned up, and an independent reviewer read them and asked why a service account edited release data at 23:14.

The investigation finds the change was an analyst, working through a shared service account, “correcting” a result that had failed specification, without a deviation, without a change record, without leaving a trace in the application. The record is invalidated, the batch is held, the shared account is decommissioned in favor of named credentials with the database direct-access right removed, and the gap, that direct DML on the results table was possible at all and that the audit trail could be stopped without an alert, becomes a CAPA. The alert on audit-trail-stopped, which had not existed, is built so the next attempt is caught in minutes rather than at month-end.

That is the entire argument for this layer in one incident. The application said everything was fine. The system log said otherwise, and it was right.

A second worked example: the SaaS variant

A biotech runs its eQMS on a vendor’s multi-tenant SaaS platform. The tenant’s own application audit trail is complete and looks exactly as it should: every document change, every approval, every signature, correctly attributed.

The quarterly vendor review, performed against the SaaS logging checklist signed at onboarding, requests the vendor’s privileged-access log for the review period, the export the contract commits the vendor to provide. The export shows a vendor support engineer accessed the tenant’s underlying database directly for eleven minutes during a support ticket, a type of access the contract permits only for troubleshooting with the customer’s prior authorization logged in the ticket system. The matching ticket exists and references a performance issue, not a data change, but the ticket contains no record of what, if anything, the engineer actually touched during those eleven minutes, because the vendor’s own internal tooling for that layer does not capture statement-level detail, only session start and end.

Nothing in the tenant’s own audit trail shows a problem, because there is nothing to show; if a record had been touched during that session, the tenant-level trail would never see it, the same structural blind spot as the on-premises case above, now sitting on the vendor’s side of the boundary. The reviewer cannot prove harm occurred, but cannot prove it did not, either, and that gap is itself the finding: the contract promised a tamper-evident privileged-access log covering vendor-staff access to tenant data, and what the vendor actually produced stops at session boundaries, not statement-level detail. The disposition is not a deviation against the biotech’s own systems, which performed exactly as controlled; it is a vendor corrective action, tracked in the supplier file, demanding statement-level logging for direct database sessions before the next support ticket of that kind is authorized, and an interim control requiring the biotech’s own designated reviewer to join any future session of that type live rather than review it after the fact.

The lesson sits next to the first worked example rather than apart from it. Whether the privileged access happens on a host you administer or a host the vendor administers, the question an independent reviewer asks is identical: who was in the system at the privilege level that can touch a record invisibly, and is there a record of what they actually did. The only thing that changes in the cloud is where you have to go to get that record, and whether the contract obligates the other side to produce it in enough detail to answer the question.


Roles

RoleResponsibility
IT / system administrationConfigure and maintain event capture and forwarding; do not own the log store or its review
Security / log-store ownerOwn the off-host store, enforce immutability and retention, administer access that the GxP host admins do not hold
Quality assuranceOwn the independent periodic review of administrator activity; raise and track exceptions; approve the logging configuration as a controlled item
System / process ownerDefine which events are integrity-relevant for the system; ensure the logging design is in the data integrity assessment; resource the reviews
Incident responder (named on-call)Act on real-time alerts within the target time; document the response
Validation / CSVQualify the capture, forwarding, retention, and alerting; include log integrity tests in OQ and infrastructure qualification

The non-negotiable structural rule across this table: the party that administers the GxP hosts does not own the log store and does not perform the independent review. If those collapse into one team, the control is cosmetic.


Common 483-type mistakes

  • The system log layer simply does not exist. Plenty of effort on the application audit trail, nothing underneath it. No record of who logged into the OS or database, no detection of direct database edits. The application trail is treated as the only integrity control, leaving every back-channel unmonitored.
  • Logs live only on the host. Security events are captured but stored on the same machine the administrators control, so the highest-privilege users can clear the evidence of their own actions. No off-host copy, no immutability.
  • No detection of audit-trail-disabled. The application audit trail can be stopped, and nothing records or alerts on the stopping. The one event that most undermines the primary integrity control is invisible.
  • IT reviews its own administrators. The periodic review of privileged activity is performed by the same function whose activity is being reviewed, with no independent challenge. The conflict is structural and an inspector will say so.
  • Direct database access is possible and unmonitored. Application users or administrators can edit GxP tables directly, bypassing the application audit trail entirely, and nothing flags it. This is the classic invisible-falsification path.
  • Alert fatigue or no alerting at all. Either no real-time alerts exist for the time-sensitive events, or so many alerts fire that responders ignore all of them, including the real one.
  • No clock-change detection. System clock changes are not logged or not reviewed, so backdating cannot be detected. A backward clock change near a contested result is one of the most serious findings there is, and many programs cannot even see it.
  • Logging not qualified. The logging configuration was never tested. Nobody confirmed that triggering each integrity-relevant event actually produces a log entry, so the program is trusting capture it has never verified.
  • Review with no record of scope. Someone “reviews the logs” but cannot show what period, what source, what events, or what exceptions. An undocumented review is not evidence a review happened.
  • Retention shorter than the records it protects. Security logs roll off after ninety days while the batch records they protect must be kept for years, so when an old record is questioned the proof of who touched it is gone.
  • Tamper-evidence qualified once and never re-checked. The WORM or hash-chaining control was proven to work during installation qualification and never tested again. A storage migration, a policy change, or a lapsed retention setting can quietly remove the protection years later, and nobody would know until the evidence is needed and is not there.
  • A SaaS vendor’s logging promise exists only in the sales conversation. The quality or technical agreement never actually commits the vendor to a tamper-evident privileged-access log, event export, or incident notification timeline, so when a question arises about vendor-side access to tenant data there is no contractual basis to demand the answer, let alone a log to produce it from.

Interview Q&A angle

Q: What is the difference between an application audit trail and a system security log, and why do you need both? The application audit trail records what users do inside the application: who changed which field, from what value to what value, when. The system security log records what happens at the operating system, database, and infrastructure layer: who logged into the host, who accessed the database directly, whether the audit trail itself was stopped, whether the clock moved. You need both because the events most useful to someone falsifying a record, direct database edits, disabling the audit trail, backdating the clock, are invisible to the application and visible only to the system log. Relying on the application trail alone leaves the back channel unmonitored.

Q: An administrator on a GxP server can edit the local security log. How do you make the log trustworthy? You get it off the host in near real time into a store the host administrators cannot reach with their host credentials, owned by a separate function such as security, with write-once or immutable storage and retention that meets or exceeds the records it protects. Local-only logs are a control administered by the controlled party, which is no control. The off-host, tamper-evident store is what makes the highest-privilege actions provable.

Q: Which security events would you alert on in real time, and how do you avoid alert fatigue? Real-time alerts for the time-sensitive, high-signal events: audit trail disabled or paused on production, a backward clock change outside a maintenance window, logging stopped or forwarding gone silent, a failed-login spike against a privileged account, a new member of an administrator group, and a cleared security log. Everything else goes to the periodic review. Avoid fatigue by alerting only on those, setting thresholds from observed baseline behavior, and documenting them, so the channel stays quiet enough that responders still act on it.

Q: Who should review administrator activity, and why does it matter who does it? Quality assurance, or a security function reporting outside the line being reviewed, must do it, never the administrators whose activity is under review. It matters because IT reviewing its own privileged actions is a structural conflict, the same reason record creators should not be the sole reviewers of the audit trail of those records. The reviewer reconciles every privileged action to an authorizing record and raises anything unexplained as a deviation tracked to closure.

Q: How do you prove your security logging actually works? You qualify it. During OQ or a focused configuration verification you deliberately trigger each integrity-relevant event in a controlled environment, a test login, a test direct-database access, a test audit-trail stop, and confirm each appears in the log and arrives intact at the off-host store with the right who, what, and when. Capture you have never tested is capture you cannot rely on, and “we assume it logs” is not an answer an inspector accepts.

Q: Your system runs on a vendor’s SaaS platform and you cannot touch the host. How do you handle security logging? The host-level log moves behind the vendor’s operational boundary, so the control moves from something you configure to something you contract for and monitor. Before signing, you require a tamper-evident log of privileged and vendor-staff access to your tenant’s data, an export or streaming feed so your own review draws from a copy you hold, retention matching your records, and timely incident notification. After go-live, you reconcile the identity and tenant-configuration logs you do control, ingest the vendor’s audit feed, and alert on that feed going silent the same way you would alert on a host that stopped forwarding. Where the vendor holds a log you genuinely cannot get, the honest answer in the data integrity assessment names that gap as a contractual and monitored control rather than pretending the tenant trail sees the whole stack.


Security event logging is the layer that catches what the application cannot see. Build the integrity-relevant event set, get the logs off the host into a tamper-evident store the administrators do not control, alert on the handful of events that cannot wait, and have an independent function reconcile privileged activity to authorizing records on a risk-based cadence. Do that and the back wall is closed, not just the front door. For the application-side counterpart, see audit trail design and review and operationalizing audit trail review; for the access controls this monitoring assumes, see CSV cybersecurity and access control.

Use madhadi.com as an app Full screen, works offline, one tap from your home screen.