What an auditor, or Sekit's evidence engine, asks for.
AI system monitoring and event logs
How AI systems are watched once in use, and what gets recorded: monitoring of performance, drift, misuse, harmful outputs and control failures against set thresholds and defined responses, and what the event log captures about inputs, outputs, decisions and human overrides.
From the Sekit evidence catalog
In practice
An AI system does not fail the way normal software fails: it can keep running while quietly drifting off target, producing worse answers, or being misused for something it wasn't approved for. The AI system monitoring and event logs record performance, drift, misuse and control failures against thresholds, plus samples of inputs, outputs and human overrides. Auditors ask what the threshold is for "this AI has drifted" and who gets alerted, and check whether monitoring exists for the vendor chatbot the sales team uses, not only for the flagship internally built model.
Common gaps
Monitoring exists for internally built AI systems but not for the vendor chatbot the customer team uses daily.
No threshold is defined for what counts as drift or degraded performance, so nobody would notice a slow decline.
Human override events are not logged, so there is no record of when staff had to correct or reject an AI output.
Questions your auditor will ask
What gets monitored for an AI system once it's live?
Performance, drift, misuse and harmful outputs against defined thresholds, recorded in the AI system monitoring and event logs.
Does monitoring cover vendor AI tools, or only systems you built?
Both, since a purchased chatbot or copilot carries the same drift and misuse risk as an in-house model.
What happens when a monitored threshold is crossed?
A defined response is triggered, escalated the same way a security alert is triaged and assigned to an owner.
Do you keep a record of human overrides of AI decisions?
Yes, sample logs capture when staff corrected or rejected an AI output, alongside the input and output that prompted it.
Where regulation demands it
NIS2 art. 3.2 (Monitoring and logging) requires this, which for an AI system extends to drift, misuse and human override, not only infrastructure events.