Design the audit logging for a system handling financial transactions. What is logged, where does it go, and what makes it hold up?

What is logged

Significant actions only, defined explicitly rather than logging everything — an audit trail nobody can search is not usable evidence:

Authentication events including failures. Authorisation denials, which are the signal of probing. Permission and role changes. Every data mutation with before and after values. Reads of sensitive records, which is the one usually omitted and the one a breach investigation needs most. Data exports. Administrative and configuration actions. And every use of a break-glass or elevated credential.

Each entry: who (a real principal, not a service account), what, which resource, when (with timezone), from where, the outcome, and a correlation ID tying it to the wider request.

Where it goes

A separate account or system whose credentials are not available to the audited environment. This is the highest-value structural decision, because compromising the application then does not grant access to its audit trail — and removing traces is the first action of a competent intruder.

Append-only, with object lock in compliance mode for the retention period. Governance mode can be overridden, which defeats the control against the exact threat.

Hash-chained so alteration of any entry is detectable.

What makes it hold up

Attribution through service hops. Identity must be propagated, or everything downstream is attributed to the gateway. This includes asynchronous processing — the originating principal must travel into the queue, or the trail breaks at the first async boundary.

Retention exceeding plausible dwell time, or the initial access is gone before the investigation starts. Financial regulation will usually specify a longer minimum anyway.

Completeness monitoring. A log that stops recording during an incident is as useless as one that is edited — so absence of entries must itself alert.

Separation from application logs. Different requirements, different retention, different access control. Mixing them means the audit trail inherits the application log's rotation and its readership.

What must not be in it

Card numbers, credentials, tokens, and personal data beyond what the audit purpose requires. An audit log is widely readable, retained for years and exported to tooling — which makes it a common and poorly-defended location for regulated data to accumulate.

What a strong answer adds

Noting the tension with erasure: an immutable audit log conflicts with a deletion request. The usual resolution is that audit records are retained under a legal obligation, which is an exemption — but that position should be established with legal counsel and documented, not assumed by the engineering team.