Data and Knowledge Systems · Principal
Three data sources are current and one is a day behind. Is the AI report current?
The question
Interview question
An AI assistant writes a daily enterprise risk report using a payment ledger, support tickets, a compliance table, and a customer profile index. At 9 AM the first three sources are current through 8:55. The profile index last incorporated changes yesterday at 4 PM. The report says “as of 9 AM” and recommends suspending two accounts. Design the publication contract. The profile source says its ingestion timestamp is 8:58, even though its last successful source watermark is yesterday.
Take a few minutes to form your approach. Then open a worked answer and compare the decisions.
Reveal a worked answer
“As of 9 AM” is a claim about the evidence set, not the time the model started typing. The index might have accepted a heartbeat at 8:58 without applying yesterday's account changes. We need to know, for each source, the latest source event or snapshot represented in the data, the time it became queryable, and whether the ingest is complete through that boundary. These are not interchangeable timestamps.
I would give the report an evidence manifest. It names the payment ledger snapshot or transaction boundary, ticket stream offset and completeness status, compliance table revision, profile index source watermark and index generation, plus the query and report version. A single global timestamp is often impossible across independent systems. Instead specify a required freshness rule per decision. For a risk suspension, current compliance status and account ownership may be mandatory, while yesterday's support sentiment might be acceptable as supporting context. The report renderer should show the slow source and the exact claim it affects, not hide a vector of watermarks behind one green dot.
If the profile index is required to identify the account owner or active exemption, hold the suspension recommendation. The assistant can still produce a partial report with unaffected sections, labeled with their own cutoffs, or ask the authoritative profile service directly if its read contract permits. If the profile is only decorative context, publish the other findings and omit or label that section. The decision depends on what the stale data could change. A one-day lag is not automatically harmless or automatically fatal.
The team suggests taking the minimum timestamp and saying the entire report is current through yesterday at 4 PM. That may be honest about a bound, but it can also be misleading if one source has a newer ledger event that the report used while another source is old. There may be no actual system-wide snapshot at 4 PM, particularly if ticket deletions and ledger corrections were applied later. Better to pin each source version for reproducibility and state the scope of the claims. If a coherent cross-source cutoff is legally required, build an explicit publication barrier and only materialize a report once all required streams have crossed it. Data that arrives late by event time can still revise the past, so define how correction and republication work.
A useful probe is a profile update at 8 AM that changes one account's exemption. The index's ingestion timestamp at 8:58 does not prove that update was included. Follow the source event ID through the connector and index, or read the authoritative record. If it is missing, do not recommend suspending that account. The source corrected yesterday's data. Should the assistant rewrite yesterday's answer? covers correcting a report after source data changes. Here the decision is whether to publish a report whose sources were never aligned enough to justify the claim in the first place.
I would test delayed and out-of-order events, a heartbeat without progress, a source that is down but returns cached results, an index generation with partial partitions, and a report export that starts before all checks finish. Measure freshness by source and by consequential claim, plus the time to a publishable report. An AI report is useful when it can say what it knows and what it does not. A polished 9 AM header cannot create a 9 AM data snapshot.
Continue reading
Related questions
Read beyond the question
Explore more data and knowledge systems
Follow another question in this area, or return to the full Interview Prep index.
Browse this area →