Data and Knowledge Systems · Principal
The source corrected yesterday's data. Should the assistant rewrite yesterday's answer?
The question
Interview question
An assistant answered that 1,240 customers breached a quota yesterday. This morning the usage team backfills missing events and the correct count for that day becomes 1,410. The answer was shown to 300 people, copied into a report, and used to approve a credit batch. Design the correction path. The report has already been downloaded, and a few readers no longer have permission to see the underlying customer data.
Take a few minutes to form your approach. Then open a worked answer and compare the decisions.
Reveal a worked answer
I would not overwrite the old answer as if it had always said 1,410. That destroys the evidence of what people actually saw and what the credit decision relied on. I also would not leave 1,240 as a current answer. Preserve the original response with its query or metric version, source snapshot or watermark, answer time, audience and any downstream action ID. Mark it superseded by a correction that explains the changed input and the new count. On the live page or conversation, show the correction next to the old answer so a reader does not carry on with a quiet false premise.
First check that 1,410 is a valid replacement. Was the backfill complete and deduplicated? Did the quota rule or account population also change? Compare the same metric definition over two data revisions, with an explicit event-time window and ingestion watermark. If only 170 late events arrived, that does not necessarily mean 170 new customers breached the quota. Several events can belong to one account, and a breach may depend on usage aggregated over a quota window. The answer's original 1,240 and revised 1,410 are hypothetical counts in this scenario, not numbers inferred from the event delta. If the source is still settling, label the revised number provisional and avoid a new final decision until the owner states the data cutoff.
The correction system needs dependency edges. A source revision or data partition feeds an aggregate run, that run feeds a claim in an answer, and the answer may feed a report or credit decision. OpenLineage's specification describes dataset and field lineage, while W3C PROV-O gives terms for derivation and revision. Neither automatically records which people read a generated answer or what they decided because of it. Capture answer IDs, exposure records where product policy permits, report export IDs, and action references separately. A missing edge means we may miss an affected consumer, so reconcile by query window and publication channel as well as by recorded lineage.
Who do we notify? At least the owner of the credit batch and report, and users who were directly shown the materially wrong count if the product has a notification channel and permission to use it. Severity is about the consequence of the difference, not merely its percentage. The credit batch should be paused if not committed, then recalculated against the corrected snapshot and approved again under its business rule. If already committed, treat it as a separate remediation decision. Do not assume that the credit difference is 170 units or that it can be reversed safely. The original batch operation IDs and provider results still matter.
The downloaded report cannot be edited on someone's laptop. Version the report, mark the original export as superseded in the portal, publish a corrected export, and notify known recipients. A forwarded file outside the product may be unreachable. Say that plainly in the incident record. If the report was used in an external board decision, the business owner decides the correction communication. The system provides a traceable affected-artifact list and the exact revisions, rather than inventing that communication policy.
Permission changes constrain the notification. A former reader can be told that an earlier answer or report was corrected without receiving new underlying customer rows they may no longer access. The corrected aggregate itself may also be restricted, so the notification route needs current authorization. Historical audit can retain what the reader saw under the documented retention policy, but a citation to a historical snapshot is not a current access grant.
The most important question for the design review is what happens next time the same query runs. It should use the corrected data revision and display freshness. The old answer remains in history with a visible correction relation. Test a late backfill, a backfill that is itself retracted, a missing dependency edge, a report export, and a decision already committed. A source correction is not finished when the index says “updated.” It is finished when affected claims and consequential decisions have an owner and a truthful status.
Continue reading
Related questions
Read beyond the question
Explore more data and knowledge systems
Follow another question in this area, or return to the full Interview Prep index.
Browse this area →