Data and Knowledge Systems · Principal
The event schema is backward compatible. Why did old requests become approved?
The question
Interview question
A team adds `approved: boolean` to a request event with a reader default of `true`. Schema Registry accepts the evolution. An AI report then says thousands of old requests were approved, even though nobody made that decision when those events were written. Is the registry wrong?
Take a few minutes to form your approach. Then open a worked answer and compare the decisions.
Reveal a worked answer
It checked whether the reader can decode the old data. That is a narrower contract than whether the decoded data tells the truth. Under Avro schema resolution, a reader uses its field default when the writer's record has no matching field. The old events did not contain approved. The new reader supplied true. This differs from a newer writer explicitly encoding false or null. Confluent's schema evolution guide explains the compatibility mechanism, but the business meaning of a default belongs to the product.
I would inspect raw events with their writer schema IDs and the exact reader version the report used. Count how many approvals came from explicit values versus a default substituted while decoding. Compare against the authority that actually records an approval decision. A field named approved is especially risky because an old request with no decision is not equivalent to an approved request. Nor is it automatically equivalent to a rejection. It might be unknown or pending.
The repair is a data contract with an explicit state model and provenance, not merely a different Boolean default. One option is a separate approval decision event keyed to request ID, with actor, time and policy version. Another is an enum with UNKNOWN, PENDING, APPROVED and REJECTED, if the lifecycle really fits those states. The old events should map to UNKNOWN unless a separate authoritative decision can be joined. Run a bounded backfill from that source, version the report and disclose any restatement. Downstream authorization must never infer permission from a compatibility default. Gate it on an explicit current approval record.
What if someone proposes default false because it is safe? It may be a safe execution fallback when a decision is missing. It is not a truthful historical label saying all old requests were rejected. Keep operational fail-closed behavior separate from reporting semantics. Test each reader version against old records, explicit false, explicit true and explicit null. The CDC field is still called amount. When did its units change? asks what happens when a field called amount silently changes units. This page is about a newly added field whose valid decoding invents a business decision.
Continue reading
Related questions
Read beyond the question
Explore more data and knowledge systems
Follow another question in this area, or search the complete Question Library.
Browse this area →Browse Question Library →