Design a system of daily batch pipelines for several hundred tables with dependencies between them. How would you handle reruns, backfills and late-arriving data?
~10 min by voice
How to approach it
The interviewer checks whether the candidate can build a reliable batch platform where reruns, backfills and catching up on late data are standard, safe operations rather than manual cleanup. The key is idempotency through overwriting partitions and parameterizing by the logical date.
The interviewer will ask this question and listen. Think out loud and ask for a hint if you need one. When you're done, pause or tap “That's my answer”. In about a minute, your score and the points you covered show up here.