Walk me through how Spark executes a query from the moment an action is called to the result: what are stages, shuffles and lazy evaluation, and where is performance usually lost?
~4 min by voice
How to approach it
The interviewer checks whether the candidate understands Spark's actual execution model: lazy plan building, Catalyst optimization, splitting the DAG into stages at shuffles, the link between tasks and partitions. Plus whether they can use that model to explain where slowdowns come from, rather than just "add more memory".
The interviewer will ask this question and listen. Think out loud and ask for a hint if you need one. When you're done, pause or tap “That's my answer”. In about a minute, your score and the points you covered show up here.