Question library
Questions from real interviews. Drill any of them by voice in a couple of minutes, with an interviewer, a score and feedback.
Tell me what Apache Spark is and how it processes data.JuniorA Spark job that joins two large tables runs for hours, and the Spark UI shows that almost all tasks finished within a minute, while a few hang for a very long time. What could be the cause, and how do you fix it?Mid-levelYour Spark job writes data to a data lake as Parquet, and over time queries against that table keep getting slower, while the folder now has hundreds of thousands of small files. Why did this happen, and what will you do?Mid-levelA Spark job is processing a large join: almost all tasks in the stage finish in a minute, but the last two hang for an hour and sometimes fail with OOM. What's going on, and how would you fix it?SeniorWalk me through how Spark executes a query from the moment an action is called to the result: what are stages, shuffles and lazy evaluation, and where is performance usually lost?Senior