Data & AI
Data Engineer Comprehensive guide covering SQL & Database Fundamentals, Python for DE, ETL/ELT Pipelines, Data Warehousing, Big Data Technologies (Spark, Kafka, Airflow), Cloud Platforms, and System Design.
300 questions Updated 2026-02-03 Beginner Intermediate Advanced
What you will be asked about SQL & Database Fundamentals Python for Data Engineering ETL/ELT Processes Data Warehousing Big Data Technologies Cloud Platforms NoSQL Databases Data Quality & Testing DevOps & CI/CD Data Modeling & Architecture Performance Optimization Real-time & Streaming Scenario Behavioral
How to prepare Go through the topic list above and mark every one you cannot explain for five minutes unprepared. Those are your gaps. Pair every concept with a story from your own work — interviewers probe depth, and depth comes from having actually done it. Do the DSA rounds anyway. Almost every role in this list still screens with coding. Prepare two projects you can whiteboard end to end, including what you would change now. Data Engineer interview questions300 1. What is the difference between WHERE and HAVING clauses? Beginner 2. Explain INNER JOIN, LEFT JOIN, RIGHT JOIN, FULL OUTER JOIN, and CROSS JOIN. Beginner 3. What are window functions and provide use cases? Intermediate 4. How do you optimize a slow-running SQL query? Advanced 5. What is the difference between DELETE, TRUNCATE, and DROP? Beginner 6. Explain clustered vs non-clustered indexes. Intermediate 7. What is query execution plan and how do you read it? Advanced 8. What are CTEs and how do they differ from subqueries? Intermediate 9. Explain partitioning in databases. Intermediate 10. What is database sharding? Advanced 11. What is the difference between OLTP and OLAP? Beginner 12. Explain ACID properties in databases. Intermediate 13. What is a materialized view? Intermediate 14. How do you handle database deadlocks? Advanced 15. What is database replication and what types exist? Intermediate 16. Explain the CAP theorem. Advanced 17. What is the difference between normalization and denormalization? Beginner 18. What are the different normal forms (1NF, 2NF, 3NF, BCNF)? Advanced 19. How do you design a database schema? Intermediate 20. What is a star schema vs snowflake schema? Intermediate 21. Explain fact tables and dimension tables. Beginner 22. What is slowly changing dimension (SCD) Type 1, 2, and 3? Intermediate 23. How do you handle NULL values in SQL? Beginner 24. What is the difference between UNION and UNION ALL? Beginner 25. Explain database indexing strategies. Intermediate 26. What are composite keys and surrogate keys? Intermediate 27. How do you perform incremental data loads? Intermediate 28. What is a transaction and transaction isolation levels? Advanced 29. Explain optimistic vs pessimistic locking. Advanced 30. What is database connection pooling? Intermediate 31. What Python libraries do you use for data engineering? Beginner 32. Explain the Global Interpreter Lock (GIL). Advanced 33. What is the difference between multiprocessing and multithreading? Intermediate 34. How do you handle large files in Python? Intermediate 35. What are generators and why are they useful? Intermediate 36. Explain decorators in Python. Advanced 37. What is the difference between list comprehension and generator expression? Beginner 38. How do you implement error handling in Python? Beginner 39. What is context manager and the 'with' statement? Beginner 40. Explain *args and **kwargs. Intermediate 41. How do you optimize Python code for performance? Advanced 42. What is the difference between shallow copy and deep copy? Intermediate 43. Explain Python's garbage collection. Advanced 44. What are async/await in Python? Advanced 45. How do you work with APIs in Python? Beginner 46. What is the difference between @staticmethod and @classmethod? Intermediate 47. How do you implement logging in Python? Beginner 48. What is pickle in Python? Intermediate 49. Explain lambda functions and map/filter/reduce. Beginner 50. How do you handle memory management in Python? Advanced 51. What are Python dataclasses? Intermediate 52. How do you implement unit testing in Python? Intermediate 53. What is virtual environment and why use it? Beginner 54. Explain Python's type hints. Intermediate 55. How do you profile Python code for bottlenecks? Advanced 56. What is ETL and ELT? What's the difference? Beginner 57. Explain the ETL process you follow. Beginner 58. What ETL tools have you worked with? Beginner 59. How do you handle incremental loads vs full loads? Intermediate 60. What is data lineage? Intermediate 61. How do you implement error handling in ETL pipelines? Intermediate 62. What is idempotency and why is it important in data pipelines? Advanced 63. How do you handle slowly changing dimensions (SCD)? Intermediate 64. What is CDC (Change Data Capture)? Advanced 65. Explain different CDC methods. Advanced 66. How do you ensure data quality in ETL processes? Intermediate 67. What is data validation and where do you implement it? Beginner 68. How do you handle duplicate records in ETL? Beginner 69. What is data transformation and give examples? Beginner 70. How do you optimize ETL performance? Advanced 71. What is parallel processing in ETL? Intermediate 72. How do you handle schema changes in source systems? Advanced 73. What is data reconciliation? Intermediate 74. How do you implement retry logic in data pipelines? Intermediate 75. What is backfilling in data pipelines? Intermediate 76. How do you monitor ETL jobs? Beginner 77. What is SLA in data pipelines? Beginner 78. How do you handle timezone conversions in ETL? Intermediate 79. What is metadata management in ETL? Intermediate 80. How do you version control your ETL code? Beginner 81. What is a data warehouse? Beginner 82. Explain the difference between data warehouse, data lake, and data mart. Beginner 83. What is dimensional modeling? Intermediate 84. What is Kimball vs Inmon methodology? Advanced 85. Explain fact and dimension tables in detail. Beginner 86. What are factless fact tables? Advanced 87. What is a conformed dimension? Advanced 88. How do you handle late-arriving facts? Advanced 89. What is a bridge table? Advanced 90. Explain grain in dimensional modeling. Intermediate 91. What is an aggregate table and when to use it? Intermediate 92. How do you implement SCD Type 2? Intermediate 93. What is a junk dimension? Advanced 94. What is a role-playing dimension? Intermediate 95. How do you optimize data warehouse queries? Advanced 96. What is partition pruning? Intermediate 97. Explain columnar storage vs row storage. Intermediate 98. What is data vault modeling? Advanced 99. How do you handle historical data in warehouse? Intermediate 100. What is the medallion architecture (Bronze, Silver, Gold)? Beginner 101. What is Apache Spark and its architecture? Intermediate 102. Explain RDD, DataFrame, and Dataset in Spark. Intermediate 103. What are Spark transformations vs actions? Beginner 104. What is lazy evaluation in Spark? Intermediate 105. How do you optimize Spark jobs? Advanced 106. What is data skewness and how do you handle it? Advanced 107. Explain Spark partitioning and bucketing. Advanced 108. What is Apache Hadoop and HDFS? Beginner 109. Explain MapReduce paradigm. Intermediate 110. What is YARN in Hadoop ecosystem? Intermediate 111. What is Apache Hive and HiveQL? Beginner 112. What is the difference between Hive and traditional databases? Intermediate 113. What is Apache Kafka and its use cases? Beginner 114. Explain Kafka producers, consumers, and brokers. Beginner 115. What are Kafka topics and partitions? Intermediate 116. How does Kafka ensure message delivery? Advanced 117. What is Apache Airflow? Beginner 118. How do you design DAGs in Airflow? Intermediate 119. What are Airflow operators, sensors, and hooks? Intermediate 120. What is Apache Flink? Advanced 121. What is the difference between batch and stream processing? Beginner 122. What is Apache Beam? Advanced 123. Explain data lakehouse architecture. Intermediate 124. What is Delta Lake/Apache Iceberg/Apache Hudi? Advanced 125. How do you implement ACID transactions in data lakes? Advanced 126. What is Parquet file format and its advantages? Intermediate 127. What is ORC file format? Intermediate 128. What is Avro and when to use it? Intermediate 129. How do you choose file formats for different scenarios? Advanced 130. What is data partitioning strategy in big data? Intermediate 131. What cloud platforms have you worked with (AWS/Azure/GCP)? Beginner 132. What is AWS S3 and its storage classes? Beginner 133. What is AWS Redshift? Intermediate 134. Explain AWS Glue and its components. Intermediate 135. What is AWS Lambda and serverless architecture? Beginner 136. What is AWS EMR? Intermediate 137. What is AWS Kinesis? Intermediate 138. What is Azure Data Factory? Intermediate 139. What is Azure Databricks? Intermediate 140. What is Azure Synapse Analytics? Advanced 141. What is Azure Data Lake Storage? Intermediate 142. What is Google BigQuery? Intermediate 143. What is Google Dataflow? Advanced 144. What is Google Cloud Storage? Beginner 145. What is Google Pub/Sub? Intermediate 146. How do you implement data security in cloud? Advanced 147. What is IAM and how do you manage permissions? Intermediate 148. What is VPC and network security? Advanced 149. How do you optimize cloud costs? Intermediate 150. What is cloud data migration strategy? Advanced 151. What are managed vs self-hosted services? Beginner 152. How do you implement disaster recovery in cloud? Advanced 153. What is multi-cloud vs hybrid cloud architecture? Intermediate 154. How do you monitor cloud resources? Beginner 155. What is Infrastructure as Code (IaC)? Intermediate 156. What is the difference between SQL and NoSQL databases? Beginner 157. When would you choose NoSQL over SQL? Intermediate 158. What are different types of NoSQL databases? Beginner 159. What is MongoDB and document-based databases? Beginner 160. What is Cassandra and column-family stores? Intermediate 161. What is Redis and key-value stores? Beginner 162. What is Neo4j and graph databases? Intermediate 163. Explain eventual consistency. Advanced 164. What is BASE properties in NoSQL? Advanced 165. How do you model data in NoSQL? Intermediate 166. What is denormalization in NoSQL? Intermediate 167. How do you handle transactions in NoSQL? Advanced 168. What is sharding in NoSQL databases? Intermediate 169. How do you choose between different NoSQL databases? Advanced 170. What are the trade-offs of using NoSQL? Intermediate 171. How do you ensure data quality? Beginner 172. What are data quality dimensions? Intermediate 173. How do you implement data validation? Beginner 174. What is data profiling? Intermediate 175. How do you test data pipelines? Intermediate 176. What is unit testing vs integration testing in DE? Intermediate 177. How do you implement data reconciliation? Advanced 178. What are data quality metrics you track? Intermediate 179. How do you handle data quality issues? Intermediate 180. What is schema validation? Beginner 181. How do you implement data monitoring? Intermediate 182. What is data observability? Advanced 183. How do you set up alerts for data issues? Beginner 184. What is regression testing for data pipelines? Intermediate 185. How do you implement data quality frameworks? Advanced 186. What is CI/CD for data pipelines? Beginner 187. What version control systems have you used? Beginner 188. How do you implement Git workflow for data projects? Intermediate 189. What is Docker and containerization? Beginner 190. What is Kubernetes and container orchestration? Advanced 191. How do you implement automated testing for pipelines? Intermediate 192. What is infrastructure as code (Terraform, CloudFormation)? Advanced 193. How do you manage environment configurations? Intermediate 194. What is blue-green deployment? Advanced 195. How do you implement logging and monitoring? Intermediate 196. What is Jenkins/GitHub Actions/GitLab CI? Beginner 197. How do you handle secrets management? Advanced 198. What is deployment strategy for data pipelines? Intermediate 199. How do you implement rollback mechanisms? Advanced 200. What is observability in data engineering? Intermediate 201. How do you design a data model? Intermediate 202. What is data modeling best practices? Beginner 203. What is entity-relationship diagram (ERD)? Beginner 204. How do you handle many-to-many relationships? Intermediate 205. What is data modeling for analytical workloads? Intermediate 206. How do you design for scalability? Advanced 207. What is Lambda architecture? Advanced 208. What is Kappa architecture? Advanced 209. How do you design real-time data pipelines? Intermediate 210. What is microservices architecture for data? Advanced 211. How do you implement data governance? Advanced 212. What is master data management? Advanced 213. How do you handle data privacy (GDPR, CCPA)? Advanced 214. What is data catalog and metadata management? Intermediate 215. How do you design for high availability? Advanced 216. How do you optimize database queries? Beginner 217. What is query optimization techniques? Intermediate 218. How do you handle large-scale data processing? Intermediate 219. What is data compression and when to use it? Intermediate 220. How do you optimize Spark jobs? Advanced 221. What is partition pruning and predicate pushdown? Advanced 222. How do you optimize data pipeline performance? Intermediate 223. What is caching strategy in data engineering? Intermediate 224. How do you handle memory optimization? Advanced 225. What is broadcast join in Spark? Advanced 226. How do you optimize storage costs? Intermediate 227. What is data tiering strategy? Advanced 228. How do you implement parallel processing? Intermediate 229. What is batch size optimization? Intermediate 230. How do you profile and debug performance issues? Advanced 231. What is stream processing? Beginner 232. What is the difference between batch and streaming? Beginner 233. How does Kafka streaming work? Intermediate 234. What is event-driven architecture? Intermediate 235. How do you handle late-arriving data? Advanced 236. What is windowing in stream processing? Intermediate 237. What is exactly-once processing? Advanced 238. How do you implement real-time analytics? Advanced 239. What is stateful vs stateless processing? Advanced 240. How do you handle backpressure? Advanced 241. What is stream-table join? Advanced 242. How do you implement event sourcing? Advanced 243. What is CQRS pattern? Advanced 244. How do you handle streaming data quality? Intermediate 245. What is real-time data pipeline architecture? Intermediate 246. Design a data pipeline for processing millions of events per day. Advanced 247. How would you migrate from on-premise to cloud? Advanced 248. Design a real-time recommendation system data pipeline. Advanced 249. How would you handle a failed ETL job in production? Intermediate 250. Design a data warehouse from scratch. Intermediate 251. How would you optimize a slow-running Spark job? Advanced 252. Design a solution for handling duplicate data. Intermediate 253. How would you implement data lake architecture? Intermediate 254. Design a CDC pipeline from MySQL to data warehouse. Advanced 255. How would you handle schema evolution in data pipeline? Advanced 256. Design a solution for real-time fraud detection. Advanced 257. How would you implement data retention policy? Intermediate 258. Design a multi-region data replication strategy. Advanced 259. How would you handle data quality issues in production? Intermediate 260. Design a solution for processing streaming and batch data together. Advanced 261. How would you implement disaster recovery? Intermediate 262. Design a solution for handling PII data. Advanced 263. How would you optimize cloud costs for data infrastructure? Intermediate 264. Design a data pipeline for IoT sensor data. Advanced 265. How would you implement data versioning? Intermediate 266. Design a solution for A/B testing analytics. Intermediate 267. How would you handle timezone issues in global data? Intermediate 268. Design a data pipeline monitoring system. Intermediate 269. How would you implement incremental processing? Intermediate 270. Design a solution for handling late-arriving dimensions. Advanced 271. How would you build a customer 360 view? Advanced 272. Design a clickstream data processing pipeline. Intermediate 273. How would you handle data pipeline failures gracefully? Intermediate 274. Design a solution for cross-region data compliance. Advanced 275. How would you implement data lineage tracking? Intermediate 276. Tell me about the most complex data pipeline you built. Advanced 277. How do you handle production incidents? Intermediate 278. Describe a time when you optimized a data pipeline. Intermediate 279. How do you stay updated with data engineering trends? Beginner 280. How do you handle conflicting requirements from stakeholders? Intermediate 281. Describe your experience with agile/scrum methodology. Beginner 282. How do you prioritize multiple projects? Beginner 283. Tell me about a time you made a critical mistake. Intermediate 284. How do you document your data pipelines? Beginner 285. How do you mentor junior engineers? Intermediate 286. Describe your code review process. Beginner 287. How do you handle technical debt? Intermediate 288. Tell me about a time you disagreed with a technical decision. Intermediate 289. How do you ensure data pipeline reliability? Intermediate 290. Describe your testing strategy. Intermediate 291. How do you handle on-call responsibilities? Beginner 292. What's your approach to learning new technologies? Beginner 293. How do you communicate technical concepts to non-technical stakeholders? Beginner 294. Describe a time when you improved data quality. Intermediate 295. How do you handle tight deadlines? Beginner 296. What's your experience with cross-functional collaboration? Intermediate 297. How do you balance speed vs quality? Intermediate 298. Describe your problem-solving approach. Beginner 299. How do you handle ambiguous requirements? Intermediate 300. Why do you want to be a Data Engineer? Beginner