How To Prepare For A Data Engineering Interview

[@venchia3]

Very on point to focus on results rather than solely on tech achievements when showcasing a project. Not only at amazon but generally any big company wants someone that can not only lead technically but also earn trust from stakeholders and be business aware (e.g. know the success metrics and how to reach and improve on them). Well done sir! Not many views you are having for now but be consistent in posting this kind of content and surely the channel will grow in the near future :)

[@SeattleDataGuy]

Thank you for the comment! Yes, amazon is not the only company that does it, but it was my first experience for sure. Yeah there aren't too many views. I will probably make a new version of this video with my actual face since that seems to be doing better.

[@vedanthasm2659]

Very good info!. Request you to do one Data Warehouse - Data modelling examples in detail

[@anushavenugopal5371]

Can you please go in more detail with respect to Datawarehouse design models for real life examples like you mentioned for Costco

[@SeattleDataGuy]

Added to my video list! I will work on this

[@darkTranquillity1]

One thing I've been trying to find out is, what is the upper bound in coding challenge complexity for a DE vs a SWE? I get asked anything between flatten a binary tree to a Linked List or something simple like find the first non repeating character in a string. Also, some DE technical screens don't include any SQL. It's all so arbitrary.

[@SeattleDataGuy]

Thank you for the comment! Data engineering interviews really are quite broad. Like you brought up, I have had to answer questions all across the board. Just like you brought up. It's funny you say that some DE interviews don't even ask SQL because I have been on the flipside where I had no programming questions and only had SQL and design questions. At the end of the day hopefully your recruiter gives some information on what you will have to answer.

[@Stoney-g1o]

@@SeattleDataGuy Ask for the names of the people who will interview you from the recruiter and then look at their LinkedIn profile. Did they study computer science, physics, mathematics or business. What is their work experience. Search for things that they have written or comments to find strong opinions or preferences. Some recruiters are more helpful than others. If possible, avoid the recruiter who does not respond to your questions and says just send me your resume. They are more likely to be playing the numbers game without adding value.

[@SeattleDataGuy]

@@Stoney-g1o I love the idea of asking recruiters for who will be interviewing you! If you are interviewing at a FAANG, generally they will send you a list of who.

[@thecollector6746]

1. There is no upper bound. It's entirely up to the interviewer. 2. If some one is demanding that you flatten a binary tree in a DE roll, ask them what would be the point as no one in any DE role is flattening binary trees....most SE are not dealing directly with data structures like this as no one has need to do so thanks to the tools provided by any modern programming language STL. If they can't answer the question with a straight forward answer that makes sense...then do you really want to work with these people ? 3. What's included in ALL technical interviews is completely arbitrary. It's entirely up to you to determine if investing your time in memorizing the solutions to "elite code challenges" that they may or may not ask you to solve.... or immersing yourself in actual DE where you will actually get hands on experience with DE ?

[@murugesanrm]

Thanks! Keep up the great work and looking forward to learn more from your channel!

[@SeattleDataGuy]

Thank you! Will do.

[@aznfoever35]

DW design video using the Costco example please! Thanks for this!

[@SeattleDataGuy]

I have it on the list. I should really share my content plan one day. So everyone knows what is coming

[@phasematerialsresearch9319]

Would all of this be required for a Junior Data Engineering interview as well? A video on the junior position interview would be great.

[@SeattleDataGuy]

It can really depend. Sometimes yes sometimes no. Usually they might have a similar interview process for a jr and mid level engineer but the bar might be different.

[@ayushipatra9976]

Hi Benjamin, Thank you so much for the checklist!! It really helps!! Do you have any link or resources to get hands on experience on Data Warehousing? I am looking to get an intuitive thought process to create a data model for any particular business requirements. I am not able to query from the model mostly because I have only worked on OLTP side of it.

[@tarunleekha7375]

Hey! Bro . Great video . It would be great if you can show end to end use case for like 30-40 minutes for any specific use case .

[@SeattleDataGuy]

I will have to work on that!

[@hantt]

Thank you for this!

[@SeattleDataGuy]

You're welcome!

[@niteshkamboj1981]

Awesome! Super informative and helpful.

[@SeattleDataGuy]

I am glad you found it helpful! What other questions do you have around data?

[@MrKaNuke]

Hey Ben did you ever do the example with Costco or DoorDash like you mentioned?

[@farPositive]

How ETL testing (QA) is done?!

[@SeattleDataGuy]

That's a great question. I will have to make a follow-up!

[@liangbinchen2519]

Great insight! I found it very helpful.

[@SeattleDataGuy]

Thank you!

[@amolnimbekar4817]

Airflow for logging? How? Do you mean orchestration. ?

[@SeattleDataGuy]

Where in this video did I reference airflow? Yes, airflow is used for data pipelines and workflow automation and I am not sure where I reference airflow in this video. The only thing I can assume I meant was I like airflows logging (because you can easily export custom messaging). But if you can provide me the timestamp it would help.

[@amolnimbekar4817]

@@SeattleDataGuy 5:20 to 5:40. IMHO ELK could be a better choice for logging and capture all the metrics but it could vary from case to case basis.

[@SeattleDataGuy]

@@amolnimbekar4817 I see, thank you! Yes when I said that, I was implying the fact that airflow does provide some logging as one of its many features but I can see how it comes off as me referencing airflow as a logging tool only. I think I may have also been thinking about some other ETL/automation tools I have worked with that don't provide any form of logging what so ever and you need to in turn put it into play. I apologize for the confusion.

[@anjalishar1829]

hi there, nice video my question is that is learnin hive, py spark, SQL and python good enough to crack an interview and to what extent is programming matter in cracking a job as a fresher?

[@SeattleDataGuy]

It depends company to company. Some are very heavy on DS and A others are heavier on SQL and data warehouse design.

[@Wes-Tyler]

I've never worked with streaming data (only batch) and never had to improve a pipeline's performance. Can you give your own examples of each so I have a better idea of what those entail/what you're talking about?

[@SeattleDataGuy]

I have a few examples. But, instead of me just giving you a few sentences. You should check out this article. It discusses why netflix switched from doing heavier batch jobs and OLAP to streaming. It is very well written and it connects to a lot of other netflix examples so you can get a lot of context. zhenzhongxu.com/the-four-innovation-phases-of-netflixs-trillions-scale-real-time-data-infrastructure-2370938d7f01

[@Wes-Tyler]

@@SeattleDataGuy thank you so much!

[@Wes-Tyler]

@@SeattleDataGuy, hey, I was wondering if you knew of any other resources where I could learn more about data streaming and ETL interviews?

[@christophermartinez1769]

Thank you!

[@SeattleDataGuy]

No thank you!

[@siddharthsingh5031]

Hey! Bro. That's really a great video.

[@SeattleDataGuy]

Thank you! Glad you liked it!

[@golammuhaimeen2825]

what level of interview was this for? I'm guessing intermediate to advanced? what could I expect for a more entry level interview? Thanks

[@SeattleDataGuy]

Here is the hard thing. Truth be told, for data engineers, this is generally the baseline. I am seeing a lot more JR data engineer positions out there, so maybe those ones have a lower bar. But in general it goes Intern or 2-3 years experience level interviews.

[@2015murat]

The video is great! Thank you! I heard that some of FAANG do two ETL rounds in their on-site interview. One of them is about batch and the other is about streaming use cases. Do you know about those type of interviews? I thought they only ask data structures and algorithm questions for Python part.

[@SeattleDataGuy]

I can't provide too much here since I work at a FAANG. I would say that in my experience some FAANGs do more DSA and others do more operational programming questions.

[@maybenew7293]

'innerview'

[@jasparta993]

6:59 why U said "sequel" not "S Q L". Yes, I also hate ppl which pronounce is as "skull".

[@SeattleDataGuy]

I also called it sequel at 0:16. At the end of the day, I switch between both.

[@jasparta993]

@@SeattleDataGuy Ye, but I get triggered when I heard 'sequel' for multiple times :P But video was good in deed ;)