Data Engineer Mock Interview | ADF | Medallion Architecture | BRONZE, SILVER & GOLD Layer| ADLS GEN2

To enhance your career as a Cloud Data Engineer, Check trendytech.in/?src=youtube&su... for curated courses developed by me.
I have trained over 20,000+ professionals in the field of Data Engineering in the last 5 years.
30 INTERVIEWS IN 30 DAYS- BIG DATA INTERVIEW SERIES
This mock interview series is launched as a community initiative under Data Engineers Club aimed at aiding the community's growth and development
Our highly experienced guest interviewer, Ankur Bhattacharya, / ankur-bhattacharya-100... shares invaluable insights and practical advice coming from his extensive experience, catering to aspiring data engineers and seasoned professionals alike.
Our talented guest interviewee, Sasiram Kolisetty / kolisetty-sasiram-0285... has a remarkable approach to answering the interview questions in a very well articulated manner. Where he talks about the complete Medallion Architecture followed in the projects that he worked on belonging to Insurance and Green Energy Domain. He describes how the raw data was initially added to the bronze layer tables. Then performed cleaning and transformations on this data which was then added to the Silver layer tables. Finally, based on the business requirements, data was aggregated and the results were added to the Gold Layer.
Link of Free SQL & Python series developed by me are given below -
SQL Playlist - • SQL tutorial for every...
Python Playlist - • Complete Python By Sum...
Don't miss out - Subscribe to the channel for more such informative interviews and unlock the secrets to success in this thriving field!
Social Media Links :
LinkedIn - / bigdatabysumit
Twitter - / bigdatasumit
Instagram - / bigdatabysumit
Student Testimonials - trendytech.in/#testimonials
Discussed Questions with Timestamp
3:51 Brief discussion about the project.
9:06 What business problem are you solving in your project?
9:56 What work have you done to secure your data?
11:20 Do you use Unity Catalog for role-based policy?
11:33 How do you resolve bottlenecks in terms of latency in your pipeline?
15:14 If you have 2 large tables, what kind of join strategy will you follow?
16:12 Difference between bucketing and Z-ordering.
16:48 How do you perform data ingestion?
18:20 How would you create a pipeline for real-time data updates, aggregation, and live dashboard creation?
20:13 What triggers are available in ADF?
21:18 How does lazy evaluation help in Spark optimization?
22:45 What approach would you take for cluster configuration
24:53 SQL coding question
Tags
#mockinterview #bigdata #career #dataengineering #data #datascience #dataanalysis #productbasedcompanies #interviewquestions #apachespark #google #interview #faang #companies #amazon #walmart #flipkart #microsoft #azure #databricks #jobs

[@satyanarayanabadam9764]

This is the best interview till now. He worked on security , CDC concepts etc. Thanks sumit sir for conducting these interviews. Really helpful who is preparing for the interviews.

[@joerokcz]

Ram is one of the best candidates of the series. All the best Ram.

[@sumitmittal07]

yes definitely, he has a great exposure

[@NaveenKumar-ln9vg]

I am getting prepared for the interview next week. As you said failed interview better then not attending interview and these mock interviews giving me the confidence to face the interview. Great initiative, learning and help. Thanks!

[@sonurohini6764]

Definitely. It is a great help sir.

[@sandhyasandy9072]

This is one of the best interviews and got good insights Sumit sir.Learnt the new concept of thin executor and fat executor and his optimisation techniques are also unique like join re order Adq etc…

[@skillhorizon]

You are doing great work. Lot of respect

[@ramprasadh5322]

This serires is great and gives me confidence to and sit in the interview.

[@niridha23]

Thanks for conducting these mock interviews Sumit sir. It is really helpful😊

[@nsinha27]

Sumit - these mock interviews are worth it's weight in gold.

[@asaisvlogs7258]

It just wonderful task... Its very much helpful ❤❤❤❤❤

[@user-jg2tn1wb3d]

Thank you sumit sir and team, your hard work really means a lot

[@Sudeep-ow4pe]

The interview series is really helpful, Thank you

[@FF-bp9bj]

Sumit sir you are a legend in BIGDATA... You are the best trainer I've seen so far... Thanks for making the bigdata enthusiastics life easier....

[@sumitmittal07]

Always happy to help!

[@sandipansaha6847]

Very helpful video.... gr8 help ...

[@prasadrajupericharla5545]

Great one 🙌

[@bhagirathpandey3560]

Gr8 effort 🌟

[@pradumanyadav493]

this is very helpful series

[@user-dj4ht7rg2f]

Good one!!

[@AnandPatil-eu1tl]

Thank you sir , this videos are very helpful

[@BooksWala]

best interview till now

[@sumitmittal07]

absolutely

[@swapnildande4706]

Really helpful for all Data engineer

[@sumitmittal07]

Happy to know that you are finding the interview series helpful

[@pranavadhav597]

What an amazing interview. This series is so helpful. Thanks sumit sir

[@sumitmittal07]

Keep watching for more such engaging interviews

[@rajdeepreddycharla]

Been watching the interviews everyday. Definitely helpful. IMO based on the performance of the interviewee feedback at the end would be great.

[@sumitmittal07]

we will add it in upcoming ones

[@ravikanth6178]

The interviews are very helpful… Thank you Sumit and Team🎉🎉

[@sumitmittal07]

Glad to hear that you found the mock interviews insightful. More such sessions are scheduled for release in the upcoming days.

[@ameygoesgaming8793]

The series is amazing , I am learning a lot. Please keep making it

[@sumitmittal07]

Happy to hear that!

[@anikethdeshpande8336]

Thanks for sharing. Very helpful

[@sumitmittal07]

Glad it was helpful! Keep watching for more such interesting content

[@salilagarwal7915]

Best interview, every interviewer should give feedback at the end. it can be more helpful.

[@sumitmittal07]

noted the suggestion, will incorporate it from the upcoming interviews

[@ManaviVideos]

Thanks for the interview series!!

[@sumitmittal07]

glad that you liked it

[@DataJourneyHub]

Thank you as always 🙏🏻

[@sumitmittal07]

Always happy to share good content for all the data enthusiasts

[@RohitSharma-ny1oq]

Best

[@chinmayamahapatra6797]

It was really effective to us very very helpfull

[@sumitmittal07]

Happy to know that you found the interview helpful

[@abhirambandi6091]

Too technical man🔥

[@bhushanthoke858]

Realistic interview ❤

[@sumitmittal07]

glad that you enjoyed it

[@VishalKumar-ie3gr]

Very informative.

[@sumitmittal07]

thank you

[@boreddymanohar8339]

It was awesome

[@sumitmittal07]

Glad that you are finding the mock interviews informative.

[@gouthambheema1779]

Very helpful ❤

[@sumitmittal07]

Glad it was helpful!

[@PurbaragPChoudhury]

Most of the interviews seem to focus on junior and mid-level DE. I’d also appreciate some video for Senior DE role.

[@sumitmittal07]

let me know if you would like to attend?

[@chandranshuyadav3515]

Very nice interview sir can we have an interview for someone who is having more than 5 years of experience and is looking to join as an Architect level or Team Lead, Manager in Data Engineering world.

[@sumitmittal07]

Definitely, there would be many more such videos released as part of this interview series. Will try to consider all possible scenarios

[@vijaykumarvc3037]

Sumit Sir, Please share any mock interview which will cover Azure cloud alone if possible

[@sumitmittal07]

As soon as possible, noted

[@ThanmayiTan]

I would like to know the answer for the question, "why in spark why we will get duplicate of the column on which join is performed, whereas in sql we do not get that duplicate ?" From my limited knowledge, I know that in both spark and sql joins we will get the duplicate columns when join is performed on two tables. I would really appreciate it if you could please let me know the correct answer.

[@sumitmittal07]

The reason behind this lies in the fundamental differences between Spark and traditional SQL databases. Spark performs parallel processing to handle partitioned data distributed across different nodes in the cluster. It is inevitable that there would be duplicates while performing joins on such datasets in this context. In contrast, traditional SQL databases usually operates on a single machine or a tightly coupled cluster and ensures uniqueness by preventing the duplicated from being generated in the final results

[@ThanmayiTan]

@@sumitmittal07Thankyou so much ! That really makes sense now.

[@ameygoesgaming8793]

I would like to attend interview, let me know what is the process?

[@sumitmittal07]

Please fill this form - forms.gle/UMpNCZvAHgoLvvuJ6