Apache Spark Architecture - EXPLAINED!

[@t_geek9211]

I just want to express my gratitude for this video, Emil. It's the first time I feel like I truly understand it, and it feels much less obscure now. I really appreciate the part about wide transformations, data shuffling, and why they are computationally intensive. I had a big "aha" moment when you described the process of saving data from memory to disk and then sending it over the network to another worker. Now it makes perfect sense why it is so slow. Pozdrawiam, Teodor

[@DatabricksPro]

Amazing! Such feedback means really a lot and motivates for furthrer work. Thank you so much. Pozdrawiam

[@metinkonuk]

An hour felt like a few minutes thanks to your teaching skills, thank you for this tutorial.

[@MassimoManfredino]

Emil, finally I have fount one comprehensive tutorial about Spark architecture. I have subscribed your channel. Thanks a lot for your unvaluable support!

[@DatabricksPro]

Pleasure on my side. I am very happy you like it and find it useful!

[@yashshah8593]

Very well explained and best part was to include examples for all the concept and explaination. Thanks for such great efforts.

[@DatabricksPro]

Thanks! I am very happy that you liked it:)

[@PankajSharma-z1h]

The best spark tutorial on youtube!!

[@DatabricksPro]

Thank you!!

[@aazimparwaz5386]

Amazing content! You've done absolute justice to explaining Spark! The breakdown of concepts is spot on and super easy to follow. Keep up the great work! Thank you!

[@DatabricksPro]

Thanks for feedback. I am really happy to see this being useful!

[@honeyjohn2997]

I came across your video while searching for spark architecture, and it's incredibly well explained. The content is delivered at a great pace, not overly complicated but perfect for a beginner. I'm expecting more videos like this in the future. great job

[@DatabricksPro]

Thank you for feedback! Video about Delta Lake coming in 1-2 days. Hope you will like it as well.

[@honeyjohn2997]

@@DatabricksPro can u plz provide the link to ur git repo for the code

[@vidhikumar1664]

Amazing tutorial, I know so much about how spark works!!

[@yashvardhancn44]

My sincere thanks for this tutorial. It was of great help. I wish to see more videos from you. Keep up the good work.❤ The way you have explained things, I don't think any other tutorial has explained it with such simplicity and adeptness.

[@AdityaSingh-ps3vn]

This channel will explode in due time, great job! Looking forward to more such videos! (hopefully on spark optimisations soon). Thanks a lot!!

[@DatabricksPro]

Thanks! Next week there will be a Delta Lake movie with a bit of optimizations approach.

[@sekhar7419]

That was really a great explanation, and covering each topic indeeply, I dont see anyone talk about sparkUI and the pwoer of it. Thanks really again

[@maheshpatil3435]

The best way to explain the Spark in action. great tutorial..

[@Lance-tr1qw]

Thank you sou much for this informative tutorial. The best there is out there yet.

[@yvanroan6485]

absolutely fantastic, you went through all the stages. Thank you!

[@yashvardhancn44]

Hey Emil, I wanted to improve more on Databricks, PySpark and other essential technologies on a professional level. After watching your video, I felt I could reach out to you for further guidance to streamline my preparation. I wish you had more videos already in this channel, which would have helped me as a one stop get everything place. But given all this, Can you reply to this comment with some study references which you feel are really good that I can use to become better in these Data technologies.

[@haripandey1798]

Very well planned and explained in details. It really helps me to uncover most of the doubts and gives me clear direction of how to and where to look as what is happening in background when a query is executed. Subscribing your channel for knowledge enrichment.

[@DatabricksPro]

I happy it helps. Check out Delta Lake vid as well :)

[@haripandey1798]

@@DatabricksPro Sure. Thanks a lot.

[@Gaurav-pk2qc]

You nailed it perfectly Dimitri 😇

[@RavindraCh-tj6oz]

Best video on spark Architecture

[@techmetothecloud]

You nailed it. Super useful video!!!

[@marczeller1112]

Great tutorial, thank you.

[@wojciechjaniszewski9086]

really good content, right on the money! thank you for your effort!

[@rangareddy3722]

i’ve watched many videos and tutorials, none of them are in detailed. Awesome work🫡

[@ramasahayamvarshini3816]

Too good explanation!!

[@mertsevenz]

Thanks for the great explanation! It would be perfect to also hear about your production deployment experiences in Databricks Delta Lake projects, including CI/CD, deploying bundles, monitoring, etc.

[@DatabricksPro]

Yes. I also think its a good idea. it will come

[@cybersubhendu]

Hi! Thanks … indeed super thanks for such learning session on Spark. I wonder how many months of learning and understanding been put into 1 hour! I recomend this to all #databricks users to have this understanding of #spark to make life easy with the #development of #datapipeline or #dataanalytics or #dataanalysis or #optimizing your #codes. Thanks !!

[@udittalks949]

so good ,Respect Man !!

[@goldymalik29]

Thank you so much for the great videos. Can I get access to the notebooks as well? Appreciate !!!😊

[@DatabricksPro]

Thanks :) i am on business trip, but will upload in 2 days once back

[@lukaszwybieralski9520]

Brilliant content, thank you!

[@ViralTube369]

🤙🤙💥Worthhhhhh spending time on this video. feels like zero to Hero

[@DebrupMondal-l1m]

AMAZING

[@DatabricksPro]

Btw. Maybe you will also like the latest movie, about Delta Lake. Cheers

[@sanjeevreddy3691]

thanks for your efforts

[@thamimmo]

Great work and thanks Could you please make a similar video like this on delta lake

[@DatabricksPro]

I am glad you like it :) Delta lake is actually really good idea for the next movie.

[@AliRjile]

Thank you :)

[@DatabricksPro]

You are welcome!

[@sanjeevreddy3691]

query regarding cluster configuration -> while cluster configuration for example we specify node capacity as - 64 gb memory (it is RAM) - 8 cores how spark allocates -> storage (disk) ? from this 64 gb how it allocates for -> ram + disk?

[@SonuKumar-fn1gn]

Great video, Please im requesting you please where i get all this 5GB data please provide me link 🙏🙏🙏 I want to use your data which hii have used so i understand eusly 🙏 please provide all 5Gb data

[@avskr-04]

Jiga bytes , airdd