God Tier Data Engineering Roadmap 2024 with End-To-End Projects

Join My Data Engineering Course Here: datavidhya.com/courses
You can directly purchase here:
➡️ Python + SQL + Data Warehouse Combo Plan - courses.datavidhya.com/checko...
Purchase Individual Courses:
1. Python for Data Engineering - pay.datavidhya.com/cop/UHfHDh...
2. SQL for Data Engineering - pay.datavidhya.com/cop/x4UpBc...
3. Data Warehouse for Data Engineering with Snowflake - pay.datavidhya.com/cop/BpfGFS...
Complete Roadmap Document: datawithdarshil.notion.site/D...
Word Document - docs.google.com/document/d/1U...
👦🏻 My Linkedin - / darshil-parmar
📷 Instagram - / darshilparmarr
🎯Twitter - / parmardarshil07
🌟 Please leave a LIKE ❤️ and SUBSCRIBE for more AMAZING content! 🌟
3 Books You Should Read
📈Principles: Life and Work: amzn.to/3HQJDyP
👀Deep Work: amzn.to/3IParkk
💼Rework: amzn.to/3HW981O
Tech I use every day
💻MacBook Pro M1: amzn.to/3CiFVwC
📺LG 22 Inch Monitor: amzn.to/3zk0Dts
🎥Sony ZV1: amzn.to/3hRpSMJ
🎙Maono AU-A04: amzn.to/3Bnu53n
⽴Tripod Stand: amzn.to/3tA7hu7
🔅Osaka Ring Light and Stand: amzn.to/3MtLAEG
🎧Sony WH-1000XM4 Headphone: amzn.to/3sM4sXS
🖱Zebronics Zeb-War Keyboard and Mouse: amzn.to/3zeF1yq
💺CELLBELL C104 Office Chair: amzn.to/3IRpiL2
👉Data Engineering Complete Roadmap: • Data Engineer Complete...
👉Data Engineering Project Series: • Data Engineering Proje...
👉Become Full-Time Freelancer: • Best Freelancer Series...
👉Data With Darshil Podcast: • Podcast Series - Data ...
✨ Tags ✨
✨ Hashtags ✨
#dataengineering #roadmap #dataengineer

[@DarshilParmar]

Let me know your thoughts in the comments and don't forget to hit LIKE and SUBSCRIBE :)

[@sohamshah4666]

why you join Microsoft? you have your freelancing and your own one!?

[@Coolthoom]

@darshilparmar Thank you so much for your videos. I'm trying to purchase your 3 courses through the combo plan link but the page isn't loading for some reason. The individual courses load properly but the combo one doesn't. Kindly advise further.

[@sivasankardevendla4108]

​@@CoolthoomDid you purchase Courses?

[@BShabnam]

Can you please provide me your discord link, the link which mentioned is showing as expired

[@logsofzany]

Bro can you please mention the duration for each of your courses?

[@samarjeetsinghkheda9460]

1) Book:- fundamentals of data engineering 2) Data warehouse (olap vs oltp, dimension table, ETL, ER-modelling, dimension modelling) 3) Snowflake (data warehouse) 4) Book:- The data warehouse ETL toolkit 5) Data processing (bath processing, real-time data processing) 6) Apache spark - real time data processing (core architecture, high level API, functions available) 7) Data spark (Pyspark) 8) Apache Kafka 9) Apache Airflow (workflow) 10) cloud 11) open table format 12) data observability tool 13) docker, Kubernetes

[@kavyamuralidharan6878]

Very clear and driving roadmap video, thanks for the guidance!😃

[@alansmailov6147]

watched till the end and already committed for 4 month, just learned basics of ci/cd in azure devops. hope getting a decent job in new year. massive thanks to u Darshil and all the community u building here! happy new year from Kazakhstan, Almaty

[@LOFI_WORLD_SONG]

Hello can you tell me what are challenging and hard in data engineering also. I am in devops should I continue or should I switch to data engineer. As I don't want coding !!!

[@alansmailov6147]

guys i did it, landed my first job as Azure Data Engineer, huge thanks to Darshil

[@alansmailov6147]

@@LOFI_WORLD_SONGhonestly i don’t know😅

[@MohamedSahith87]

@@alansmailov6147 where i can learn data engineering?

[@guillermocelisprada]

Thanks for the video man, I found it very informative and well-structured. Keep up the hard work! 👍🏻

[@JoseRolandoVillegasMendizabal]

Thanks for this roadmap. I'll start with this since today

[@Shahnawaz_Ansari_]

Great video and resources mentioned Appreciated 👏

[@DarshilParmar]

Glad it was helpful!

[@hritikapal683]

Already followed your 2023 roadmap and learnt everything mentioned just building projects these days thankyou Darshil watched this to get the knowledge of modern data engineering tools quality content as always!

[@DarshilParmar]

Thank you

[@dawateden-yh4yy]

@@DarshilParmar MAKE COMPLETE COURSE AS SHASHANK WEBSITE GROW DATA SKILLS

[@prof.mangabhai]

Hey hritika , how did you build your project, any good KZbin channels or resources you used?

[@jbh7079]

Excellent content. Keep the good work up!

[@faruqolasupo2277]

Thanks for sharing this roadmap - watched to the end with ❤ from Nigeria

[@peterobumneme3911]

Good day. I am also a Nigerian pathing on the journey of Data Engineering. can we link up??

[@ahmetalperenyildirim6184]

Darshil, thank you! I'm following you for 2 years and now working as Data Analyst. I want be a DE! Great topics quite clear, thank you for sharing!

[@khushbukeshari5577]

Hi Can you please help me on this? Can you provide me the roadmap how to move to as a data analyst? What are ths courses are required?

[@haridatta5495]

Watched till the end!! Great content. Will start learning from today

[@thedataholics]

Thanks for the knowledge, you seem really truthful. Keep helping us sir

[@AryanRaghuvanshi-tm9el]

I have watched your end to end project videos previously, really amazing content you have been providing, I will check out your courses.. keep uploading videos like these ..

[@DarshilParmar]

Awesome, thank you!

[@ShivamKumar-pk1rf]

Got the roadmap Thank you so much

[@prof.mangabhai]

Highly anticipated for spark and aws courses !!

[@DataScienceAndEngineering]

Thanks for the video. Really Great Video.

[@reynardtcloete6959]

Great video brother :) thanks man

[@tamarainwang6362]

Getting my first data engineering role in 2024. Thank you!!

[@grant5941]

Thank you, Darshil! I watched until the end! I already started Harvard Cs50 last week so I'm on my way. Thank you for this roadmap.

[@hazarijayesh9807]

Intrested , watched video till the end. ready to start journey for data engineer.

[@Gokulprasanth-gs8zi]

I am already in the journey, thanks for sharing valuable content

[@adib_shaikh]

As per your roadmap that you mentioned in 2023, i just completed the advance postgresSQL , now i sharpening my skills with python, thanks for guiding us❤️ hope I'll become Data engineer , with pursuing your roadmap in 2024😊

[@kimathinewton855]

Same case here

[@trygreenfirst1275]

This was helpful. Thanks

[@ayandazwane_]

Thank you, this video was very informative, I watched all of it and I will certainly become a data engineer.

[@lily0721]

I’m going to become a Data Engineer! Thank you for your useful information!

[@mrweirdo163]

I am gonna become a Data Engineer in 2024 💪

[@mr.chintuji]

Which coding language u prefer

[@TM-op1xs]

Me too

[@rishabhjain1381]

Me too

[@viraleditzofficial789]

Mee too

[@teamxipratikop9247]

Python sir ​@@mr.chintuji

[@payalpokale214]

Hey Darshil,I am working as data engineer since 9 years almost now, couldn’t stop myself to go till last, good content man, almost have touched based all the tools & technologies are mentioned. Yeah it’s like big sea one should keep learning and growing. Keep up the good work and keep creating good content.

[@newagene8276]

what would you suggest a person wanting to switch from swe to data engineering

[@ankurghosh2387]

hey can I connect with you for guidance?

[@viveksingh112]

Thanks Darshil, I watched this video till end and will keep following you to build my journey to become a Data Engineer

[@user-om7lq4yk8s]

thanks for the video Darshil.Pls keep posting such awesome content.

[@DarshilParmar]

Thank you, I will

[@radhikab4864]

i have watched this video till end. I love the way you are teaching concepts and projects.

[@surajshivakumar5124]

Hey darshil! Can you give us some tips/advice on contributing to open source as a data engineer?

[@njorogekamau3820]

I'm giving myself at least one year to learn data engineering without any expectations. I hope to revisit this comment again next year having landed a data engineering role.

[@ankitapandey398]

Thank you so much for this video, This has cleared all my doubts. You are doing great job for beginners like me and many others. I have watched this video till the end

[@DarshilParmar]

Glad it was helpful!

[@joelsarpong1847]

I will become a Data Engineer in the year 2024 due to your road maps and your course provided. Looking forward to the rest of the courses coming up.

[@sanjanagoud1155]

Thank you for the video Darshil..... I am going to start my career in the field Data Engineer. for sure in future i will become an Data Engineer.

[@SanjayKumar-rw2gj]

I'm going to become a high skilled Data Engineer in 2024. Will follow the roadmap thoroughly. You mentioned that we can get all these skills in 6-8 months, how much time daily I need to spend and there are bunch of tools to learn like, sql, python, Kafka, spark, cloud and many more is there any specific order to be followed or can few things be done in parallel?

[@Cheer4SK]

I’m going to become a data engineer! 🥳 You guys have got this, too! 🌟

[@okorochaobinna7701]

You're a blessing to an aspiring data engineer

[@vasundras4746]

This is the best video on KZbin right now 💯 we need more of this energy !!

[@ManiKanturi]

Thank you :) I'm going to become the Data Engineer. Please make the course for the remaining topics or tools.

[@sukritisachan5773]

Hey Darshil ! I had a question, my free subscription for Azure Dats Factory has ended, so is there any other way I could do hands on for that(without pay as you go model)?

[@DarshilParmar]

Use different email and card, this is the only way! if you have student id then you can get some free credits

[@amitjaiswal781]

Nice Video, Please also make a video on a GCP Data Engineering Roadmap.

[@simpledrawing7284]

Hi Darshil, can you make a video on role of data engineering in Generative AI and LLM models.

[@DarshilParmar]

I’ll put this in list

[@icon5819]

Hi Darshil, can you please drop rest of the courses quickly. I want everthing related to the course at a single place and your course is worthy enough for that

[@yogeshkumar-du4jx]

Hi Darshil, its really a great content 👍 . I have watched your video till the end and you have shared the information in very good manner. I want to become a data engineer

[@karthikvibuthi9635]

I will start my data engineering journey today. With many projects I will gain skills from these road map as well as essential technologies outside the course and as they emerge

[@mr.amitdas1541]

i ve completed this vedio.. This is very insightful.. Thnk you for the guid.

[@Moses.Daryoue]

Thanks a lot, you have given me a lot of information and clarity on how to start data science engineering

[@soumobratamanna6681]

I alreasy started learning data engineering after watching your past videos , I will be become data engineer . Thanks for the clear roadmap and instructions .

[@DarshilParmar]

All the best

[@soumobratamanna6681]

@@DarshilParmar Thanks a lot for sharing such insightful content

[@soumobratamanna6681]

@@DarshilParmar Previously I was working as a data engineer in a early growing startup company in Bangalore. I resigned last month only due to some project issues. Now actively looking for a job . Would like to get some suggestions from you regarding job hunt in data science domain

[@navarrodba]

20 years as an Oracle DBA. Finally I’ll move do DE

[@yousef5886]

Thank you for this video Darshil! I am already a Data Engineer but my foundational knowledge is not the best and there is so much I still do not know, so thank you for explaining so thoroughly 😊

[@yogendrasharma7534]

Hi Darshil, you are doing good thing at best, and i am sure that i'll be a good data engineer in 2024 itself, although I have 7 years experience in SCM in manufacturing sector.Hoping to connect both in 2024😊

[@mijailsan241]

Good Resume of maybe 5 years learning😊

[@harishparamathmananda1385]

Watched till the end! Thanks for the tips and tricks, I'll become data Engineer in less than 6 months. 🎉

[@vivektyagi5744]

Great video Darshil. Watched till the end.

[@brjkumar]

Thanks Harshil for this video.

[@DarshilParmar]

Welcome!

[@funwithsrivatsav6889]

I will be data engineer by March. And watched your video completely. ❤

[@oshebase]

Thanks for this guide. I'll bevome data engineer come 2024

[@nasirkadri8361]

Hi @DarshilParmar Please let me know when are you going to launch apache spark course? I am eagerly waiting for your course. So please give us tentative dates when are you launching it...

[@gudiatoka]

Is dbt and snowflake required for azure data engineer???

[@DarshilParmar]

Yes

[@lamboarun]

Informative & Well Structured 😊

[@kalyanishende618]

Thank you Darshil... I am on the journey to become DE now watching this video and the roadmap you mentioned in the video is pretty cool. Thanks for sharing such useful content

[@dreamlands3970]

I am now watched complete video i will become data engineer in 2024 from basic to strate from now❤

[@namitabehera4954]

Yes i follow you and i completed your video. Thank you for your suggestion.

[@SatyaMalik-id5vm]

Thank you for being this clear and complete roadmap i just started my data engineer journey ❣️❣️

[@rishiraj2548]

Thanks

[@srajangupta2921]

00:04 Data engineering is a fast-growing field with high demand and numerous tools available. 02:24 Acquire computer science fundamentals and foundational data engineering skills. 04:41 SQL is the universal data language for communicating with databases. 06:58 Learning approach: Combine video learning with reading a book for effective learning. 09:11 Learning about data warehouses includes foundation and tools. 11:33 Data engineering involves bringing various data formats together and processing them efficiently. 13:52 Learning Data Engineering and Cloud Platforms 16:09 Open table format and data observability tools are key trends in data engineering. 18:10 Stay consistent and learn in public 20:15 Encouragement to engage with the video content

[@karenemody]

Thanks Darshil. Very informative. Watched till the end.

[@farantaimoorbutt5028]

Hey I am from a non tech background.I am a self taught full stack engineer.I couldn't find a job .Now thinking of switching to data engineering after watching you video .Whats your take on this? ->I am going to become a data engineer

[@arjitsingh3519]

I would like to start my career as data engineer. I already familiar with Python, SQL , DSA , and linux. Learning all those additional technologies/tools its not a big deal for me. But the problem, I have a 4yr career gap. Few people told don't go for data science related domain because it is difficult for a fresher with a gap to get job. Is it that true can i go for full stack rather than DE? I'm stucked Kindly enlighten me bro..

[@DarshilParmar]

Hi, You can always explain your career gap and also showcase you took efforts to learn new things, if you can prove you have skills by projects then no one can stop you

[@mukeshmahadev7419]

Good commitment hack. Thanks.

[@bhoopendrasinghkashyap1202]

Sir, when Spark for Data Engineering module will launch on Data Vidya ?

[@shyamsundarrao7444]

Hy, hi I tried to join your're course but the coupon code is invalid can you help me out

[@faiqkhan9641]

Sir can you suggest which cloud certification should I do azure or aws

[@shandeepraula]

I am not able to open the aws account, it is showing invalid as soon as I enter the debit card information in the payment option. please give some solution

[@sriharisrinivasan1307]

Quick question: If we purchase your all-in-one combo course, Will you add the Apache & Pyspark course, or will it be launched separately as an additional purchase? Would like to have all round course material at one place for All the Data engineering topics.

[@DarshilParmar]

It will be separate, current price is according to 3 courses only

[@sriharisrinivasan1307]

@@DarshilParmar Okay thank you

[@nagalakshmia1291]

i have already started my data engineering career that's from now

[@AbhishekSingh-xq6lz]

Hey Darshil , i am working in GCP from last 5 years , Do i need to learn Snowflake for warehouse or proceed with current BQ ? Kindly suggest

[@angellavado763]

I'll become a master in Data Engineering in 2024! 💪🏼🤩 Greetings from Peru 🇵🇪.

[@aditisingh4018]

Stayed till the end ❤ A data analyst here, will be a data engineer in 2024 ❤ Thanks for the video

[@Soneone-kz4fp]

I am gonna become a data engineer Plus, watched your video till the end, I also followed your uber analysis project, it was a real good one

[@anything_and_everything921]

Hi @DarshilParmar love your content. And have followed your content. Can you please please make a roadmap for interview preparation. Thanks in advanced

[@manojreddy2871]

thank you sir, for giving us a helping hand to become a data engineer.

[@wckd6726]

Watched it till the end and I swear to god I'm going to become a data engineer this year

[@hamzaabbas8563]

My goal is to become a data engineer by the end of the year 2025 mastering all the concepts and technical tools related to this field

[@sivasaiemail]

Good post

[@JayPatel-yf7ot]

I have a question regarding the combined course. It the combination of separate 3 (pyhton,sql and data warehouse) courses or the content is different a bit. Plz guide.

[@satish8437]

I will switch my career from web developer to an Azure data engineer by mid of 2024. Let me check back in June how I’ve progressed !!

[@Rits2203]

update??

[@Reactionz.3402]

I'm going to become by Data Engineer before the end of 2024. The journey just began.

[@prof.mangabhai]

Average salary trends bhi share krdo bhaiya for DEs

[@mrafam626]

Glad I found your channel! How adept do I need to get in SQL and Python (as a complete beginner), or should I get the basic knowledge then focus on aspects of both programs that are used in data engineering?

[@njoy2075]

I would say get the basics bare minimum. Then find one YT python and SQL project and start building it. Build 2 , 3 projects then go deep in python sql

[@dhyeyhiteshkumarsuthar7150]

​​@@njoy2075 Can u provide complete roadmap pls? Because in this video i didn't understand that much. I m learning python and SQL. Should I make projects after learning this or after learning whole data engineering thing. Pls just can u provide roadmap like what topics to master in sequential manner?

[@njoy2075]

Once you are okay with basics move to projects then again back to sql python advance then again project

[@DANNYEL20122]

So I'm learning python specifically for data engineering. What aspects of python do you think i need to focus on

[@njoy2075]

Think of building a project where you have to parse a log file then join it with scraped web data then once the final dataframe is available, push it to sql. Now you can Google the components needed for the project. That's what you need to learn.

[@nikhil09090]

I am becoming a data engineer! Thank you for your guidance. By the way, I love your hoodie. Could you please share the link? I want to buy one. 😁

[@LOFI_WORLD_SONG]

Data engineering is easy to learn ? I dont want to code like dsa or oops. So , devops or Data engineering will be good ?

[@neeleshvishwakarma6583]

I am going to become data engineer before may 2024

[@deepthireddy3480]

I am going to become data engineer.. I have watched your video till the end.. And i wanna say you something that you are going to reach 200k subscribers very soon... Thank you so much for spreading such a great knowledge..

[@DarshilParmar]

Thank you, jai Shree Krishna

[@deepthireddy3480]

@@DarshilParmar jai shree krishna

[@sabarishv9043]

Excellent Road map, sure I'll become data engineer in two months ❤

[@johnsharma597]

going to be future data engineer / scientist

[@Korudo]

I’m going to become a data engineer. I do data integration in my day job, and I want to grow and become truly excellent at this.