What Does a Data Engineer Do? Explained Simply

Combo Package Python + SQL + Data warehouse (Snowflake) + Apache Spark: com.rpy.club/pdp/yYnEMzLOX?pl...
USE CODE: COMBO50 for 50% off
Learn more about courses here - datavidhya.com/courses
The Data Engineering market is going to cross the $100 Billion mark by 2028, this is the best time to learn about data engineering and start your career, in this video we will understand what a data engineer really do and skills required for data engineering
👦🏻 My Linkedin - / darshil-parmar
📷 Instagram - / datawithdarshil
🎯Twitter - / parmardarshil07
🌟 Please leave a LIKE ❤️ and SUBSCRIBE for more AMAZING content! 🌟
3 Books You Should Read
📈Principles: Life and Work: amzn.to/3HQJDyP
👀Deep Work: amzn.to/3IParkk
💼Rework: amzn.to/3HW981O
Tech I use every day
💻MacBook Pro M1: amzn.to/3CiFVwC
📺LG 22 Inch Monitor: amzn.to/3zk0Dts
🎥Sony ZV1: amzn.to/3hRpSMJ
🎙Maono AU-A04: amzn.to/3Bnu53n
⽴Tripod Stand: amzn.to/3tA7hu7
🔅Osaka Ring Light and Stand: amzn.to/3MtLAEG
🎧Sony WH-1000XM4 Headphone: amzn.to/3sM4sXS
🖱Zebronics Zeb-War Keyboard and Mouse: amzn.to/3zeF1yq
💺CELLBELL C104 Office Chair: amzn.to/3IRpiL2
👉Data Engineering Complete Roadmap: • Data Engineer Complete...
👉Data Engineering Project Series: • Data Engineering Proje...
👉Become Full-Time Freelancer: • Best Freelancer Series...
👉Data With Darshil Podcast: • Podcast Series - Data ...
✨ Tags ✨
✨ Hashtags ✨

[@DarshilParmar]

hit that LIKE button and let me know what do you want to see next?

[@tenet7846]

Hii darshil may i know please Whas is the duration of apache spark +databricks course??

[@sanafondu7743]

I want to join on discord, can you please share invite, older invite seems to be expired

[@worldtour666]

Hi Darshil, Please make a series on AWS cloud server less API and datalake development using python. Where glue jobs should be developed using pyapark, source data must be fetched from msk connectors, upload data in s3, create Iam users, using AWS boto3, unit testing using moto3. Maintain db and tables in lakeformation using lf-tags. Athena automation for visualization. Please make series complete based on programming automation

[@adilmajeed8439]

Some complex project on Microsoft Fabric ...

[@pavanparvathanenii4471]

Wow you explained my job in just 10 to 11 mins in a smoother way. Much appreciated Darshil❤

[@DarshilParmar]

Glad to hear that

[@davebatista8636]

aaj sabse pehle is channel ki main main saari videos dekhunga....

[@ARACON_DATA_0605]

How to collect data generated from survey websites and derive insights??

[@tenet7846]

Hii darshil may i know please Whas is the duration of apache spark +databricks course??

[@ShyamDeepak-sp1dx]

is this course availabe on web OR app ? Is course can be accessible anywhere ?

[@haripriyap517]

I've got aws dva certificate 1 year back, but I didn't worked/practiced on aws. And am planning to switch product based company. Please guide me where to start my learning.

[@dhanushtharun8190]

Hey bro books vs online tutorial/udemy course which one is best for beginners to intermediate ❤

[@vaibhavarora9458]

Hi Darshil, I am from a non technical background having BCom(Hons) degree and right now I am having almost 3 years of experience as Data Analyst with skill set of SQL, Power BI and Python (still learning). I have recently left my job and looking for new opportunities for Data Analyst. Also am aspiring for a Data Engineer role in future. I am thinking to get the job first in Data Analyst role and then I'll start learning for Data Engineer. Is this a good way to follow or shall I start preparing immediately for Data Engineer?

[@SagarKumar-lf5dg]

Hello Darshil Sir, Is it possible to switch career from one domain to another domain. Currently I am in Accenture but want to change my career to another domain...could you please guide me

[@rishabhrai9282]

More end to end basic to complex projects using Microsoft Azure Services

[@divyeshpatel2463]

Can you recommend books on python, sql and linux?

[@techtricks1052]

this is a good video

[@PrathameshMawlankar]

Hi great information shared as always. Btw did you start taking protein shake and working out? Just curious since you are looking more buffed

[@DarshilParmar]

Haha I am working out from last 3 years but most of the time I wear long sleeves this tshirt is tight so it makes me look buff

[@BishanTamang-rk5ji]

Love you Brother ❤️

[@DarshilParmar]

Thank you

[@tusharlambat887]

In 2024 : what pays more and has good job vacancies data anlyst or data engineering

[@rohittyagi3065]

great

[@keshavgupta6150]

Great video but next time decrease background music or remove it

[@user-gt9vb6xt9d]

hey darshil is there any way I can get a discount on your 4 course combo as I am a student so it is difficult for me to pay the full fee.Thanks

[@DarshilParmar]

Please email me at [e-mail]

[@rudroroy1054]

I am curious what is the difference between a data engineer and a business intelligence developer?

[@DarshilParmar]

www.linkedin.com/posts/darshil-parmar_dataengineer-datascience-dataanalyst-activity-7196031026966495232-gzE3?

[@BabeRose6]

Hey there, am new to your channel & your study info on Data Engineer and the motivation you give is really appreciated⭐ One clarity/query plz which one is suggestible to set as a career is it Data Analyst or Data Engineer (Azure) plz suggest...

[@DarshilParmar]

Starting with analyst is much easier than engineering

[@paragvade]

I am targeting data engineer role. I have aws solutions architect associate. Lack good grasp over python and sql. Shall I focus on python, sql first or go for aws data engineer associate and databricks cert?

[@navneetkummar4811]

Go with Python and SQL first

[@paragvade]

@@navneetkummar4811 thank you

[@DurgaPrasad-go4gm]

Hi sir i have applied for many companies by watching your videos and doing your projects and making changes but donk know where I am lacking behind

[@DurgaPrasad-go4gm]

drive.google.com/file/d/1hVVoqvitd2wCSFv44f2aVWew2Cbgpg6-/view?usp=drivesdk

[@DarshilParmar]

Check resume and portfolio projects they are very important

[@DurgaPrasad-go4gm]

@@DarshilParmar drive.google.com/file/d/1hVVoqvitd2wCSFv44f2aVWew2Cbgpg6-/view?usp=drivesdk

[@DurgaPrasad-go4gm]

@@DarshilParmar here's my resume sir drive.google.com/file/d/1hVVoqvitd2wCSFv44f2aVWew2Cbgpg6-/view?usp=drivesdk

[@DurgaPrasad-go4gm]

drive.google.com/file/d/1hVVoqvitd2wCSFv44f2aVWew2Cbgpg6-/view?usp=drivesdk

[@shashankpatel634]

Bro... I got all the skills which are you mention in your thumbnail... I will finish my final semester exam in first week of june... And still unplaced... Entry level job pe bhi 3-4years ka experience maang rhe h🙃😭

[@DarshilParmar]

Try Data Analyst position or get internship

[@prathamgupta9256]

R uh stay in Mumbai ?

[@viraj_kadam]

​@@prathamgupta9256 same here mumbai

[@subratparida5647]

Hey bro; for data scraping selenium is a good option or not

[@DarshilParmar]

No, lot of website have bot protection now, not worth doing

[@subratparida5647]

Then what is the good suggestion for web scraping

[@DarshilParmar]

@@subratparida5647 No web scraping go something else like data analyst/engineer or SDE

[@subratparida5647]

@@DarshilParmar Thank you for your suggestion 😀

[@divyeshpatel2463]

Sir what is the duration of your zero to data engineering course ?

[@DarshilParmar]

Recorded content around 50hours for 4 courses with lifetime access, more will be released soon

[@niladripramanik8362]

Hey Darshil I have been in Data engineering field for 1.5 yrs, Is there any research opportunity/ scope in data engineering?

[@sunilKumar-ci9vn]

How you got your 1st job as fresher.?

[@Rhly001]

Hey can you please tell me how you got started with this field.. I'm currently in college and been studying ML/DL mostly but now I'd like to expand my domain.. Is it even worth it and if so then how to I go about it.. Will really appreciate if you take some time and give an in depth response

[@niladripramanik8362]

@@sunilKumar-ci9vn hi it was campus placement only. But the role in company was data engg

[@niladripramanik8362]

@@Rhly001 see if u r already doing Ai ml for a long time, i suggest not to spent time in data engg. But if u want to then sql (all joins and when to use them, then group by , combination of group by and joins, self join, once done with these then windowing functions) - practice from leetcode 50 basic sqls... Python only upto lists.. shell scripting to automate.. Here , mostly u will work on sql only. Python+shell scripting to automate tasks.

[@niladripramanik8362]

​@@Rhly001 bhai youtube is pathetic it's deleting my reply

[@Tranics1]

1view love from Lucknow brother ❤

[@DarshilParmar]

Thank you!

[@shiv8578]

Who edits your videos

[@DarshilParmar]

I hired new editor, what happend?

[@shiv8578]

They are good, i mean how they use exact flow diagrams , are they teach persons? Or you have to collaborate with them to make it happen?

[@dhanasekarank1743]

Bro do you know dbt ?

[@DarshilParmar]

Yes

[@dhanasekarank1743]

@@DarshilParmar thank bro, I am runing the dbt models with databricks and I want to know the running time(I am able to see the running time in VS code) but I want see the running time and bytes consumption in databricks for each models. Is this possible bro?

[@sindhujareddy4659]

Combo Package Python + SQL + Data warehouse (Snowflake) + Apache Spark how much is this course cost in India

[@DarshilParmar]

You can try India VPN for that

[@mayankkrsingh-zq1vj]

I want to connect with you for discount offer? is it possible?

[@DarshilParmar]

Please email me at [e-mail]

[@__nda_channel]

Data engineering is not what I think But you are exactly what I thought You keep milking the topic data engineering rather than telling something else on data engineering

[@DarshilParmar]

Bro, I have 8 projects on the channel, 130+ videos My goal is to target just beginners and fresh to get started, there is no point in going on advanced level stuff because that won't work in this case 10min series to explain Spark, Airflow, Snowflake Videos on new trends End To End Projects Tutorials there are many things, I need to keep balance between different target audience so that everyone can understand different parts