If you guys want to learn more about data engineering, then sign up for my newsletter here seattledataguy.substack.com/ or join the discord here discord.gg/2yRJq7Eg3k
@yuen36608 ай бұрын
inv is invalid :
@NSFWrated8 ай бұрын
Also upload videos on same format on Data Analyst and Data Scientist I have 300 days to learn 🕶️
@moradabbouazaz3658 ай бұрын
This is really funny because this year my company offered me the posibility to enter the Data Department (I'm a Software Dev.). I'm kind of fed up with coding everyday and for now I'm learning Elastic Search, Apache Nifi and Airflow, then I'll head to Spark and AWS. This video is pretty intereseting to serve as a guidebook. Thanks for stressing the part of "What have you learned so far?"
@ishaan72878 ай бұрын
How is the work life balance in this career?
@JKyadav1117 ай бұрын
What do you think about ai replacing software and data jobs?
@Someone-yl8sb5 ай бұрын
@@JKyadav111 About as good of a chance that AI will replace doctors and engineers
@gobrax8 ай бұрын
This couldn't have come at a better time for me, and a free sheets document to track all of this? So valuable, thanks!
@SeattleDataGuy8 ай бұрын
glad you're finding it helpful!
@jameshizon48618 ай бұрын
One potential key topic is how to bounce back from difficult client/project that did not go as successful as desired (especially in data consulting if you have had that experience). For me, because of this experience, it made me lose confidence during a client screening to try & explain this experience & re-frame it as positive.
@SeattleDataGuy8 ай бұрын
Hmm, this does sound like a solid video topic! Also I hope you feel confident again!!
@jameshizon48618 ай бұрын
@@SeattleDataGuy thanks. U the best :)
@Yavin48 ай бұрын
Data Engineering means different things to different companies. For example, you don't list SAP as a core competency, but some companies do. WIth that said, isn't it better to pick a lane and stay in it? I.E. one or two of the cloud providers, AWS or Azure or GCP, and master data engineering, data analysis, machine learning, AI services, etc.
@tenzinnamdhak4 ай бұрын
Will be following your map for the next 6 months as I am into 9-5 routine.
@HI-bw8fe4 ай бұрын
Thanks for the 3m plan. Exactly 👍🏼 just what I need!
@SeattleDataGuy2 ай бұрын
good luck!
@yeaaayeaa11535 ай бұрын
looking into this now, solid background in data primarily SSIS, informatica, Alteryx with a good amount of "BI" work. Thanks for this guide, we use Azure in our current work environment but mostly only at the corporate level. I'm going to challenge myself to follow this guide so I can transition into being a data engineer!
@travishawks77028 ай бұрын
Thanks for the video. Another video idea would be one for finding / getting your first DE job. It's a bit of a struggle finding entry DE positions
@SeattleDataGuy7 ай бұрын
That would be a great video, i can add it to the line-up
@arulnidhi64828 ай бұрын
Thanks for sharing brother! I'm just going to stick with this plan.
@SeattleDataGuy8 ай бұрын
You can do it!
@mintlatte13766 ай бұрын
Would pet projects be enough to show some knowledge of Spark/Databricks or Azure? I feel like without previous real work experience it's impossible to even land an interview for any data engineering related position.
@EndlessLearner7 ай бұрын
Like number 1,000🎉 Great video! I’m going to start this challenge, you gave us a lot good information😎
@SeattleDataGuy7 ай бұрын
glad you liked it!
@waynelast168518 күн бұрын
Please make another video for people who are new to the field
@madukomablessed47122 ай бұрын
Hi Ben, awesome roadmap, even though you are yet to update some parts such as the article you are yet to upload and the live Q and A😂
@davidkennedy48118 ай бұрын
Starts at 3:58
@SeattleDataGuy8 ай бұрын
Yeah I didn't realize how long my intro was!
@jagannathishere8 ай бұрын
can we get into data engineering with no experience? I'm good with Python and SQL.
@SeattleDataGuy7 ай бұрын
its always a challenge, but often I see people go from data analyst to data engineer a lot.
@gogyooАй бұрын
The spreadsheet points to the AWS Solutions architect cert course. It's 10 hours long. Since then the same author published a 50 hour video as an update. Is it worth it to go through it all?
@antgonz722 ай бұрын
Hi, I recently decided to change my career into the data field and interested in data engineering. Your channel has been very helpful thank you. My question is, I never coded before. Would you recommend taking a Computer Science course as a prerequisite to have a baseline foundation on computers and coding? (For example: Harvard CS50 course?)
@zakariaelmoumnaoui49128 ай бұрын
Is data integration a part of data engineering or what exactly?
@agentsmidt32097 ай бұрын
Yes it is. It is part of the "ecosystem" i.e. one of the things you might be required to do.
@Sanusiinc8 ай бұрын
is Data Integration engineer the same as Data Enineer
@agentsmidt32097 ай бұрын
I think it comes down to semantics. Some companies might call it ETL Engineer but it is essentially the same thing. Think of it this way, companies have pain points and they usually search for someone with a specific skill set to fix that specific problem (or a set of problems). It's like companies who are looking for a "python developer", "MySQL Developer", "PostGres Admin" etc.... but when you read the job description, the skills requirements listed are basically a Data Admin, Data Engineer, or even Data Analyst. Just know in the IT industry in general there is a lot of crossover in skills and duties. A Data Engineer is a "Software Engineer", "Database Admin", "Data Analyst" (some times), " Data Cloud Engineer/Cloud Engineer" etc...just read the job description and determine if it something you can do. Do not get hung up on job titles.
@pineapplepyramid57985 ай бұрын
I’m up for the challenger
@SeattleDataGuy2 ай бұрын
Good luck!
@brainydel8 ай бұрын
Thanks alot for the roadmap man! I appreciate. I just have a question for the cloud services like GCP, AWS courses should I take them all or just pick one. Also many are suggesting GCP. What's your take
@darshantawte74358 ай бұрын
Start with AWS. My suggestion would be to be book an AWS associate exam (Solutions Architect Associate ) 6months/1year from now and start preparing for different services through udemy course by stephaane Marekk. I did the same and cleared the above exam in 5 months (won't recommend to do it though as it takes a lot of effort in preparing , i did this only because my company was paying for the certification which i had to pass anyhow).
@brainydel8 ай бұрын
@@darshantawte7435 Thanks alot
@SeattleDataGuy8 ай бұрын
This was more meant to give you an overview, AWS is a solid place to start!
@beyyugi923 ай бұрын
So if im only interested in AWS, do that 10hr video, and skip the other cloud days and move on? That cloud video is 4-5 years old, is it still relevant? Does solutions architect teach you all the relevant tools in aws for a data engineer?
@kiranyeluru73928 ай бұрын
Great awesome...best wishes from kiranu😢
@SeattleDataGuy8 ай бұрын
Thank you but why the tears
@teonaloving76547 ай бұрын
Thank you
@SeattleDataGuy7 ай бұрын
You're welcome!
@StartDataLate8 ай бұрын
Can you tell more about the project 11:51 "transcript some videos, podcast and then summarize it with chatgpt"
@SeattleDataGuy7 ай бұрын
It wasn't my project but they set up a bot to scrape podcast and videos and then used chatgpt to shorten it.
@kol43218 ай бұрын
Hey this is awesome. Thank you!
@SeattleDataGuy8 ай бұрын
you're welcome!
@tqbchannel32968 ай бұрын
Hi! I'm newbie in this field. Now, I'm following your roadmap in the video posted 2 years ago. Is this new roadmap better? Should I change my current roadmap to this video?
@SeattleDataGuy8 ай бұрын
if you've gone through most of the prior roadmap, i'd finish that. I just wanted to create a timeline vs. the old roadmap that doesn't have a clear time constraint
@diegowargenosky80108 ай бұрын
I am doing the IBM Data Engeneering course from.Coursera. I hope to be ready to create a project and get hired for a first job after that.
@SeattleDataGuy8 ай бұрын
GOod luck!
@daniell.70266 ай бұрын
I have a video idea for you. What would be an example of the minimum project portfolio to begin applying for jobs without previous engineering experience? Like 5 legit projects that include ____ & _____ & ____? I have no idea what's a realistic amount of coding to be considered for a coding entry level job....
@pradhyumansinghmandloi82408 ай бұрын
But you told in your vedio that DE will end in a decade....so why this roadmap?
@SeattleDataGuy7 ай бұрын
I don't recall saying DE will end, but it will change. When i started everyone was on hadoop, now everyone wants to be on snowflake, databricks, spark, etc.
@omarhassan67648 ай бұрын
Luffy reference at 2:04 🐐validates the entire video!! Really appreciate the list breakdown, Benjamin!
@SeattleDataGuy8 ай бұрын
hahha glad you liked it!
@i_youtube_6 ай бұрын
Should I learn a lot about database administration or just writing SQL?
@fung46298 ай бұрын
is data engineer a recession proof or layoff proof job?really worry if I should learn this under economy downturn😢
@SeattleDataGuy7 ай бұрын
I don't think any job is recession proof. It's a reliable job but you never know when a company decides to cut teams
@ChildOfYahweh138 ай бұрын
I have a question? I am interested in becoming a data engineer but I want to ensure I go about things diligently. I see a lot of boot camps where I live in the UK and also the option to go to university. I’m open to both but what would be better for a learning time frame of 3 years?
@ballinspalding112 ай бұрын
Data Engineering is not an entry-level role.
@goadyjerry9678 ай бұрын
Hello what course , would you recommend for study ?
@monicakherajani58408 ай бұрын
Amaaaazzziiinnngggg
@SeattleDataGuy8 ай бұрын
Glad you're finding it helpful!
@willianferreira66137 ай бұрын
Would you recommend any cloud platform course like acloudguru or cloud academy?
@Kamal-ju6qx8 ай бұрын
just curious.can you land on data engineering intern with this 100 days knowledge.
@SeattleDataGuy8 ай бұрын
It likely depends on your background. Perhaps, but you'd have to focus on making sure you could pass an interview
@Kamal-ju6qx8 ай бұрын
@@SeattleDataGuy thanks I'm actually a Computer systems engineering student currently in 4th year have basic knowledge on Databases and cloud computing I'm hoping to enter Data engineering field in the future
@xinyatang96337 ай бұрын
Thanks for the video!! I have a question: what is the book's name at 7:03 ?? Thank you!
@vikramadityapandey89667 ай бұрын
The Data Warehouse ETL Toolkit: Practical Techniques for Extracting, Cleaning, Conforming, and Delivering Data
@danielpolanskyi28128 ай бұрын
can't access that spreadsheet :(
@SeattleDataGuy7 ай бұрын
Were you able to access it? I have tested it and it should work
@evelyns52186 ай бұрын
Hi Ben, I recently got laid off and looking into following your guide here. My background is in google analytics and Looker but we didn’t use too much SQL. Debating between Data Engineering or Cloud Engineering. I do love to build and create new things. Do you have any advice? I need to find a job quick and concerned data engineer could be saturated. What do you think?
@brothermalcolm2 ай бұрын
You from digital marketing?
@evelyns52182 ай бұрын
@@brothermalcolm
@evelyns52182 ай бұрын
@@brothermalcolmyes for the most part but also have project management background and looking into PMP
@technicaltech81416 ай бұрын
@SeattleDataGuy Can AI effect totally to job role of data engineer
@brainydel8 ай бұрын
The first sql video uses SQL Server management studio but I use Ubuntu. Is It a must I learn it or install windows VM?
@SeattleDataGuy8 ай бұрын
You could also use a different database like mysql
@supermarkethobo95678 ай бұрын
is there a way to learn cloud tools without spending a ton of my own money?
@SeattleDataGuy8 ай бұрын
I believe most cloud solutions offer a free tier or trial. For example snowflake does like 30 days free or aws has its free tier - aws.amazon.com/free/
@supermarkethobo95678 ай бұрын
@@SeattleDataGuy hmm okay thanks I'll probably make a new account if my project takes too long 🤣
@mostofamojlish82558 ай бұрын
But it’s hard to crack data interview now
@SeattleDataGuy8 ай бұрын
It can be difficult, but i believe you can do it!
@muhammadyasir76718 ай бұрын
@seattledataguy i have started the datacamps data engineer track. completed about 6 courses in there. what should i do . start this one or continue with that one
@TheShuchika5 ай бұрын
Which datacamp have you started attending?
@ericxls938 ай бұрын
Fabric is the future…
@jagannathishere8 ай бұрын
the discord invite in the Google sheets doc is invalid :(
@feveredpanda8 ай бұрын
the link in the pinned comment works.
@SeattleDataGuy8 ай бұрын
let me add it
@fabmeyer_ch7 ай бұрын
Which is Kim's book? kzbin.info/www/bejne/b3e5lJuNrJh4mLcsi=JkVtysrM5iKPSNr4&t=423
@SeattleDataGuy7 ай бұрын
there are a few references in the excel link but you can find the book here 160592857366.free.fr/joe/ebooks/ShareData/The%20Data%20Warehouse%20Toolkit.pdf
@zxtee7 ай бұрын
really like your content but your body shakes quite a lot and that is quite distracting
@malcorub7 ай бұрын
THIS GUY! 🤣 LOOK AWAY THEN. HAHAHAA
@zxtee7 ай бұрын
guess i have to turn on audio-only mode @@malcorub