How to Become a Data Engineer

[@mohamedaminchairi]

Graduated in IT, (Bsc) I worked as a BI Consultant for 3 years , mainly build consultancy skills and learned to write nice ETL-jobs for a Big 4. Then started coding in Python as a beginner for another consultancy company :-D . Then Started with another company as a Data Engineer Consultant working on Cloud (AWS and Azure), then started Freelance :-D. For the record, Data engineering is quite wide role, you'd do a lot of different things : Creating API, industrializing Data Models, Creating data pipelines, deployment (Devops) etc ...

[@ozland7172]

Do you work with cloud solutions (azure, aws) as a freelancer? I am working as a contractor consultant and no idea how to find a freelancing jobs where I can work on Azure platform.

[@itsmemcmc]

Simple and direct 😊 The way you tell your experiences is awesome! A few years ago, I fell in love with SQL, ETL and Python (the last one was a love-hate relationship). Now I'm about to graduate and becoming a data engineer is what I want. Thanks a lot for your videos!

[@karolinasowinska]

Thanks for your comment and good luck with becoming a data engineer! You can do it! ;)

[@DAJakaRedAries]

Thanks for your videos and all the inspiration you gave. I finally got a career switch inside of my company from Data Analyst to Data Engineer. Feels good.

[@karolinasowinska]

That is awesome! Congrats, and have fun as a data engineer now! :)

[@jlrifer]

I am designing a plan of career change, from Senior DBA Oracle (Postgres, MongoDB, Redis...) to Data Engineer, but the initial salary must be better than a Junior position for me, because I have a family to take care, so I am trying some ways to start at least as a Medium Data Engineer, getting better with Python and some usefull tools for a Data Engineer position, studying before the carrer migration, 6 up to 12 months, I don't know! I've just knew your channel and I am liking a lot the way you put the things on the table! Thank you Karolina!

[@johnwood1551]

Thank you!! Finally, someone who's honest in how to learn to get into this kind of industry. I became a data scientist by applying for junior roles, I avoided grad schemes, barely read any books, and courses just never did it for me. I learned by doing! And believing I could do it. Now I'm about to start a new job as a data manager in a startup, and need to learn data engineering. Hence why I'm here. I'll absolutely be following and subscribing! Keep up the good content!

[@karolinasowinska]

Thanks so much John! Definitely, there's nothing more valuable than hands on experience in this industry!

[@soshamike1718]

How did you get a foot in? Tech talks or internships? Aslo what is your background in?

[@johnwood1551]

@@soshamike1718 I did work placement with my masters. But I know that's not possible for everyone. Background is in public policy.

[@statisticianwordnpl4071]

There are at least more than 20 channel of people that talk about data engineering and related. But you are the best one! hope the best, cya again from italy :)

[@karolinasowinska]

Wow, that's quite a distinction! Thanks!

[@danballzeta]

This give me a hope for transitioning from mechanical to data engineer.

[@karolinasowinska]

I'm glad this was useful, and good luck :)

[@rahulvyas5749]

hahaha

[@leftsidedrive_]

I'm literally on the same track in my career change. We can do it!

[@rushikeshbhavsar1564]

@@leftsidedrive_ me too 🙌🙌🙌

[@MrMultiMediat0r]

Oh yeah, no problem. Swap out your Matlab knowledge with python, and your stress strain relations with relational databases and you're good to go friend!

[@higormonteiro7636]

Super informative! I'm finishing my masters and feels good to see that I've already good skills to start doing data science/engineering (I'm exploring the two options right now). Thanks for the SQL tutorial link, btw!

[@karolinasowinska]

I'm glad I could shed some light on data engineering! Good luck with finishing your Masters! (I can imagine it can be a little stressful towards the end!)

[@benmack3629]

kzbin.info/www/bejne/n6C9f4Gcrb2Lp9E so we are confusing data engineers with data scientists and data analysts. Data engineers are Network, Computer systems, and telecommunications engineers

[@karolinasowinska]

I have done some research since releasing the video, and I can recommend following resources for learning data engineering: 1. Learn Python I recommend following the Python for Everybody specialization course on Coursera, which is one of the most popular courses there: imp.i384100.net/x9gVO3 2. Learn SQL SQL is still the lingua franca of data. I recommend going with Learn SQL Basics for Data Science course, because it contains some chapters which are very releavant to data engineering in partcular, e.g. distributed computing with Spark imp.i384100.net/QOMZ09 3. Learn Bash scripting/Linux I wouldn't take a full course on it, but at least read a good article. if you do prefer to take a course/guided project, I think this one is short and good: www.coursera.org/projects/command-line-linux 4. Learn how to develop on the cloud, e.g. on AWS There are a few good courses around there, but I think the Coursera one is the most comprehensive imp.i384100.net/P0MJBM

[@AngadGrewal]

There's some very useful information in here for those that need it, a great video as always! p.s. I really agree that the way to learn python for data engineering (or anything really) is to work on a project as soon as possible :)

[@karolinasowinska]

Definitely, getting out of your comfort zone is always the most effective for, as you said, learning anything!

[@shanemurphy1774]

Thanks for your videos. Has certainly given a better insight on Data Engineer and does make me I haven't wasted my Physics course completely!

[@helovesdata8483]

You were so correct. I see Data consultancies everywhere

[@claudiusandika5366]

I always like the way you tell stories 😂😂 funny but knowledgeable ❣️

[@karolinasowinska]

Haha thanks! ;)

[@cucumberingin]

omg I love your vivid style and of course content! Hope you’ll keep the serie of DE videos rolling!

[@karolinasowinska]

Will do, my pleasure! ;)

[@ai.201]

I applied for Twitter developer API after watching your video today, hope it works.

[@sujaykha]

I have been job less for 4 months ...because i was looking for roles as an analyst. I was best in class while programming, but I do lack communication and case study solving skills required for analyst position. I am pretty sure i could learn any programming or technology super quickly. I am going to apply and study for data engineer roles for next 3 months. Thank you so much for such a detailed video.

[@karolinasowinska]

My pleasure and good luck! You can do it!

[@RenWater]

Exactly what I needed, very informative video and thank you very much for sharing! Definitely worth the wait ☺️🙏🏼

[@karolinasowinska]

I'm super glad that you think so! Thanks! ;)

[@CesarSantosLopezYolo]

Just subscribed and loved the video all the way to the end... Thank you for your deep insight. Much love from 🇲🇽 México.

[@karolinasowinska]

I'm super glad to hear that! Thanks! ;)

Great video, thanks! You have answered almost all the questions I had in mind concerning the topic. Also `sl` and `cmatrix` had been the first two quirky bash commands I had installed first too to make myself feel empowered in taming the command line. XD

[@victorborges5291]

Thanks much for your feedback Karolina! As I'm just a babe in the woods, would want to first study for the Bachelor in Computer Applications while simultaneously working towards becoming a self-taught Data Engineer. And, of course, watch your and your guru's videos to accentuate my learning curve.

[@jmediaonline9749]

Great content, thanks Karolina. I am about to graduate this fall and am trying to figure things out!

[@jujubats]

I also have an undergrad in Economics and am wanting to make a switch to Data Engineering. This is inspirational

[@ravishankar0901]

I have given a like before even viewing the video because I knew it would be helpful. Really thank you, great content.

[@karolinasowinska]

Aw that's so nice, thanks for your trust haha! ;)

[@ravishankar0901]

@@karolinasowinska I have also gone through the link which you have shared - Data Engineering Pipelines. Its good framework as well. Thank you

[@iamchesco]

Thanks for sharing, your insights. I always find your videos very informative 👍

[@karolinasowinska]

I'm really glad that you think so, thanks Francisco! :)

[@vivekkeshri536]

once again really good overview on DE tech stack learing resources :)

[@karolinasowinska]

I'm glad you think so! :)

[@frannas1477]

Title: How a data engineer to become

[@karolinasowinska]

Ahaha oops... I wouldn't make a good UX designer

[@Robi2009]

2k subs! Congrats!

[@karolinasowinska]

Thanks! ;)

[@nickseferovic6472]

So informative video. Thank you Karolina.

[@karolinasowinska]

My pleasure, thanks! ;)

[@omarelfoudeh4370]

your videos are great, entertaining, funny and informative, keep it up!

[@karolinasowinska]

Thanks a lot, that's the plan! ;)

[@grokes_workshop]

I agree, they are often spoonfeeding you too much in the online courses, therefore working on your own project can be a better idea.

[@karolinasowinska]

Exactly!

[@waydathemayor]

your videos are so helpful! Thank you Queen!

[@karolinasowinska]

I'm glad you found them useful!! :) Thanks for being here!

[@bhuvaneshrao6322]

Watching your video is just sooo satisfactory 🙂 I was just feeling like this phrase "cake walk " 😄

[@karolinasowinska]

Hahah I love the phrase! And I'm glad you think so! ;)

[@cagan8]

Hey loved the vid! Im going to graduate w CS degree in Spring 21. Im still not sure whether to follow saturated SWE path or something like Data Engineering. If you were to talk about salaries as well, how is it compared to SWE roles? (West Coast)

[@karolinasowinska]

No idea about the West Coast specifically, sorry :( I'm biased, but I would pick data engineering anyway;)

[@Soulis98]

Finally some quality content for data engineers 🙏

[@Smoothbluehero]

I would appreciate it in you could go more in-depth on making the switch from Economics to Computer Science. I majored in Chemistry but went into Sales. Things have not gone well so far. I'm honestly considering going back to school and getting a Masters in Computer Science but before that I will go to a community college and take a variety of classes first to see if Comp Sci is for me. I'm really at a cross roads right now, would appreciate your insight, maybe a video?

[@karolinasowinska]

I can relate to your situation - I guess I had similar feelings before making the switch. Sure, I'll think about a making video about this soon! :)

[@KP-nc9gk]

YOU ARE A LEGEND! 🤟

[@karolinasowinska]

Oh, how flattering!!

[@remyahari1922]

Thanks . Everything is so clearly explained.

[@andytheindividual3862]

This video was helpful!! Thank you! :)

[@karolinasowinska]

I'm glad you think so, despite the absence of my guitar! :D

[@ziad_m_404]

Thanks for your efforts ^^

[@karolinasowinska]

My pleasure! Love making videos! ;)

[@mytechchannel2519]

Really informative.

[@karolinasowinska]

I'm glad you think so! :)

[@nazreshid1348]

Hi Karolina, do you happen to know any data consultancies in San Francisco Bay Area?

[@karolinasowinska]

Unfortunately not :(

[@leeanne562]

This video is amazing💪🏻💪🏻💪🏻

[@Javvid-ipynb]

Thanks a lot ❤️

[@bettatheexplorer1480]

Hey, im an econ grad too. And currently doing data engineering work.

[@karolinasowinska]

Hey, high five!

[@davidmorales3659]

Great video! I was wondering if you had any suggestions on topics regarding "Learning/Experienced in Linux" I always see things like knowledge in Linux or stuff like that, but I am not sure what does that mean job-related.

[@karolinasowinska]

That's a good question, I'll try to look for a good resource.

[@jofla]

there is something about data that attracts me -.- even tho I'm a software engineer with 2 years of experience I might just yolo it

[@karolinasowinska]

Yolo it ;))

[@zentamil]

informative video... thanks for sharing

[@karolinasowinska]

My pleasure!

[@codingevelynmiami4230]

What are some questions a person should ask at the end of a junior data engineer interview? Or ask during an informational interview?

[@karolinasowinska]

Find out what tech stack they use :) what databases, what scheduling software etc

[@benmack3629]

so we are confusing data engineers with data scientists and data analyst. Data engineers are Network, Computer systems, and telecommunications engineers

[@BestevertechBlogspot]

Could u make a detailed video on starting with data engineering projects for undergrads in college

[@karolinasowinska]

I'll think about it, thanks for the suggestion! ;)

[@KehOlaleye]

Nice Info. Do you have any course on data engineering

[@TomCatesGPLUS]

"How much SQL is enough SQL!?" 😍

[@TomCatesGPLUS]

These are questions I've been asking - longtime SWE trying to make the switch to a slightly less easy-to-automate job . . . but I have been focussing on Python/SQL/AWS tools for pipelines etc (studying for new DE cert in hopes it'll give me a leg up)

[@achiradissanayake3580]

All our Data Engineers are on vacation for Christmas. So I am here to minimize the damage while they were gone...🤣🤣🤣 "Oh.. boy...!!! I didn't break it.. All I did was what ya'll told me." Wish me luck...!!!

[@karolinasowinska]

Hah good luck!

[@achiradissanayake3580]

@@karolinasowinska Thank you 😊

[@omkarbhiwande7723]

Do I need to know machine learning for data engineering roles? I do not like ml but I am good with SQL, python and Linux.

[@karolinasowinska]

You don't need to learn ML for data engineering:)

[@okzapradhana5600]

Hi Karolina! Thanks for providing us this informations, and I love the way you answer the questions with using simple terms/words. I had curiousity about Database Design since I need to learn it as well , at 9:02 you mentioned about the link from your ex-colleagues but seems it didnt appear on the video. May i ask for the link to the sources? thanks in advance❤️

[@karolinasowinska]

Oh, interesting, the link shows up for me! :O Thanks for letting me know. Here's the video: kzbin.info/www/bejne/pqvJmJWJm9-bjsk :)

[@okzapradhana5600]

@@karolinasowinska great! thank you so much!

[@kyledeato1496]

Nice video! From no experience at all how long do you think it will take from 0 to be ready applying to somewhat entry level position?

[@karolinasowinska]

Hmm that depends. If we're talking about full days dedicated to this, then maybe 3-6months

[@gkhfbnhfvng]

"Similar hopeless position"??? Economics and CompSci is kinda great bakground for data related jobs, at least you know some high level math. I'm a humanitarian grad so i looked ridiculous, awkward, pa-the-tic on an interview i'd recently had haha.

[@karolinasowinska]

Hah I'm sure you didn't look that pathetic if you did learn some data-related skills beforehand! ;)

[@gkhfbnhfvng]

@@karolinasowinska Just a bit. Thanks for a cheerful response haha. You're very cute btw :)

[@yomismo5711]

Hi Karolina! 😊 Is there any relationship between Data Engineer and Devops?. I had an opportunity to make an internship about Devops and I refused it 🥳. I don't know if I did the right thing but I still have the contact of the person who interviewed me and was so interested in me. Then, ... from video do you recommend Data Engineer more than Data Scientist way?. At the moment I am following with my Python and SQL and any other thing more there 😉 Thank u very much for the videos and your work! I hope to see you soon 😍 Kind Regards! 👍

[@karolinasowinska]

DevOps is different than data engineering. DevOps is a about setting up the infrastructure on which the code is run (so like setting up Github, connecting it to a scheduling software i.e. Airflow etc etc). There's no right answer to whether DE or DS is better, pick which one you like doing most ;)

[@TheVidlove]

Hi Karolina! What about Data Quality engineers? It’s relatively new position and I’m curious if it’s a good start for data engineering career

[@karolinasowinska]

I think data quality engineers are essentially data engineers, but perhaps less focused on building pipelines :) So why not, it will give you exposure to the whole ecosystem!

[@TheVidlove]

@@karolinasowinska thank you! 🤍 I couldn’t find many differences too)

[@MsS4a6]

i love you

[@gamaleldinsamy9416]

Your videos are both informative and funny. I love them. I have an interest in becoming a Data Engineer, however, my bachelor's degree is not CS-related (Petroleum Engineering - hard to land a job theses days) and I found out that u had a major in economics and had a master degree to make the switch. Can you shed some light on the direction to finding such a program? When I looked them up, nearly all of them require a bachelor's degree in CS, EE, or IT. THANK YOU for reading thus far!!

[@jordan7027]

Check out Glasgow MSc IT - it’s made for those with no computing background, I’m doing a bioinformatics MSc there and I take 2 modules from the computing school.

[@potatoostudies2881]

Between the two...Database Admin or Data Analyst? As a stepping stone towards DE

[@karolinasowinska]

Hmm probably database admin!

[@mithunnambiar1433]

Hi Karolina, Could you tell us the SQL guru whom we could follow that you mentioned in this video?

[@karolinasowinska]

Hi, I refer you here: kzbin.info/www/bejne/pqvJmJWJm9-bjsk. :)

[@corallover3616]

I would avoid Kubrick Group like the plague. They are frequently bullying consultants and putting them in roles not related to the field they were trained in. Joining Kubrick as a Data Engineer doesn't necessarily mean you'll become a Data Engineer, but you'll still be locked into the contract with 20k to pay to leave.

[@karolinasowinska]

That is true :( Generally, I'd advise against it

[@soumiksaha]

hi karolina... Hope you are well....what happened to your eye sockets???

[@karolinasowinska]

Hahah make-up I suppose, they call it 'smokey eye' :D

[@soumiksaha]

@@karolinasowinska hahaha good one....and nice info on data science topic 👍😊

[@NguyenPhuong-nk4lm]

omg,you are so pretty!! Now, I am struggling with creating a project about Data Engineer. Could you show me where I can start? Thank you so much

[@karolinasowinska]

Oh thanks :) Check out my data engineering course for beginners :)

[@convallariamarjalis2805]

do i have to know data structure & algorithms to become a data engineer ?

[@karolinasowinska]

It helps with the interviews, yes!

[@peterukwadinamor2622]

I have a bsc in accounting and I’m currently a data analyst. Is it possible for me to transition to data engineering?

[@ssbrunocode]

I think tutorials like this are really usefull ;D

[@Naiuhz]

9:02 The link didn't appear on mobile and I couldn't find the link in the description. Can someone reply to me with the link?

[@karolinasowinska]

kzbin.info/www/bejne/pqvJmJWJm9-bjsk

[@Naiuhz]

@@karolinasowinska thank you!

[@brandonwilson8115]

What is the equivalent "Technology graduate schemes" in the US? Is that mostly like what we would call an internship?

[@karolinasowinska]

Oh, I always thought they are called "graduate schemes" everywhere! I don't know any US-specific name for that! I think internship often implies that you do it in the summer during university. Graduate schemes are like the first job you get after getting a degree

[@laughoutmeow]

I'm about to graduate with an applied computational mathematics degree from California. Should I look for data analyst or data engineering roles? Seems like data engineering is more lucrative around here...

[@karolinasowinska]

It might be more lucrative indeed to pursue data engineering. I guess it depends!

[@barajasfelipe16]

I just obtained my B.S. in Mechanical Engineering but I want to transition to a Data Engineer career. Would you recommend the Google Data Analytics Certificate? That certificate teaches SQL and Python and data bases.

[@karolinasowinska]

It should be a good start :)

[@samirz7]

Ohh the dreadful tutorial trap ! :0 I am already drowning in it and in the process of trying to save myself before I get sucked in further into this madness. Great video and also another wakeup call for me.

[@karolinasowinska]

It really is dreadful! Enjoy applying your skills to some real projects soon! ;) (and don't get discouraged if it seems really hard)

[@okzapradhana5600]

Did you have any books recommendation that we should read to mastering data engineering? E.g books about database modelling or database design, books about big data, etc. Thanks before!

[@lukevassor7743]

Great video! Do you have recommend knowing any other languages above Python? Java over C++? Thanks!

[@karolinasowinska]

For data related jobs, definitely SQL :) in general it might be handy to know one of Java, C#, or C++ but in my experience it's not typically a requirement for data jobs.

[@lukevassor7743]

@@karolinasowinska Thanks!

[@snowydadog2788]

Are data engineers considered professional? do they hold license? do they have the initials "Engr." before the name?

[@karolinasowinska]

You can finish any STEM degree, or even have no degree at all to be a data engineer. Essentially you just need to know how to code. :)

[@isaacmolina9818]

What do you think about the Microsoft Azure Data Engineering certification?

[@ajaygautam360]

Hey karolina, i've watched your videos on ETL process and that was really helpful for me. I am from India and i wants to join "Technical graduate program" offered by HSBC. Please guide me with the procedure.

[@mohamedakl9159]

Hi Thanks for the fruitful video At 3:10 Do you mean a salary between 3000 and 5000 pounds per month is fair? Or is the average

[@yrysf777]

Hi is Data Engineering and Big data Engineering the same thing ?

[@sivankakkar2281]

Sensible person: how a data engineer to become Engineer: how to become a data engineer

[@karolinasowinska]

So you're suggesting that I'm a sensible person but not an engineer? :D Hmm. I guess I'll take it!

[@youtux2]

Linux is used _everywhere_ in the server space.

[@karolinasowinska]

100%!

[@rmp5640]

Great video! I'm almost bachelor degree in Informatics (I have undergradutae thesis left) and 1 year as a Linux support technican with some basic knowledge of SQL and something little of Python, there is one company that wants junior data engineer, do you think that I have good chance to get that job? :)

[@karolinasowinska]

Definitely! Go for it! :) just make sure your CV matches the job description

[@rmp5640]

@@karolinasowinska Aww thank you so much for a fast response you are so kind! :) Sure my CV will match job description I just hope my interview will go well and that I wouldn't look stupid or inexperienced...

[@unnaipoloruvan98]

How many of u tried to clean the display!

[@karolinasowinska]

Haha why? Was my lens dirty?

[@unnaipoloruvan98]

@@karolinasowinska 😅 I thought i deleted this cmnt.... that was a mistake ya...Had a crack in my screen protector

[@unnaipoloruvan98]

@@karolinasowinska And ur doing super coooool Karol.....Plz ans me a query!.......im stucked between a dilemma of shall I choose information security career or data engineering!? What will u suggest....which is difficult to learn? And which is easy to get a job with good salary....which will be recession proof....

[@karolinasowinska]

@@unnaipoloruvan98 Hahah I see! :D

[@karolinasowinska]

@@unnaipoloruvan98 All great questions, in general no job is recession proof, and I'm biased towards data engineering :)

[@axlquispe]

you're funny... Subscribed.

[@karolinasowinska]

Haha I try to! Thanks :)

[@rasseliodiack7159]

Now it’s sure, the next video will be about 2k subs

[@karolinasowinska]

Who knows! :)

[@ichbrauchmehrkaffee5785]

My pea-sized brain misread the title as "How to date an engineer"

[@starmach87]

FIrst video i see from you but i dont know what is your problem with voice modifications lol

[@karolinasowinska]

Too much free time at hand to play around with Final Cut Pro effects!

[@medotop330]

Do you use apache spark or no ?

[@karolinasowinska]

I personally don't, but it is very popular in the data engineering world

[@ankushojha5089]

Hey, i have started watching your videos .. I have knowledge of all tech stacks as beginner/intermmedate level.. I want to apply in jr data engg , as i have around 5.5 yrs of exp in IT industries alond with working skills like python sql, kafka, ariflow ... if you can suggest some jr engg jobs it will be helpful to get any

[@karolinasowinska]

You seem to have all the necessary skills. Just apply to some job openings! :)

[@ankushojha5089]

@@karolinasowinska Yes, trying to get in.. facing little challenge while screening my resume .. i think because of resume interviews are not scheduling for tech round.. hope the day will come 🤞

[@sergeyovchinnik1444]

7:49 relatable

[@karolinasowinska]

Haha 😂

[@mulle171]

did your economics degree helps you in any way or was it a waste of time ?

[@karolinasowinska]

I've just released a new video where I answer your question! :)

[@mulle171]

@@karolinasowinska haha amazing :D i have the totally same story us you up to the CS degree.. i stuck with econ in the master and did some CS couses in Machine learnning and Database.. i hope to get with this a junior data poistion, becouse as you i have also made a data science internship and loved the creative side of it^^

[@user-ib9rs6qo7t]

Каролина, привет. Хочу стать ДИ. Че делать? Хэлп

[@karolinasowinska]

Hey, I don't speak Russian.

[@fruitfcker5351]

How a data engineer to become?

[@karolinasowinska]

Oops, it seems that I am a Mind F*cker

[@fruitfcker5351]

@@karolinasowinska I'm registering that name.

[@viduminisulochanaperera3553]

Noiceee

[@kakamoora7874]

Un pay starting 😢

[@sahivolg1513]

I am srilankan people..so can i work in us

[@cezarysnochowski]

lol, "no one uses linux"? XD Who does say that? ;) Overall nice video and channel :) keep it up 👍

[@karolinasowinska]

Haha some uninitiated folks! Thank you! ;)

[@nikos9082]

you win you confused me more

[@karolinasowinska]

What's the confusing bit? Maybe I can help to clarify here :)