Why NOT to become a Data Engineer

[@TheVivaciousNerd]

I'm actually currently trying to move away from data science and into data engineering, which is apparently a rare path to take. Personally, I don't really love the exploratory nature of data science (though I used to think that would be awesome)... I'm just the kind of person who likes well-defined tasks/problems that have a real solution - so much of data science is just seeing what's there, and sometimes, there's nothing useful at all (explaining to the business why you just did two weeks of work and have nothing to show for it sucks). I also really don't care about applications of the theory or how the data supports business goals, I simply have no interest in business at all. So engineering really appeals to me - all the tech without the boring business things! Also, less talking and meetings lol :P

[@karolinasowinska]

That's fair enough, it's great that you're so self-aware about your own preferences! I hope that you'll enjoy data engineering, now is definitely a good moment to pivot (given the rising demand for DEs)!

[@TheVivaciousNerd]

@@karolinasowinska Thanks! It definitely feels like the right move for me, and your videos have made me feel more confident in that decision, qnd identify the skills I need to grow. Thanks so much for this content! ♥️

[@artemsokolov5007]

> likes well-defined tasks/problems that have a real solution > no interest in business at all is'n it a contradiction?

[@Musica3725]

Wow, many of those cons sounded like pros to me hahaha

[@karolinasowinska]

Haha I know right! ;)

[@sanchesrfl]

yeeees!!!!

[@datadrivendecisions0]

@@sanchesrfl it's a great industry!

[@charlesegbunike4596]

Exactly what I thought...😁

[@tonysamosa1717]

Same here. Let me code and build pipeline and I am happy

[@RemiGabillet]

You make such good points @Karolina. It's so true that by being in a support function, data engineers are often invisible unless things break. But the counterpoint is that the work is less rushed, more long-term and so much more satisfying. This makes for an awesome work-life balance.

[@karolinasowinska]

Definitely, like you said, there are always two sides of a coin! Being in the spotlight all the time basically means worse work life balance, being in the shadows means awesome work life balance :)

[@eralddavid6892]

Hi Remi! I work as an analyst at one of the decacorn at Southeast Asia, and I'm pretty sure that if you work at a company who has "move fast and break things" culture, there's no work-life balance in there :) Don't get me wrong, this is also the condition that suitable for you if you want to learn more. I agree with your point on "more long-term" and "satisfying", that's also why I envy data engineers and programmers because they can focus on the deep work. Just my 2cent.

[@wmgonzalbo]

As a data engineer/architect this is spot on! Some analogies on reason #1 - data is your plumbing, while your data engineers are your plumblers. You dont see the plumbing but you are thankful that you have clean water in the house. It may be taken for granted at times but any leaks or burst pipes is a big problem. Another analogy is in the body data is the blood. Its not a main organ but organs need it (clean blood) to function properly. On #3 true same with data science or ML where they now can be a black box provided as a ready to use product. But in the future someone still needs to know when and how to use these tools. Great video, more power!

[@karolinasowinska]

If you *do* want to pursue a career in data engineering, visit: dataappreciationsociety.com/ (also don't forget to switch on 4K for the best experience (I finally got a new camera!) haha)

[@scienceaddicted3431]

Done! Wow amazing quality!

[@severlight]

My display cannot do this(

[@medotop330]

No need 4k when we view karolina

[@strezz1560]

surely, the most important tip :D I can see every little wrinkle now! 😂

[@angel-fear5067]

Now we have four times more than just one Karolina.

[@tonysamosa1717]

3 minutes into the video: “as a data engineer you are not at the heart of the business you’re behind the curtains” That is music to my ears. I am an analyst and I don’t know what I got myself into. I HATE talking to clients and I hate talking business. I enjoy my job the most when I get to build automation tools for myself or others and feel that’s where I would keep my sanity; being ETL focused. I want to be a data engineer

[@karolinasowinska]

It's so good to be aware of you like and what you don't like! And it sounds like data engineering will be the perfect career for you - you can consider yourself lucky, given how in-demand those skills are now! ;)

[@tonysamosa1717]

@@karolinasowinska any advice on progressing other than reading and doing tutorials (just picked up designing data intensive applications last night). No much opportunity to get mentored i. My current role

[@ohdude6643]

Precisely like I think.

[@filipspasic3563]

exactlyyyyy, I am currently an analyst and I am here because I realized I much more enjoy building tools and pipelines than analysing and searching for insights :D

[@matheusrodrigues1563]

Its so nice to see that i'm not the only who thinks like this.

[@randomnessdiary]

Hey I just started my internship as an etl developer and while going through various articles online I found it comes under data engineering. And while research about that I found your profile and I'm so glad to have found you!

[@karolinasowinska]

And I'm so glad that you've found me! Welcome aboard! ;)

[@dorianzxc7996]

I can relate so much to this. I'm a data analyst being bribed so much to keep doing data engineering, I gave up on it and enjoying being a hot bun 🤑 btw part of my learning process was your ETL videos, so thank you! That's why I keep coming back to watch your channel!

[@karolinasowinska]

Haha you might as well! If you enjoy what you're doing ;)

[@datadrivendecisions0]

There are a few more examples where you can be closer to the business as a data engineer like myself! Companies where data is their main asset can be a huge one e.g. Opta who are the leading sports data provider, Nectar who collect data about customers to analyse their shopping habits and even Bloomberg/Reuters who have amassed so much financial data that they can sell it for tonnes of money! I'm also hoping technologies like Denodo and Databricks will just change the role of a data engineer instead of getting rid of it. Great video as always and the new 4K looks great :)

[@karolinasowinska]

Oh, YT put this comment in "held for review" for some reason! I completely agree, how crucial and "central" you are as a data engineer depends on what industry you're in. And I also hope that Denodo/Databricks will only make data engineers' lives easier! :)

[@IamDanish99]

I am a Data Engineer now. Lets see how far I can take the load. LOL. Thanks for the wonderful video Karolina.

[@nas4799]

The first reason sounds like an advantage to introverts.

[@karolinasowinska]

I thought so too!

[@scienceaddicted3431]

You know, I have realized few weeks ago, that my current job is actually Data engineering :)). It has a very weird name, but I actually build data pipelines in clinical trials, and I love it. I'm happy with my job, and I got here without knowing how fancy it will be in 2021 :D

[@karolinasowinska]

Haha nice one! Definitely, if you're building data pipelines then you're a genuine data engineer! ;) I'm happy that you enjoy it!

[@alfredama]

Data Engineering is where is at guys :) , we are the backbone of Machine Learning and AI

[@brendasofia2735]

As a data engineer student I have to say that your videos help a lot!!! There a lot of questions that you just easily answer. I hope one day I’ll be as talented as you.

[@jannedrumsit]

I am an introvert but I am in consulting. Talking to clients drains me big time. After 31, I am now looking into starting all over again and shift to data engineering.

[@karolinasowinska]

All the best, you can do it!

[@AnweshDash]

wishing you a belated Happy Birthday :) A fellow Data Engineer who is inspired from your videos :)

[@karolinasowinska]

Thank you so much! :) 😊

[@gauravarora5165]

Hey, Thank you for the video. At first i just wanted to stop the video when you said "these 3 reasons are just your thoughts". But later on as you speak and explained the unbiased scenarios 'giving the listener a chance to decide on their own knowing the + and - of the field', i loved connecting to this fact. and yes, i am also trying to switch to Data Engineer role from Business Analyst role and thats what pulled me to watch this video. Again thank you for honest and helpful thoughts and contributing to the community.

[@Diego0wnz]

I just got rejected from my data engineering internship, so this video came at a good time 😃😭

[@karolinasowinska]

I'm sorry to hear that. If you enjoy DE, don't give up! :)

[@angularsteve7544]

Always good to hear the cons of a career from someone who is in that career. In this case, some of the cons can be viewed as pros :)

[@karolinasowinska]

That was my impression as well!

[@jameshizon4861]

You had me at work-life balance is great. I like to do many things outside of career, so I am sticking to the field (currently 95% finished with my DE Bootcamp). I also think that one can go from Data Engineer to ML Engineer later on (if they have data science background already, but the job market is too competitive).

[@karolinasowinska]

That's awesome!

[@SanusiAdewale]

Exactly!!

[@Jon-dk4qu]

Love the quality of the camera, and thanks for the content!

[@pippo1734]

I thought I wanted to be a Data engineer. I mean, it sounds pretty cool, but I like more machine learning in my life. Now I know I have to be ML engineer to have a little of both. Very helpful!

[@paulifea7072]

Thank you for adding your perspective and experience to this. I have been presented a choice to work either as a Data Engineer or a Cloud Engineer and I'm still deciding which to go for. I have barely started my career as a cloud engineer, but I've build quite a bit of knowledge over the past 2 years to prepare myself for a cloud engineer role, only to be presented with the choice of becoming a data engineer now. Both seems attractive to me because I like to grow as a technical professional, I like the hands-on and practical aspect of both roles.

[@cidrisonly]

I landed on this video by searching "why it is difficult to comprehend Data Engineering".. Actually I'm a Data passionate person with a Data Science degree, on a career change, realized that Data Engineering was the core of data and many companies would have requirements for it in the future. But unlike Data Science I feel DE demands a lot of IT roots. As I'm from Finance background with business experience, getting overwhelmed with hundreds of jargons is sometimes annoying. But, if I think of what I can accomplish with DE, I keep moving. Thanks for this video. It has motivated me to continue.

[@Inconnituz]

I'm moving into a DE role from a Data Analyst role. I have a degree en economics, so I get what you're saying. You are not the only one struggling with this.

[@aureliusnt]

Greetings from Brazil, @Karolina. You are being completely honest. Your video is fully true.

[@karolinasowinska]

Thanks, I do try to!

[@jaymeelias]

Great Video. About the third reason, I myself work to a big retail chain in Brazil and they are migrating the ETL to DataBricks, this is been made by an entire data engeneer team

[@navarrodba]

Good points! As a DBA I'm totally in the shadows. Your video helped me a lot to find my way in a carreer change. Thanks

[@kenrysanchez898]

The point regarding Machine learning... it depends. Today we have MlOps and AutoMl Tools. I'm pretty sure these things are starting to be more related to data engineers than data scientist.

[@arifluthfi1174]

Saying data engineering will be irrelevant because of automation is like saying web dev is good for nothing because there is web builder. Anyway, love the video!

[@karolinasowinska]

Hah well, true, I guess!

[@stjepan_8902]

I actually think it's quite nice to be 'behind curtains' , knowing that you're indispensible. Just like so many people who work on designing, building and maintaining physical infrastructure we use every day. There are a lot of jobs this pandemic has highlighted as crucial for our normal functioning. These workers were often underpayed immigrants, very vunerable to pandemic. So, I see this as much as the consequence of our ignorace and lack of empathy for our fellow workers. THAT has to change.

[@anzonyquisperojas4800]

Thank you for your time and video. I would appreciate a video of resources to learn data engineer. I love your videos. Thank you for your help.

[@karolinasowinska]

Great idea, I'll think about it! ;)

[@victorpinasarnault9135]

Thank you for that. I'll watch it again later. Liked, subscribed and activated the bell.

[@karolinasowinska]

My pleasure! ;)

[@LilisNotes]

I can't agree more! Thank you for sharing your experience so openly!

[@karolinasowinska]

I'm glad that this is relatable! :)

[@SciHeartJourney]

I took a course in SQL and did well, but I don't want to work with that tool again. I want to apply machine learning to robotic controls. She makes it very clear to me that I do NOT want to be a Data Engineer working for a bank or finance company. I want to continue working in the aerospace industry applying machine learning algorithms to flight controls, or robotic controls.

[@karolinasowinska]

I'm sure there is a demand for those kinds of roles. If there's a will, there's a way, so good luck with getting your place there!

[@SciHeartJourney]

@@karolinasowinska about 20 years ago I heard about this technology called "sub vocal communication". They used it to help divers communicate underwater. They placed a transducer over the vocal cords to pick up the nerve impulses from normal speech. The system had to be trained so that they could connect the electrical pattern with known words. Very cool stuff BEFORE the term machine learning was widely used. I wanted to use it to help someone like Dr. Hawking recover speech and have some mobility. I feel this should be my life's work!

[@arjaleman]

In you opinion how will the demand for data scientist positions be in the future? There're also plenty of platforms that aim to reduce or simplify the data science tasks. What career between data science and data engineering will have the best future?

[@karolinasowinska]

Like I mentioned in the video, no job guarantees the security that it'll be in-demand for decades. I wouldn't think about it too much. Go for what you enjoy doing most, and new opportunities will emerge for your skillset in the future. :)

[@user-hr4me7ei3v]

Your cons are totally pros for me) thanks

[@karolinasowinska]

You're welcome:)

[@MrThedukekid]

love it and yes we do use hot bun saying, also the unification of the data sources would be great cos now everything is going to cloud so you just have to adapt.. data cloud engineer.. move from ETL to ELT.

[@mental_suicide]

hey Karolina you look like a smart person thanks to your honesty

[@rafaaferid1789]

“Reason number two: it has nothing to do with machine learning “. That’s exactly why I like it 😂

[@slavokeruslavokeru7936]

hey, yr speak is very nice. my russian ears like yu) I am data engineer, but I`ve wanted be ds. BUT my job is better and easier and very interesting than I worked Data Scientist.

[@samparkpradhan8849]

Probably the 4th reason is- the job becomes monotonous after 3 months ( if you know airflow, pyspark and snowflake). Typically you would do the same kinda transformation each time. But in ML you keep learning coz every month there's something new coming up. (take BERT and GAN for examples- they were like state of the art last year but this year there are so many improvements).

[@karolinasowinska]

Good point! Although I'd say that there are new data engineering technologies coming out as well, or systems ar migrated to different environments like AWS. Obviously depends on the company you work for! If the company doesn't innovate, then you're right, writing ETL processes can get monotonous

[@samparkpradhan8849]

@@karolinasowinska - Agreed. But what I am saying if you see the sheer variety of work in ML(let's say an anomaly detection, ranking algorithm, customer retention etc) is also more(which makes it more interesting and challenging) when compared to data engineering work. Great video by the way!!!!! I hope you reach 100K subscribers soon :) :)

[@karolinasowinska]

@@samparkpradhan8849 I see what you mean, you're definitely right! :) And thank you! :)

[@0x007A]

If you are doing the same transformations across multiple projects, automate the process(es) and let the computer(s) do the work.

[@chiemekaezerie4931]

I feel the Data Engineering role can also be quite diverse, depending on the company strategy and/or needs. So it's actually very possible to find a DE, that gets to do ML maybe in a startup. So it might not be a monotonous role, it depends. There are also alot of Big Data tools out there aswell. For a Data Engineer who wants to self-improve by learning all these tools will have alot to learn

[@emz8949]

Hey. This is a top notch video. I'm about of start off my career as a data engineer. I agree with the first point you make. I wanna become a Product manager in the long run. Do data engineer's transition from data to product?

[@karolinasowinska]

Yes. But being a general software engineer is probably an even more direct route.

[@darkoneontheisland9362]

great video, definitely didn't discourage me

[@softyo]

just discovered your channel!!!! Thx for the great video!!!!! Really insightful and helpful

[@enriko3694]

Do you think that some data engineers would benefit greatly from knowing machine learning because they could be building a platform for machine learning to be able to be done very efficiently?

[@karolinasowinska]

True. Although often MLs and DEs collaborate in teams, so that knowledge shouldn't be necessary. Definitely useful though, especially at start ups.

[@ahmeddawood6176]

The video is quite informative I suggest you change the video title because it is kina misleading, but thanks for the information :)

[@yrysf777]

Thanks for this informative video. I am a 4th year University student and I am currently struggling to decide between Data Engineering and Hadoop Developer career path.

[@maximpobihun5469]

I think in products or solutions that are heavily relied on data or are so called "data driven", data engineer role is main business function.

[@karolinasowinska]

True! (Unless the firm only advertises itself as data driven)

[@ninan23]

I’m trying to get into data engineering. But by watching your video, now I’m in a doubt. I’m from management background. And I have zero knowledge about IT field. I have a couple of questions to you. Can anyone from non IT field, will able to do data engineering job or not? And, Why is it so hard to find data engineers compare to data science? Thank you,

[@updateserver]

From my research a lot of employers interchange a data science title for data engineering tasks. To get familiar with tech, go through the CompTia A+ concepts to get a foundation on general hardware/software/networking etc … then look for Data Engineering courses. Coursera has one. Hope this helps you out, good luck😄

[@Crazy8xxx]

No 1. Absolutely right. You’re not making architectural decisions. That’s not the job.

[@chandanparida4278]

Thanks for the video.i need more video about data engineer.love ❤️❤️ from india

[@JayPatel-yf7ot]

Hello All, I am Jay, a Data enthusiast. I have a good knowledge and portfolio as a data analyst but I am concerned about the competition in this job role as diversed field of professionals are transitioning into it. For this reason I am trying to consider data engineering path as a career. I am currently in the canada and if someone experienced can give me insights about future prospects of data engineering then it would be very helplul.

[@karolinasowinska]

Data Engineering is a really good one. Jay, do check out my other videos on the subject. I am also working on a dedicated course on how to launch a career in data. If you'd like me to send you an alert when it goes live, please share your e-mail address! All the best, Karolina

[@vipinamar8323]

I am a data engineer and really like the content here, most of the points were correct. Except for the fact that DataBricks or any other ETL automation platform can only have as much impact on Data Engineering as the up-and-coming AutoML will on Data Science. So that's not a reason to not become a DE and pursue DS.

[@sahil11o9]

What's your LinkedIn Vipin?

[@dantemartinez3609]

Great video with awesome advice. Do have recommendation for ETL tools for some just getting into this. Thanks in advance!

[@karolinasowinska]

Hey, I have a data engineering course for beginners on my channel :)

[@Gaer56]

Wow, this really made me more confused about my career choices. Maybe I just like making stuff for myself.

[@wayneomondiomoga1722]

Where can I get entry level jobs for data engineering working remotely ?

[@salmantabatabai]

Brilliant video and I'm seeing it at the right time. I've been a researcher and a data scientist for many years and now they are offering me to move towards data engineering. Something that I have no clue about. So your video was quite eye opening. I think I should probably reject the offer? What do you think?

[@karolinasowinska]

If you actually enjoy data science side of things, then don't do it! ;)

[@salmantabatabai]

Thanks Karolina! Really appreciate to know your idea.

[@malcorub]

The big lobsters sounds better than the head cheese, I am going to start using that one.

[@sprintwithcarlos]

I really liked your "hot bun" comment :-D Thanks to your videos I discovered that data engineering actually is pretty awesome, you're completely right

[@karolinasowinska]

Haha I discovered that English say "hotcakes" instead, I was close! :D

[@carlosemiralonso7997]

IS just me or she doesnt known how to not sell this job? She just love the job so much that it is noticeable. She listed more pro's than cons and the cons (as many already mention) doesnt seem like that at all. Love the energy of this video, i totally vibes this way :D

[@MrBean350000VR]

Hi Karolina, I'm a university student in the US studying data science and I'm a huge fan of your videos! Given the high average salary of data engineers, I don't assume it is a job a new college graduate can get. What would you recommend for a prospective data engineer's first job out of college?

[@karolinasowinska]

You can do it straight out of college! And if not, I suggest general software enegineering :)

[@user-ky9qn4pg3w]

Data engineers are the IT of the data world.

[@danzilbenjamin8196]

Thanks for updating. Just wanted to know is there any age barrier if one wants to enter data industry. Thanks in advance. I have done UG in Biotechnology and master in Food science and no experience.

[@karolinasowinska]

Of course not! Go for what you love, never put limitations on yourself! :)

[@imelbertkim]

Interesting. As a data analyst, I was aiming towards becoming a data scientist but what you explained broadens my horizon. On a side lobster note, are you reading (or planning to read) Beyond Order?

[@karolinasowinska]

I might order it. Although I haven't finished his first book yet, I couldn't get through very descriptive analogies and stories. But I appreciate the guy's mind.

[@arielguzman2875]

Haha 😂 you really made the cons sounds like pros!

[@manu93ize]

Great video 👍 I have all the basic skills to qualify for entry level data analytics job so far unsuccessful to land a job. Do you have any suggestions? Thank you

[@0x007A]

Do you have a portfolio of personal projects which demonstrates your technical and communication skills?

[@karolinasowinska]

Good suggestion above!

[@TheWongAndOnly910]

Really enjoyed this channel for learning about data jobs. I career changed from being a traditional scientist into being a DE which I discovered by accident. Going from a field which is, allegedly, oversaturated into a field in demand has been quite the shock. I personally have no interest in DS although would you consider creating a mini ML course for people interested in it? On the subject of DS, I think if DE and DS had the same job prospects, a lot of people would prefer to be a DS. What do you think draws people to DS over DE? Love the channel, keep it up.

[@karolinasowinska]

Actually a great idea, I can definitely think about a short ML course! :)

[@TheVivaciousNerd]

My theory on why DS is more popular is that most people want to make discoveries / draw conclusions rather than simply building the infrastructure that lets other people do that; they wanna analyze and build models and find answers - it just sounds more exciting! The thing is you're still solving problems and using logic when doing DE, its just not as "on the surface" if that makes sense.

[@anyasyht]

haha that cieple buleczki analogy-love it!

[@karolinasowinska]

Haha ciepłe bułeczki, dokładnie one chodziły mi po głowie! ;D

[@mr_braimah8966]

Another great video. Thank you.

[@karolinasowinska]

Thanks! :)

[@korableg]

Karolina you have a good pronunciation. Nice to learn English. :)

[@karolinasowinska]

Thanks!

[@updateserver]

Hey, I was just curious about the live coding interaction within job interviews. What are some coding challenges(python,sql)/ data structures & algorithms questions you faced in interviews ? How did you approach them ? I’m a bit nervous about this portion😅.I’ll be looking for a junior DE role soon. Thanks for the great content btw😄

[@dhv7673]

Did many DE interviews and was part of interviewing Jr DE candidates too. Here are 5 tips: 1. When you're asked to write a code/query to answer a question or to complete a task, you first want to explain what your approach would be: What functions will you use? What lines of code are you considering to write? Why? What do you think the output will be? Think out loud before typing anything. That way, even if you fail to complete the task or to completely get the desired answer, you're still showing that you understood what you were asked to do and that you knew how to get the desired output. +Most of the time, the interviewer will guide you as you are brainstorming too which can be very helpful. 2. Don't be afraid to Google stuff when asked to write a query or a code. It's better than spending a long time guessing. You're not expected to memorize coding lines or functions. For example, if you know you need to use a window function then say, "I will need to write a window function here", then google how to properly write it. What matters in this example, is that you know what we had to use to get the right answer. 3. Don't guess different methods to write your code until the interviewer confirms which one is right to use. Instead, explain the method you would use to get the answer or to complete the task then try it. If it's right, great. If the interviewer says it's wrong or your code shows it's wrong, don't be embarrassed. Instead think of another approach you can try, say it out loud, and try again. We all learn and perfect codes through trial and error. So, yes of course, the interview would love the right answer on the first try. But if you're not getting the right answer the first time, it's your chance to show your problem-solving skills and your tenacity. Show that you're not afraid to make mistakes, to troubleshoot, and to keep trying until you figure it out. 4. If you really don't know how to write the code/query the interviewer is asking for, then be honest about it. You can say, "I've never done this before", or, "I'm not familiar with this". But don't stop there. Always follow with, "But I can try to figure it out". The interviewer may move on to another question or they may give you a chance to try. If you can try, I would suggest to Google what you're asked to do. You can try to complete the task with instructions you find online or if it's too hard for you, at least share what you understand from the instructions. This shows you're willing to learn and try unfamiliar tasks. 5. Be friendly and approachable during the interview. Even when you nailed all the questions, if another candidate also nailed all or most of the questions and on top of that, they were friendlier, the other candidate will be picked. Even during technical interviews, we look at the personality. No one wants to work with cold people or hire someone the team may not be comfortable with. Most importantly, every interviewer is asked to find a candidate that's a good fit for the company culture. Good luck!

[@javid4288]

Some companies never use ML so they don't need DS or ML Eng. what they do, they prepare data for Analytics or just BI dashboards, sales funnels, and so on. How about that?

[@karolinasowinska]

Even if companies dont use ML, they still at least need someone to take care of the data infrastructure,.ie. data engineers.

[@barajasfelipe16]

Any Certifications that you would recommend someone who would like to get a job as a Entry Level Data Engineer?

[@karolinasowinska]

I recommend the courses I mentioned in the description:)

[@abhishekkumarpandey1968]

This video is not making much sense to me, because this video assumes that anyone and everyone can become a data engineer for the industry: 1. Not at the heart of the business. But, at the brain of the business. I am a data engineer, not because the CEO of the company is shaking hands with me every day. I am a data engineer, only because it is paying well. 2. Machine Learning or Data Science: Does not matter to me, as long as I am paid well. Your concern is correct, but is a real concern for those who are not working as a Data Engineer for the industry. Again, not a random person can be deployed at such highly technical positions, just because they have heard such terms in the industry. No skills. No jobs. 3. The third point is incorrect. I have to clean the data that I have to feed into the model, using my fingers+brain+skills. It cannot be automated. Period. The tools invented by such companies is a regular output of the employees employed by such people. Nothing ground breaking. >Downvoted.

[@alexisaltamirano3461]

Hi Karolina. What do you thing about the people that haven't computer science background want to begin as DE? In my case, I'm a chemical engineer but, I'm really tired of my job and I need a change.

[@karolinasowinska]

I know plenty of people from STEM (not just computer science) background who are data engineers :) You'll be fine!

[@alexisaltamirano3461]

​ @Karolina Sowinska Thanks for the inspiration! I hope we will be colleagues soon.

[@TheWheatless]

I wonder how stable the demand for these careers is. By the time I learn enough marketable data engineering skills in a few years, will the field be as “saturated” as data science allegedly is? A bit scary to think about. I know you mention this on the scale of one’s entire lifelong career. But it feels like data science went from highly in-demand to oversaturated in only a few years, never mind a few decades.

[@lauren185]

Data in general isn't going anywhere, its growing. If you know SQL even at least and have good problem solving skills, you're sorted you can easily get another position within the field. Careers are fluid sometimes, it can be exciting.

[@gauravk4050]

The really con part is to maintain that pipeline. Maintaining it sometimes on weekend😢

[@karolinasowinska]

Ouch :( I feel that :(

[@moderngames8892]

What a beauty❤️ ? Beauty & brain. What a deadly combo😘

[@karolinasowinska]

Ohh, how very flattering :>

[@_dongato]

2:14 *(1)* You are behind the curtain, not at business core (in majority of cases) 5:57 *(2)* This work has nothing to do with data science 10:03 *(3)* In the future (medium term) there will be fewer job offers. Today all jobs in technology are transformed.

[@marcioandre6469]

It would be awesome to see some videos doing data engineer things

[@victorpinasarnault9135]

I love tech, but I like Machine Learning and stathistics as well.

[@karolinasowinska]

The good news is that in order to do Machine Learning, you obviously need to work with tech;)

[@victorpinasarnault9135]

I Love it

[@RakeshG619]

make a video on hadoop,hive,hbase,pig,mapreduce ETL full project ----for DE

[@karolinasowinska]

I'll see, thanks for the suggestion :)

[@ronalerquinigoagurto555]

It is like devops, no one cares of you until something gets wrong

[@alessandroardu1059]

The second point makes me think. Data engineering seems to be literally traditional programming to built information architectures, like pure developers'stuff. And I don't like this at all.... However, I would like to do data science but it's hard :(

[@user-xx6wh5nm4s]

Hi, Karolina! Thanx for the great videos) Can you pls tell us something about that cases, when you are into DS on your job, but your spouse is not into IT at all?) Does it's good or bad... any comments

[@karolinasowinska]

I'd say, it really shouldn't matter for you what your spouse is into!

[@Neuroszima]

11:45 - interesting thought, however once a given person earn big amount of money and can't progress further, with job as a whole - as a position - will going to be less and less wanted, then it isn't difficult to imagine someone wouldn't want to switch their career path to completly unrelated field. I couldn't imagine a Data Engineer moving into a spot of Frontend developer. Furthermore, i bet it would be a hilarious interview. Ofc this one is a bit less propable but even backend, or some weird mix of different things, not even connected to web dev. This is something that is comparable to losing half of the skills build over the years in a blink. Coding is coding, but the problem solving skills that were data-specific won't be of any use anymore, unless you somehow move into data science, but again i'm not eperienced so i don't even know if those two have something in common.

[@gabrielveloz3296]

this background music! super groovy.

[@karolinasowinska]

Ha, thanks! ;)

[@MomoDuality]

9:33 But wouldn't a data engineer that transitions into being a data scientist make for a better data scientist since they have more of a technical skillset/mindset. Like I think its easier to go from being a data engineer to a data scientist (Assuming you have the speaking skills/personality for it) than it is to go from being a data scientist to being a data engineer. I'd love some feedback. Right now I am in a bootcamp to become a data engineer. In the future I would like to work more on the business side, sort of like being a consultant.

[@freema9246]

Thank you. Very informative. Dou you need a degree or certification

[@nicolasrosso3800]

Wow. Being in the shadows, work-life balance and good salary. Sign me up!

[@nathannguyen2041]

Shii... I like being in the background.. should start studying to become a data engineer, as an analyst.

[@dmitriyobidin6049]

5:55 I don't get it, why it is bad to pursue a career just cause you need money. Isn't that what 80% of the population do? And what would be a better choice? I think in software engineering it would be harder to earn a big buck when you are not interested in technologies. And about work-life balance: i've heard totally opposite. That data engineers are used to work on saturdays and sundays, just cause business users often need access to the data on monday. Is this true?

[@karolinasowinska]

Because if the career you picked doesn't utilise your strengths and talents, you'll find it hard to progress, and other, more talented people will outperform you. So it's a better strategy to pursue what you're naturally good at - the money will surely come.

[@statisticianwordnpl4071]

Is it correct say that data analyst are more involved into business than data engineer? At the end a D. E is a developer in data field, right?

[@karolinasowinska]

Yep that's very correct!

[@firewalldeca2327]

Okay my mmmmmmmnmmmmm m mm m m m m m

[@hasune9172]

I'm thinking about studing data engineering but I was never intrested in technology before. It looks compicated too but I need to choose something (I don't really know what am I intrested in). Do you thing, if I will work hard, I can grow to like data engineering and be succesful at it?

[@Tiffanyii]

Thanks the vid is soo informative

[@PureGonzo]

For someone with a Bachelor in Industrial Engineering that wants to expand to technology, which path do you think it would be better? Data Science, Data Analytics, ML, SE, Data Eng...?? Meanwhile, I'm learning Python and SQL. Looks like they fit in any of those. :-) Cheers!

[@karolinasowinska]

I'd go for an engineering role, so data engineering or software engineering in general ;)

[@0712prasad]

Hi @karolina sowinska Do we need to have a compulsory professional certification for entering data Engineer domain as a fresher or can we do it with courses and certificates from Udemy and all?

[@ankushojha5089]

this time , I am little affried and confused about my path choice ... In India it is difficult to get jobs from one to another ...

[@karolinasowinska]

What job were you interested in?

[@ankushojha5089]

@@karolinasowinska I definatly looking for long term career & good stability, that's y i thought Data engg will be good option..

[@karolinasowinska]

@@ankushojha5089 then go for it! ;)

[@user-yz3cp9id2u]

I think I don't give a damn about business at all, I just love coding anything, apparently that's why I work as a data engineer😁

[@lifeisanexperiment4102]

How to change our job from data engineering to data science since i am very much interested in machine learning. What are the companies and roles for machine learning enthusiast.

[@karolinasowinska]

They can be found in almost any sector!