Thanks for your comment and for watching the video! Being a test automation engineer and already completing a data analyst bootcamp are fantastic steps toward becoming a data engineer. Here’s what I’d suggest: Leveraging Current Skills: As an automation engineer, you already have experience with scripting, debugging, and working with systems. Use those skills to start learning Python and SQL. Foundations: Since you’ve done a data analyst bootcamp, build on that knowledge by diving into: SQL: Master joins, subqueries, CTEs, and performance optimization. Data Modeling: Learn how to structure data for pipelines and reporting. ETL/ELT: Get hands-on experience with tools like Apache Airflow, dbt, or Azure Data Factory. Explore Cloud Platforms: Data engineers often work with cloud providers like AWS, Azure, or GCP. Start with foundational services like S3 (AWS) or Blob Storage (Azure) and gradually move into things like data pipelines and warehousing (e.g., Snowflake, BigQuery). Build Small Projects: For example, automate a data pipeline that pulls, cleans, and stores data from an API. Network and Upskill: Keep connecting with others in the field, (Like you are doing with me) , and consider either the AWS or Azure Data Engineer Certifications. You’re already on the right path! Keep learning, experimenting, and challenging yourself, and you’ll be well on your way to senior roles. Let me know if there’s anything specific I can help with-happy to be part of your journey!

If you have a good foundation with math and some coding then SQL should be easy to pick up. I'd start there and then pick up a visualization tool like power bi and a ETL tool. That way if you find a analysis role, you can go for that.

I'm glad you had a good time! We can probably do a whole video just on all the marketing analytics and data! I bet that would be helpful and interesting!!

That's awesome!! Congratulations!!!

on campus or off campus? i wish to start as a data engineer too as a fresher, any tips & guidance? detailled answer would be very helpful

@@argus3354 Hey, let me tell you the current situation. I applied for more than 500 openings in Data engineer, analyst and Business Intelligence role. Got calls from 4 company, gave interview in 2 of them, got an offer from one of them as a Data Engineer. Just be very very good with Spark, both theory and practical pyspark questions. Advanced SQL is must, you should be comfortable with writing window functions and joins, Cte, etc. Basic DSA questions in python is must. Focus on list and dictionary. Basic algorithms too. And just apply like hell. create a good one page resume, 2 3 good projects with end to end architecture. And that's it. Cold mail worked for me. Keep applying 😉

I agree with Aditya. If you want to flow directly into a data engineer role you will need advanced SQL skills, some pyspark as well as the basics about data modeling as well as the soft skills to be able to communicate that in a way that is easy to understand to a non technical audience. I'd throw in some cloud skills either with Azure or AWS. I'd recommend sending linkedin messages to the hiring manager or recruiter after you apply to share your linkedin profile and convey how excited you are about applying. I hope that helps!

@@GambillDataEngineering that's how it works 🙌🏻❣️

@RomanShchurko I love it! Absolutely Select * is not only poor practice but can be an efficiency killer! And, IDE is an important decision early on as it can make or break some early habits! DBeaver is a great one! I've also used Toad a bit! I'm so glad you enjoyed the video and thank you for sharing!!

Hi Durong - Unfortunately I don't know a ton about Crypto, beyond how to pull this data from coin gecko!

AWS is really good. However, there is a point at which it is time to move to the cloud vs stay on prem. There are use cases that you have hit a certain point of scale that it is time to move to AWS or Azure and, from a data perspective and there is a time that it is just as good to stay on prem.

@@akashbharadwaj1 absolutely! I'll add it to the schedule!!

@Fredmaxwelline Thank you for watching! Glad you found it helpful!

@@GambillDataEngineering Yeah Your videos should be watched by a lot of people out there, What strategies are you trying to get your channel viral if I may ask?

😄 🤣 😂 😆

I think we'll see some changes in the next few years. Focus on learning the tools and techniques that allow you to leverage and control AI. The next video I just finished and will be publishing for monday is a example of using Chat GPT to brainstorm how to optimize a SQL query!

Hello sir, I am new in the channel, I want to ask if there a roadmap to break into Data Engineering feild and thanks alot for these informative videos

Absolutely! I did this video on the data engineering roadmap! kzbin.info/www/bejne/aXeUe2yCht55lZI

@@abdooraahmaanx I also have a more granular version on my website here: www.gambilldataengineering.com/data-engineering-honeycomb

That is a great answer. My feedback would be to change push to prioritize. Wording matters in your interviews and this change will help! Find a way to make this more concise something like: I prioritize my workload by identifying which tasks have the biggest impact on the company. Then identifying which pipelines or processes are "mission critical" and can cause a business outage if they go down, finally the tasks and pipelines that have the least impact I push towards the end of the priority list.

Good answer. To enhance it: I love working in hybrid and remote settings. I find that using tools like teams and slack are great ways to stay connected when you are not in person. In addition, I keep my leaders in the loop as needed. Be careful about the "always let my supervisory know" part. It can come off as dependent or unable to take ownership.

This is a great answer! Any time you can create a picture it will help, having an analogy about something relatable enhances understanding as well!

Good answer. Be sure to have some examples prepared as well to show you have a solid grasp on joins.

Good Luck!! I will be keeping you in my thoughts! Let me know if you get it!!

@@GambillDataEngineering Howdy! I i flunked the technical interview, My python skills are not the best. I have to brush up on how to open python and create a database and upload it to a server and use SQL with in Python. I nailed the SQL portions, I made great joins, I nailed the theory portion, I nailed the Critical thinking portion, I nailed the Normalization questions. you feedback helped me ALOT, especially on saying " I like to take ownership" Im someone who never had a B.S in computer science, So im learning on my own, Ive loved your videos so far! Im very new to the Data engineer world, so there is so much to learn.

I'm glad you did well on the rest and that the feedback helped! I hate that you had a hard time with the python part! You will get there. I have some videos on python if you want to check those out and if there is something specific you need let me know and I'll put something together for you!

That is a great question! Watch out for a video I'm working on exactly that topic!

@humsterburger thank you! I'm glad you like them! Let me know if there is anything specific you would like to see covered!

I'm glad you found the channel-welcome! When it comes to Microsoft’s business intelligence tools, I agree that they’re definitely legacy and are gradually fading out. However, they still hold a significant market share in reporting and ETL processes. Many companies haven’t fully transitioned yet, so a lot of your future work might involve migrating tools like SSRS and SSIS into modern platforms like Databricks or Azure Data Factory. Because of that, MSBI still plays an important role in the data ecosystem. But I’d love to hear your perspective-do you see it sticking around in any capacity, or do you think it’s on its way out entirely? Let’s debate!

@@GambillDataEngineering Thank you very much for your feedback! I agree that a significant portion of the market still relies on Microsoft BI and will need professionals to manage migrations to cloud-based solutions like Azure Data Factory. On the other hand, while most people seem to believe the cloud represents the future, I know some corporate owners still prefer to keep their data on-premises. For instance, I once worked with a retail client whose CTO believed the cloud was not the future. Now, I find myself wondering: could substantial hardware improvements in the future make cloud migration unnecessary? While this idea may seem far-fetched, history has shown that being right too early is not always advantageous. Thanks to our discussion, I plan to study how to transition from MSBI to Data Factory and analyze the feasibility of companies remaining on-premises. This exploration will also give me an opportunity to write my first article on the subject.

I do believe continued hardware improvements and mid- to small business may be able to successfully stay on prem. It really depends on the use cases and the business needs! There is still a ton that can be done on prem!!

Added Tamil to the captions!

You are correct. but the API wants the TLS encryption. In this specific case the Marvel API required TLS. :(

@parvavlogs3722 great answer you clearly have a good grasp on the mechanics of joins. My feedback would be to make your answer sound less clinical in your interview, and provide a specific reason / use case as to why you might want a Cartesian product.

@marquinsmith970 that is awesome 👌 . First, thank you for watching! Absolutely, I will start working on putting that together. Maybe I can review that on next Friday night's live session!

Great question, Vincent! While data structures and algorithms (DSA) are not as heavily emphasized for data engineering as they are for software engineering roles, having a solid grasp of the fundamentals is still important. For a data engineer, focus on concepts like: 1. Arrays, linked lists, and hashmaps for data organization. 2. Sorting and searching algorithms for optimizing queries. 3. Trees and graphs for understanding hierarchical and relational data. 4. Basics of time and space complexity for efficient data handling. Six months is definitely enough time if you plan and practice consistently. I'd recommend dedicating an hour or two daily, focusing on applying DSA knowledge to data engineering tasks, like optimizing SQL queries or improving data pipeline performance. You could also explore real-world examples to solidify your understanding. If you're new to DSA, platforms like LeetCode and GeeksforGeeks are good places to practice. Aim for consistency rather than intensity, and you'll see results. Good luck!

@@GambillDataEngineering Thank u sir 🙏

If you're just starting out and aiming to be a data engineer, Python is likely the better choice because of its versatility and relevance to the field. However, if you're already comfortable with Java or planning to work in roles where Java is heavily used, that could also be a great option. My suggestion: Choose the language you’re more comfortable with or the one more aligned with your career goals. Both are good, but for data engineering, Python often gives you a practical edge.

Thank you Kumar. I was wanting to share a story about the impacts of data engineering. In addition this will feed into my interview tip video coming tomorrow talking about using specific stats in your interviews!

Absolutely- shoot me an email with some details about where you are with your skills and what your current job is and I can build you a roadmap if you want.

@GambillDataEngineering actually sir I am an engineering student ☺

@GambillDataEngineering Btech 3rd year sir

@Vincent45678 - I'd make sure you are working on building a portfolio. Decide on a topic you are passionate about and then find a dataset. Build a full project from beginning to end from that point. Write a python script or create a pipeline set up transformations with dlt or within your script, set up orchastration of some kind (aka schedule your pipeline with a tool or you can schedule things in windows task scheduler even), model the data in your db by creating dimensions and facts, finally create a report in a visualization tool (power bi, looker, Tableau, etc.). I hope that helps! Also as you are preparing for interviews practice with a friend, mentor, or in front of a mirror. Record yourself answering questions and then critique your own answers. Day of interviews relax no more prep unless it is to remind yourself of syntax or some specific thing right before.

@Vincent45678 That is exciting! Let me know if there is anything you have issues with and I can try to put something together for you to help!

That is awesome. Data Engineering is something that I love. It is my wife's hell and my mother has no idea what I do but, I love it! 100% Communication and learning how to explain what you do to business stakeholders is paramount to having a successful career and move up the ladder! If you are focusing on a specific domain having domain knowledge will take you far too... especially in healthcare and finance.