Sure

Glad you liked it 👍

Hi, thanks for asking the question. Redshift is columnar, so while storing data in storage blocks, it arranges all column values together. This is the actual physical storage. Distribution style determines which slice will get what portion of table data. If you chose distkey then each slice will get some portion of table data depending on the key value. Say 4 slices and 12 rows. So each slice may save 3 rows, but physical storage will have column values together. Distribution determines spreading rows among the slices and columnar determines actual way of storing data in blocks.

Yes it is very much possible. As soon as the file lands on s3 event is generated. This event can be used to trigger lambda function to run copy statement into redshift table. You can also create external table in redshift pointing to s3 path and as soon as file arrives you can see the data without loading as well.

Thank you for the comment. I'm glad you find it helpful. I will say a little bit of domain knowledge can help in tagging attributes to the entities. Many to many relationships are slightly tricky and need a bridge table to establish relationships. The attributes related to such relationship goes into bridge table.

Glad you find it helpful

Woo Hoo. Glad you liked it.

Woo Hoo 🥰

Hi Mushtaq This is an online course which I created sometime back. But this is not for absolute beginners. The course is intended for people who understand sql however find it difficult to solve sql problems. In this course, I have explained how you can use alternate approaches to solve sql queries. But if you are an absolute beginner who wants to learn SQL from scratch then this course does not cover it.

I do not provide any sql training from basic to advance level at the moment. However, I have heard good feedback for the @ankitbansal6 course and his youtube channel. You can check his website www.namastesql.com/

Hi Mushtaq, Good that you want to learn sql. I don't provide basic sql training but you can find link to few helpful resources in the description

Hi Hafiz, Good question. Zero ETL is a concept that is gaining popularity especially in aws ecosystem. In simple terms, it means that you connect to source directly and read the data at run time rather than bringing in data from source through ETL process. In redshift, you can connect to different data sources like RDS , dynamodb and read the data directly via sql query. You don't have to create etl pipeline to bring in data from these sources. Additionally, you can read data directly from data lake via spectrum query. Also if other source has data in redshift then you can read data directly via datashare from your redshift. In short, zero etl means the capability to read data directly from the source without the need to build etl pipelines. I strongly believe this concept will gain even more popularity in the future.

Woo Hoo ❤️ Glad you liked the video.

Sure

Sure. Any specific topic on redshift?

@@ETLSQL Firstly, Awesome content as usual..! Can try to cover topics under CDC options in Redshift, Orchestrate the data movement to redshift, how Redshift can be integrated with other AWS systems, data recovery options, About redshift specturm and any PROD use cases.

These are some good points. Noted.

Glad you find it helpful

Glad you find it inspirational

Glad you liked it.

Glad you liked it

Glad you liked it ❤️

Yep working on next set of videos

Thanks

Alright. Good luck. 👍

Yes you are right. We can.

Link is in the description. Sharing here for easy reference: etl-sql.com/sql-case-study-1-db-ticket/ Scroll down and click on download data button.

Alright 👍

It's a very good question and I also think about it. The question is applicable for not only beginners but also seasoned professionals. One thing is sure that due to gen AI, the productivity of developers will increase without any doubt. This includes all developers & not only data folks. For data engineers, I believe a new job responsibility will be added in the near future, where enterprises will prefer some experience with leveraging LLM models. Some existing tasks will be automated, and a few new tasks will be added. I don't think the role will be replaced by Gen AI , though some tasks may be automated using it.

Okay. Thank you. I don't miss any of your KZbin videos. They are highly helpful short and clear.

Wooo hooo. Thank you for such a lovely comment. Appreciate it 🙏

Hi, Can you share the error please.

Sure. It's on the list. Would you be interested in AWS data engineering series as well ?

More on SQL part as well because the way you are teaching SQL is very nice . Because you are the only channel on KZbin who tell us to write the query from maze method. I request you to please make some more videos on maze SQL query writing. Here I have one more example. List the department where at least two employees work Here at least means >= i.e count(*)>= 2 In this way we can easily write SQL query from problem statement. Once again thank you so much. Please create such type of more videos.

Sure. Please check this video too SQL Breakdown Structure, I am sure you will relate to it as well. kzbin.info/www/bejne/f4uxm5mjnqZqnpY

Glad you liked it. I used xmind app to create the graphics

glad you liked it. sure, added to the list.

Thank you Vibhav.

Please explain the steps with examples if possible

If you are looking for more details about the 3 strategies I will recommend following videos: SQL Pattern System (Part-1): kzbin.info/www/bejne/eXmXi3iXnJygY68 SQL Pattern System (Part-2): kzbin.info/www/bejne/m6vdqGBpmruiea8 The Maze Approach : kzbin.info/www/bejne/bafaqn6rjNSrZ80 SQL Breakdown Structure: kzbin.info/www/bejne/f4uxm5mjnqZqnpY Free Training: If you want to know more about the SQL Pattern Course , I will recommend signing up for free training : etlsql.kartra.com/page/sps-free-training

Fair point. I am working on it. Have try to fix it my new videos. Thanks for you comment 👍

Yes it can. But remember data is distributed using distkey only.

You're welcome!