AWS Athena Partitioning | Athena Tutorials | Athena Query Performance Improvement | AWS Tutorials

[@najmehforoozani]

Thanks Ajay. well explained

[@yogithakakarla1716]

Best video. Useful

[@viniciusoviedo]

Exactly what I need. Thank you for sharing!! 👏👏

[@AjayWadhara]

Glad you liked 👍👍

[@bonniesterling2478]

One of the best - can you cover BUCKETING in Athena and maybe BUCKETING vs. PARTITIONING in Athena - Great Job!

[@udayreddy3653]

Hi Ajay, Your explanation was Damn good! Can you please make a serious of vidoes on AWS Glue that would be helpfull

[@prafulmaka7710]

Bless you mate! This really helped :)

[@deboradangeloreinadearaujo2082]

Thank you! Really good content!

[@AjayWadhara]

This made my day 🚀

[@BczoffyouCh]

Thank you for your sharing, do you have the tutorial for data store in S3 as /year/month/date/hour? If so, please send me the link. Thank you again

[@AjayWadhara]

I am glad you liked it. Sorry, I don't have that tutorial. You can implement that very easily with Lambda. Lambda should be triggered on S3 object upload.

[@NehalVerma-zr4mq]

Thank you so much for this informative video. Can you please let me know that how can I do Hive-style partition and spark partitions here in aws

[@AjayWadhara]

Sorry.. i am not aware much about Spark partitions and would not be able to help you on this.

[@NehalVerma-zr4mq]

@@AjayWadhara No worries, thanks for letting me know. Can you just confirm that the demo that you showed in this tutorial comes in Hive-style partitioning type?

[@khattarjitender05]

First of thanks for sharing userful knowledge..Need to Know One thing I need to Move Data from RDS->DMS->S3->Query with Athena. Data is mainly date range, can DMS automatically create S3 partitions based on Year and Date... I know this thread is for athena.. but if u have experience please share that as well

[@AjayWadhara]

Hi Jatinder, i don’t have hands on experience with DMS but yeah DMS supports date based folder partitioning when using S3 as target. Also, try to use parquet if you want to query data from Athena. I hope this helps

[@SibasishPadhy]

Thanks for this video.

[@AjayWadhara]

Do subscribe and share... it motivates me to create more content 🎯✅

[@kumar3247]

Thank you for the video. Simple and easy to understand. It would be great if you can make video on Lamda funtion to create partition based on S3 folder.

[@AjayWadhara]

Thanks for the feedback Regarding partitioning with Lambda function, I will definitely cover that in coming week

[@AjayWadhara]

Hi, I have upload the video to create partition based on S3 folder...Do check in my uploaded videos

[@dhayas8027]

Hi bro thanks for the detailed video. Am new to AWS and learning. Just wnat to know do we need to manually create folders and place file in S3 for partitioning?

[@AjayWadhara]

Not Manually, you can write Lambda that does this for you or the process which is writing the files to S3 should do this.

[@dhayas8027]

@@AjayWadhara thanks for the clarification bro !

[@swaniketchowdhury]

Can I do partition while saving a view as a table? For instance, I'm joining 4 tables and doing some logic to build a view. I want to save the view as a table, can I use partitioning in that case?

[@jaysoni7812]

How did you change icon of windows 10 as xp in task bar? I also want like that.

[@rupeshpaneerselvam2958]

Hi Ajay thanks for the video and I have a use case of anayzing vpc flow logs using athena and glue crawler from data stored in s3 can you share your inputs to proceed on this use case..

[@AjayWadhara]

Hi Rupesh, you can follow the same strategy that I have demonstrated, only difference would be I am using CSV, you have to use a regex while parsing the log files.

[@vishalrajmane7649]

Can you provide the documentation for alter table command trigger in aws lambda

[@vishalrajmane7649]

Hi, This is Trupti.. the video is very useful but i have one que that What if ur s3 data is stored as non partitioned means it has mixed data in which there are multiple years so how we will do it??

[@AjayWadhara]

You have to write a process for the same. For new data, keep a staging bucket, trigger a lambda that partitions your data from staging to the main bucket