Dynamic Partition Pruning in Apache Spark

[@EugenePetrash]

Genious explanation. Not only on that certain topic, but all of the author's videos and articles are also totally clear. Thanks a lot. Subscribed!

[@skywalker66ful]

Best Explanation I have found till date about Dynamic Partition Pruning and infact about Predicate Pushdown and Partition Pruning as well

[@anikethdeshpande8336]

super explanation! simple to understand, thanks for showing the execution plans!

[@andre__luiz__]

Amazing explanation!!!

[@akshaygupta013]

Nice explanation. I do have a doubts what's the need for broadcast if the filter condition is already being applied to dimensions table and if it is required than tables which are greater than broadcast threshold in those case will this technique not work or just join type will be different.

[@feelings__flicks]

Same doubt brother. If u get the answer can u please share it.

[@mohammedsafiahmed1639]

from what I understand, filter condition, or predicate pushdown as Databricks calls it, works only when querying single table. When you join two tables, you need to 'broadcast' the filter to the other table being joined.

[@artemvolkov5682]

What if I just add year and month to the 'ON' statement ? I believe partition pruning will work, but AQE should be enabled.

[@ylchen5975]

Very useful and expiation is pretty clear, thank you!

[@babyscookbook2751]

Hi sir., can we use apache kafka for sending emails? Please sir, I need it help

[@hierfnhg]

Very informative thanks for deep diving.

[@lancequin5209]

Him: Make Sense? Me: Nope Him: Great

[@artemvolkov5682]

hahah, feel exactly the same

[@bhomiktakhar8226]

Nicely explained !....but how does the filter is transferred to order table..since where condition is on year and month , query on fact table would still have to figure out what is full_date (of dimension table)column values on for 2021 Feb...could be multiple full_dates for month,year .?

[@kolketzz]

probably it would do date like '2021-02%'

[@octo3010]

Neat feature

[@trainsam22]

Hi Prashant, you know your concepts. but stop saying : makes sense.. or simple etc.. that is too uncle like.. b cool