Building Pipelines with Dynamic Tables

[@mauricioz]

DT are very useful to simplify pipelines and object management, support, monitoring, documentation and support.

[@snowflakedevelopers]

Totally agree!

[@TheStamperious]

When would I use a materialized view vs a dynamic table?

[@radhekishansoni8381]

I could not find anywhere in documentation, but is there any optimisation in create-replace of a dynamic table. Does it have any way to know that may be some partial data is already there. For example when we introduce a new column, it is highly likely that no other data was changed but does dynamic table consider this when it refreshes on create-replace.

[@snowflakedevelopers]

Thanks for your question! Create or replace for Dynamic Tables works just like a create or replace for any ordinary table, so the dynamic table would indeed get rebuilt.

[@vikihome6694]

Can I create the DT on top of external table ! looks it won't allow, is there any work around if in case I need to use external table ?

[@EdchelStephenNini]

Hi, How can we write sql unit tests for a whole dynamic table pipeline? That is, given a base table data, run it through the whole pipeline thru the test and verify if the results are as expected

[@ur8946]

where you define call start_data_gen(); and call send_alert_email(); function ?

[@luisfernandocardiel9059]

THE SAME ISSUE!!!

[@TanuE-lx6zw]

dynamic tables can be monitor through snowsight->activity->dynamic table I need to know which snowflake account usage table it is using to display dynamic tables details in snowsight? I need to get all the refresh history of all the tables, so how to get this details?

[@snowflakedevelopers]

Check this page out on monitoring dynamic table refresh history: docs.snowflake.com/en/user-guide/dynamic-tables-tasks-monitor#label-dynamic-tables-monitoring-sql

[@snowflakedevelopers]

Also, table_type = base table, is_dynamic= true is how a Dynamic Table shows up in account usage

[@roberbonox]

Hi, i create a dynamic table and have task that validates data on tables (standard tables) that are used in the dynamic table. The thing is when i finish the validations i need, i put an ALTER DYNAMIC TABLE IF EXISTS xxx REFRESH; inside the task, and when the task is executed the refresh is not starting. If i execute manually the command it works but is like it doesn't when is inside the task, is this a limitation right now??

[@EdchelStephenNini]

Hi, can we write dynamic table definition using python instead of SQL?

[@sharma_ji_in_uk]

we can use non dynamic tables in the dynamic sql?

[@parthasri3412]

Yes

[@LouisMonier-vc2fs]

Pretty awesome

[@snowflakedevelopers]

Glad you enjoyed!

[@VigyaanJyoti]

The demo example is pretty complex and the presentor should have chosen a simple example to explain things.

[@snowflakedevelopers]

Thank you for your feedback!

[@luisfernandocardiel9059]

where you define call start_data_gen(); and call send_alert_email(); function ?