104. CDC (change data capture) Resource in Azure Data Factory |

[@pramodkanwar3555]

Hi, Thanks for sharing CDC feature in ADF. it is very informative. I am also trying to implement CDC feature, but here my source is Oracle and destination Postgres. How could I enable the cdc in oracle and postgres table. Please suggest, It would be great help for us.

[@pianikalje2758]

Hi Maheer , Can you tell me what is the Major difference between Incremental data Loading and CDC and which 1 is better option in Real time scenario?

[@sushmithashetty9796]

Hi Maheer, your videos are really helping me to learn ADF ....in one of the interview i was asked how i would split a data based on whether its a new data to the table or needs to be updated with the existing data (upsert) or remaining are duplicate data getting added to the existing table...

[@WafaStudies]

Could u pls check my scd type2 video from azure data factory real time scenarios playlist to understand thism

[@khayyumhussain7586]

Thanks Maheer , We have Oracle golden gate for CDC from oracle..is there any options to integrate in ADF

[@bino8797]

Hi Maheer, Really appreciate your efforts and it was really helpful. I have a quick question - when i try to do source as a Blob Storage then am unable to delete the record if not available in the source file. appreciate your help on this

[@mohdsalman01]

Hello Sir, Thanks for the beautiful play series . Can you also make a video or play series for the Azure databricks AUTOLOADER feature .

[@WafaStudies]

Thank you so much 😊. I will try to explore that and do it. 😊

[@mohdsalman01]

@@WafaStudies thank you so much!

[@MrDuggu]

Hi Maheer, one important question i have. We have a requirement hide adf activities in higher environments or client environments while deploying so that clients should not be able to see the pipeline logic. Is there any way for this? Please respond...

[@vikrantsharma3859]

Thanks for sharing. Very well explained

[@srm6350]

Hi, How to maintain historical data in the Target system using CDC For example 3 record deleted but deleted record flag need to update as 0 or N Is it possible to implement?

[@HasanCatalgol]

Dataset is using 'AzureSqlDatabase' linked service with SQLVersion 'Recommended', which is not supported in data flow. Can you help with this?

[@jeffrypaulson]

@wafa can we create this cdc integration between azure postgres as source and azure sql as sink ? Please respond. Thanks

[@MrSatishc84]

Hi, What about the Throughput when every time CDC is executed ?

[@manoj_m0393]

Can i do CDC if my source is Sql server tables and sink is Datalake? if yes, how to capture each and every row CDC and sync these into parquet files present in datalake?

[@balajidevaraj7167]

Hi, I would like to delete data from source. But it's not reflecting in destination table. Could you help me on this

[@chintusingh4439]

How will CDC work with parquet files as its shown one of the sources in CDC?

[@shubhamgomase9056]

How to do cdc between those relational database which arr not available in azure cdc list

[@NeumsFor9]

Is this ADF's answer to Databricks auto-loader and can this be done with inline targets such as delta lake?

[@saiveeramalla5507]

Databricks autoloader has a limitation it expects the data to sit in ADLS (as far as databricks on azure) is concerned.

[@ShrikanthP-je4ki]

Great explanation !!

[@sigma_male_927]

Hello, I'm learning Azure data bricks and your videos are very helpful, do you have any courses for the same? So i can join and learn from you ..

[@raghudeep8873]

Sir Total how many videos will be there for ADF.........

[@viniciusandrade6916]

In my case , i am using SQL Sever IR Self hosted. Is there any approaching for that?

[@frankopokukoduah194]

This cdc is like scd type 1

[@didimoescobar2247]

In the video exist a PK from the source table and I suppose whith that, passed the parameter into expression @supports _net_changes = 1;

[@jeffrypaulson]

👌

[@JunaidBilal-n1q]

@WafaStuies , kindly make video before eating see what you did in this video at 21.30 mints