SSIS Tutorial Part 59 | How to Incremental Load in SSIS Using Lookup and Insert & Update

[@roshnisingh7661]

Great video on incremental load.. Good to see other techniques as well

[@ssunitech6890]

Thanks.. uploaded other techniques as well..watch SSIS playlist..

[@pranaychonkar2037]

Great way to handle the incremental data insert and update the existing.👍 Just 2-3 days back, I've come accross this scenario where I've used SQL queries (with merge statement) to insert new and update the existing records. I had 40-50 tables and in each table atleast 30-35 columns. So to manage this I've used SQL queries but your solution is great too. 👍 Nice video keep it up. Keep posting such a great informative videos ❤️

[@ssunitech6890]

Thanks for your lovely appreciation ❤️😘 Keep watching and sharing your thoughts. Thanks

[@nallareddypavanreddy5625]

Thank you soo much Bhai you saved my day... Less time with more information!! 😊👍

[@ssunitech6890]

Thanks for your appreciation, Please share to others

[@oladman9058]

Very good video. Many thanks.

[@ssunitech6890]

Thanks for your appreciation Please share to others

[@jahangirkabir9271]

simple but super!

[@ssunitech6890]

Thanks Jahangir.. Share to your friends.. Thanks- SS Unitech

[@sayanbanerjee2722]

Very good vide Sirji

[@ssunitech6890]

Thanks 🙏

[@dharmenderchouhan8812]

Great video

[@ssunitech6890]

Thanks

[@amkytube5242]

keep going bro. It is nice video to understan

[@ssunitech6890]

Thanks brother. Can you please share the videos to others.? Thanks

[@amkytube5242]

@@ssunitech6890 my pleasure

[@trungtranthanh213]

tks

[@ssunitech6890]

Thanks 🙏

[@abhijith501]

Sir thanks a lot for the video. Lets say a record is deleted from the source table so how do we handle that using lookup we will be either inserting or updating based on EMPID.

[@ssunitech6890]

I have already recorded video on this and uploaded. Please check below link kzbin.info/www/bejne/g4qUfIlqp7-iiqs

[@darrylw99]

Ive assumed that the Update will update ALL records and not just the changed records. Am I correct?

[@ssunitech6890]

Yes

[@vishwajitd9844]

Hi Sir, Incremental load is better or CDC is better , when to use both of them , are they same or different?

[@ssunitech6890]

Hello Vishwajeet, Both are the incremental load techniques. Thanks.

[@vishwajitd9844]

@@ssunitech6890 thank you, I wanted to ask lookup should be used or cdc should be used , what's the difference?

[@kaushalsingh8000]

Thanks sir for this great video. All doubt gets cleared. Sir, Could you please upload 1 more video on sequence container with example like this?

[@ssunitech6890]

Thanks Kaushal, Sure will record a video on sequence container and upload soon . Stay tune .keep watching and please share to others.. Thanks

[@ssunitech6890]

Hey, Check out the video on sequence container. kzbin.info/www/bejne/eJatZKxpqZmNrsU Thanks

[@kirubalan-i2o]

Please put a video on : How do we add parameter for the ADO source, like using variables to filter the records. select * from table1 where date > ? I am connecting to a DB2 source and retrieving data, it already got 20million records on it, i just what to upload it incrementally on last modified date. need to query the DB2 using the date modified.

[@ssunitech6890]

Sure, you can add one table which will always have last execution time. Use this table in your query

[@kebincui]

Thanks for your tutorial. Very good. I have a question, in the step which you use 'OLE DB Command', can that be done also by using 'Execute SQL Task''? Thanks :)

[@ssunitech6890]

Oledb is available inside dataflow. But Execute SQL task will only available in control flow. So how can you do directly there

[@kebincui]

@@ssunitech6890 thanks for your reply

[@ssunitech6890]

Your Welcome

[@SloMoShort]

Use a checksum aggregation between each table and compare the checksum. It is quicker. Lookups on SCD are slow if you have n100k or millions of rows.

[@ssunitech6890]

Yes you are absolutely right 👍. I have explained about the techniques of incremental loads. This is a technique by which we can also do the incremental load. I have recorded other videos where I explained what you are suggesting. Thanks for your comment 🙏

[@styk924]

@@ssunitech6890 Hi sir, In which video did you explain about checksum technique?

[@quinnhogan545]

I know this was a while ago, but do you think you could further explain how to do this?

[@naveenKumar-iq9dr]

I get only o in place off false. How we change

[@ssunitech6890]

Check video again

[@naveenKumar-iq9dr]

Same also I AM getting 0 in place of false

[@sathishgoud]

why do you pulling all 12 records from source every time?

[@ssunitech6890]

Because source has 12 records.

[@rajendrakharage5392]

Same table insert update why not possible ? Stagging table ? Use ?

[@ssunitech6890]

Yes that you can do. I am explaining the concept

[@rajendrakharage5392]

Adabaly ... Understand question Adbaly

[@abhijith501]

Sir I have a requirement where I need to get data from multiple tables and views and the total records will be around 50000 per day. The target system uses REST so db data needs to be converted to JSON so how to do this. scenario - 1 If i want to send data in batch of 5k till it reaches 50k records scenario-2. Pull 50k records and then convert to json and send it as is(i think its not possible but advise on this approach)

[@kaushalsingh8000]

Sir, Can you please upload one video on SSIS configuration? Specially XML and SQL Configuration file

[@ssunitech6890]

I wanted to upload video on SSIS configurations.. I will take this request first. Thanks for your comment..

[@SloMoShort]

If you are using SQL Server 2012 or above. Use the project deployment model and do your deployment and configuration through SSISDB.

[@ssunitech6890]

Yes but I guess he wants to know how to deploy package in file deployment. Thanks

[@samuna8950]

If there are duplicate ID's. Like if I am having a fact table which is having duplicate records