31 Delete duplicate records in SQL, keep the new records

[@mmmmmmmmmmoim]

Watched your video from work, then I took the time to connect on my home PC find it again so I could like it with my personal account and leave a comment. Thanks.

[@learnssis]

Thank you so much for providing your valuable feedback on this one. God bless you.

[@drummermike5150]

Excellent tutorial. I have learned so much from you this week. Thank you for creating this channel and providing such valuable content to the community!

[@learnssis]

Thank you Mike for appreciating this.

[@drummermike5150]

Thanks!

[@deepakj6717]

Hi Aqil, Thank you for video!! In real time project which is better as per performance built-in logging or custom logging? How can we log the record counts which was getting inserted, updated or deleted when we run SSIS package using custom logging?

[@learnssis]

Logging won't affect much on the performance side. Custom logging for each package will be done differently according to the code that you have written in the SSIS package. In below video I have implemented the custom logging for number of records inserted, deleted and updated. kzbin.info/www/bejne/mIvEmq2Bq7d0btE

[@tushardeepsingh4976]

Sir . I have one requirement My Ssis package reads csv file from a particular path and dump to sql server table in staging database In first load there is no issue But my data comes every week in csv . So I need to load this data every week to the table in staging database . Please suggest how do I write code . Whenever the data is inserted in this staging table it should always be new data and not the already existing data Also suggest how do I schedule this Ssis package every week Sunday at 2 pm

[@learnssis]

I think it is pretty straightforward, the first task in your ssis package you can take is execute sql task and there you can check if staging table exists then drop the table and recreate it, now you can use a foreach loop container with file enumerator and then inside the foreach loop container you can use a data flow task and there you can fetch data from using a flat file source from the csv file and using ole db destination you can write it to a sql server table. Now when the package will run it will clean the staging table and will insert the data from csv file to sql server table. Now using sql server agent, you can create a sql job and can schedule this ssis package to be run on desired schedule. I have created a video on "loading only new files to sql server", you can use majority of the code from that ssis package, the extra thing I am doing in that video is that before loading a file I am checking if a file is already loaded then it won't load the file, if you don't want to do that then you can remove the first task inside the foreach loop container kzbin.info/www/bejne/qXzPdXirgNOUpNk Then in below video I have shown how to schedule your ssis package kzbin.info/www/bejne/p6O5Y3mYhtWIaas

[@tushardeepsingh4976]

@@learnssis firstly thank you so much for taking our time and typing this much for me . I’ll try your suggested approach. I don’t have ssis knowledge. I majorly work in sql. I’ll go through your suggested videos . Just in case if I’m still clueless on what needs to be done . Is it possible if we can meet and fix this in person ? I stay in Bangalore. I’ll pay you if required .

[@learnssis]

@@tushardeepsingh4976 This requirement is very simple and you should be able to do it easily. If you stuck and got any error then you can email your error with screenshot at [e-mail] and I will try to reply on that one.

[@tushardeepsingh4976]

@@learnssis Sure Aqil. I’ll email you just in case .. thanks again

[@hasanmougharbel8030]

Hey there,keep up the great work in your channel. I have only a general enquiry as a newbie sql learner. Could i channel shopify live data into an sql application?(This should be an application developed for a company of 50 employees mainly for the sake of generating reports and analytics)) Is this an easy process to undertake as a sql beginner? What should i keep in mind and what are the tools or softwares that i should work with right from the beginning. For notice, I made my mind to work with microsoft sql server, also our company has a dedicated server. Thanks for taking care of my enquiry.

[@learnssis]

Yes, you can get the data from Shopify into a sql server database. SSIS can be used for this project but at the moment there are no inbuilt tools available in SSIS which can fetch data from shopify but there are some third party tools available those can fetch data from shopify and then you can insert that data into a sql server database. www.cdata.com/kb/tech/shopify-ssis-task-import-2008.rst

[@ankitsaxena565]

Sir, can you make video in Hindi

[@learnssis]

Hi Ankit, Sorry as of now I don't have plans to make videos in Hindi, not sure about future.