The Best Way To Implement Soft Deletes With EF Core (and why you shouldn't)

[@MilanJovanovicTech]

Want to master Clean Architecture? Go here: bit.ly/3PupkOJ Want to unlock Modular Monoliths? Go here: bit.ly/3SXlzSt

[@silverfox3074]

Excellent video! But I was thinking if there is any way that filter by isDeleted to be automatically? For example not to check every record if isDeleted but method where to handle this automatically?

[@Isr5d]

Soft delete is a great feature if you want to add 2-step deletion process to the user. User can recover the deleted items within a respected period of time, or can delete them permanently. In case the time of deletion has reached the recovery time threshold, the system will delete them permanently. It's like user recycle bin. You can adapt hierarchy approval to it as well. So user will soft delete the entry, but a supervisor can either approve deletion process or reject it.

[@ajdinhusic2574]

There is additional reasons to use soft deletes. Sometimes you need to retain related data, like calculations, results, statistics, reports etc.. Often times you dont want to destroy those just because a record was deleted

[@MilanJovanovicTech]

That's a very nice use case

[@schoderfactory]

I enjoyed this one, again. How you think about the consequences of each option is very helpful for me.

[@MilanJovanovicTech]

Glad it was helpful!

[@PurpleDaemon_]

10:12 a filtered index on a bool fields one only makes sense in the opposite case, when you are filtering a much smaller part. In your case, your index will cut off a couple of percent at most and will most likely be completely ignored by the base. And here are two additional things to think about when implementing a soft-delete: 1) Without additional changes hidden objects will still be taken into account when checking the uniqueness of fields by the database 2) Soft-deliting an object with cascading dependencies can lead to an invalid state of the database.

[@MilanJovanovicTech]

1) That's a nice problem to consider. But I'd look at it as a business decision and see what makes sense. Maybe we can make due without a unique check, and do a query at runtime. 2) How so? As in, the dependents wouldn't be marked as deleted?

[@PurpleDaemon_]

@@MilanJovanovicTech 2 - yep, you could have items with soft-deleted parent, which can easily break FE or other stuff.

[@Kingside88]

Thank you, Milan. An incredibly good explanation.

[@MilanJovanovicTech]

Glad you enjoyed it!

[@kewqie]

At 4:15, why not just do "product.IsDeleted = true;" before calling "Update(product)" instead of doing that long ".Property()" call after?

[@ajdinhusic2574]

product.IsDeleted = true marks the EntityState as Updated, and so an Update Event is fired instead of a Delete event.

[@kewqie]

@@ajdinhusic2574 isn't that done by the Update()? He talks about it 15s earlier.

[@MilanJovanovicTech]

1) IsDeleted had a private set 2) Yeah, Update isn't necessary there. But just wanted to make it explicit since I had Remove() beforehand.

[@saleem.shaikh]

Next video, if you can, please do a video on audit trail, and how to implement it, can we use mediatr for it or not.

[@MilanJovanovicTech]

I discussed how to do this with EF Core: kzbin.info/www/bejne/o3LPgGaFqtSErNE

[@andyhb1970]

I find soft deletes to be invaluable, firstly it enables easier tracking of deletes for migrating changes via ETL processes to external systems and secondly users can't be trusted and soft deletes have saved many a painful situation over the years 🎉

[@MilanJovanovicTech]

Let's go soft deletes! 😁

[@ajdinhusic2574]

I feel like there must be a library by now for soft-deletes using EF core, right? Also Milan, what do you say about implementing ISoftDeletable using a nullable DateTime(Offset) type for the deletion marking. as this contains more information that might be useful. The filter and index would then be on: DeletedAt == null .

[@MilanJovanovicTech]

I believe I mentioned that we could use a DateTime column 🤔 Fine by me.

[@ajdinhusic2574]

@@MilanJovanovicTech yeah but I thought you meant as an additional column alongside the boolean? Or instead of the boolean? If its instead of the boolean then yes thats what I meant. Which do you prefer btw?

[@kjbetz]

Excellent video as always Milan!

[@MilanJovanovicTech]

Thanks a lot!

I think in terms of ddd and clean architecture it would be better to use a shadow property in the EF configuration instead of modifying the domain layer

[@MilanJovanovicTech]

That might make sense, yeah

[@user-kh8mv2be8t]

It is better to do this kind of thing using the SavingChanges delegate rather than overriding the SaveChangesAsync. There are multiple SaveChangesAsync overloads which internally call SaveChangesAsync(bool, CancellationToken), so you'd need to make sure you are overriding the correct one or else your logic will get skipped if a different signature is invoked, this also doesn't cover non-async calls using the SaveChanges() method, so you'd need to cover that set as well to implement this properly, finally the dispose check code runs just before that delegate is invoked, which will give you a nicer and more consistent error message if someone tried to invoke the method on a disposed instance. This also allows you to either include it in the class as you do here or attach the logic as needed.

[@MilanJovanovicTech]

I only ever call SaveChangesAsync from my code, but that's a valid concern. We could move this logic into an interceptor, though.

[@daniel_klement_photography]

More this simple and useful videos!

[@MilanJovanovicTech]

Will do :)

[@arteqppp6223]

Any reason not to do some base AuditableEntity class, which would have IsDeleted, DeletedOnUtc, Id, itd for all of entites?

[@MilanJovanovicTech]

I'm just giving ideas here, you can take it further with something like that 😁

[@arteqppp6223]

@@MilanJovanovicTech Sure, thanks!

[@ChrisWard74]

I'm new to EF Core so I have lots to learn. When you added IsDeleted you had a Migrate function do you have more details on what that is/does? Do you write that function and it's run one time to modify the database?

[@MilanJovanovicTech]

It's an extension method that applies EF Core migrations when the application starts. dbContext.Database.Migrate();

[@ChrisWard74]

@@MilanJovanovicTech Wouldn't it be better to do these migrations outside of code in a SQL script? Over time you could have more and more migrations as things change so you wouldn't want all of those migrations running every time or checking to see if they need to run every time the application is started.

[@yotelolailo]

@@ChrisWard74 They are run only in development environments. Usually for production environments you have to select a different technique to run your migrations manually or automated but not on every application start.

[@saleem.shaikh]

Nice video.

[@MilanJovanovicTech]

Thanks a llot!

[@AlejandroPerez-m9r]

Could you please tell me which visual theme you use, or could you share the colors you use in your configuration, or make a video about it? Thank you very much.

[@MilanJovanovicTech]

ReSharper

[@AlejandroPerez-m9r]

@@MilanJovanovicTech Would you be so kind as to indicate how to achieve that configuration, please?

[@matthewrossee]

Hi Milan! How would you implement cross-cutting concerns like validation or caching WITHOUT using MediatR and its pipelines? Let's say the classical service architecture. Maybe one could use a decorator pattern with Scrutor. Do you know how to implement something like this?

[@MilanJovanovicTech]

Middleware? 🤔

[@matthewrossee]

@@MilanJovanovicTech I guess it’s an option but seems a bit clunky. Can’t find a good repo that demonstrates something different than mediatr pipelines

[@MilanJovanovicTech]

@@matthewrossee Which is why MediatR is so awesome. I never understood the hate against, considering how many awesome options it unlocks.

[@matthewrossee]

@@MilanJovanovicTech have you used the Martin Othamar’s mediator implementation? The same API, but uses source generated code, maybe you’ll find this interesting.

[@playmaker1633]

Hi Milan thank you for all of your videos ! I have a question about Clean Architecture, I would to do some Business Calculations (like calculate prices), and I would like to know in what layer I should do and in what class Thank you so much ❤

[@MilanJovanovicTech]

Probably Application, but hard to tell without context

[@ajdinhusic2574]

@@MilanJovanovicTech exactly. First that comes to mind is application, but like you said, it depends. For example, many calculations can often times be attributed to the domain layer. Its often worth to think about it more closely.

[@haroldpepete]

great video, i implemented sofdelete in one project, but some questions come to my mind, what happen if i mark a isdeleted flag in one record, a producct record for taking an example, when i try to register another product with the same name like my softdelete flag and my database has a unique constraint by name? how i can manage this behavior? because i gonna have problem trying to register a new product with the same name with some product marked as softdelete

[@MilanJovanovicTech]

That's a business question, not a technical one

[@PurpleDaemon_]

Depending on you DB use nullable bool field or filtered index.

[@ibowyer]

I have used temporal tables in ef core for archiving data changes before do you think that is another possible solution?

[@ajdinhusic2574]

Yes, that reminds me of that event sourcing pattern that was recently highlighted by Nick Chapsas

[@MilanJovanovicTech]

Yes, albeit more complex

[@alexxx8338]

Found problems imlementing this on my aggregate with nested soft delete entities collections in it. Quer filters are not going to solve my problems in this case

[@MilanJovanovicTech]

True. Collections make this quite a bit harder.

[@saddamhossaindotnet]

Excellent content!

[@MilanJovanovicTech]

Much appreciated!

[@هواتف-م9ر]

Thanks 👍

[@MilanJovanovicTech]

Any time!

[@barr5221]

what about SQL server temporal table or table with from and until date? Soft deletes + full history of the changes on the table. Very usefull when debugging what happended to your data.

[@MilanJovanovicTech]

Too complex for this video 😅

[@kman12275]

I love the temporal tables most of the time. I am considering a soft delete though where the user might bring the record back.

[@andyhb1970]

SQL Change Data Capture works extremely well and I've used it to feed an ETL process to our data warehouse and it all works perfectly in Azure SQL as well.

[@RikinPatel13]

How to do same things in CosmosDB with Linq?

[@MilanJovanovicTech]

No idea

[@codenetw]

How about not creating additional entities in the database and maintaining them by moving items, but instead simply partitioning by the IsDeleted flag? When querying with a filter, the database will more easily select from the partition where IsDeleted = false. create table if not exists products ( ... product text, isDeleted boolean ) PARTITION BY LIST (isDeleted); CREATE TABLE products_new PARTITION OF products FOR VALUES IN (false); CREATE TABLE products_deleted PARTITION OF products FOR VALUES IN (true);

[@MilanJovanovicTech]

I think that's a PostgreSQL feature, right? Would it automatically move records from the products_new to products_deleted partition on update?

[@codenetw]

@@MilanJovanovicTech I thought that mssql also supports this function, but yes, you are right, it will not allow you to update the record if it is a key for the partition :(