Weekly Update 424

[@Shae-d3v]

Are you able to flush cloudflare cache in segments and flush it bit by bit over a period of time to reduce the sudden load on your service?

[@AndyWBurns]

Regarding trying to improve performance of inserting email addresses - could you use something like a bloom filter, held locally to the uploader, to check is the email address has been seen before? I.e. Rather than just an "insert where not exists", first check if you've already got it in SQL, and if not, then send the SQL insert. I bet checking a local filter would be faster than doing a SQL query into Azure.

[@neillarisa]

Is it so important to flush the cache immediately? Couldn't you have a TTL of let's say 24 hours and have a continuous but smaller load on the db? If you're loading new data breaches and flushing about once a day anyway then the average would be about the same.

[@TheGameIsOverCy]

03:00 what do we mean by updated hashes?