Rust at speed - building a fast concurrent database

This is a guest lecture I gave at Two Sigma in November 2018 where I discussed the experience of using Rust for building larger, high-performance systems. In it, I cover what makes Rust an attractive option for such projects; Noria, the high-performance research database prototype I've built using Rust; an interesting concurrent data-structure we use in Noria; and how I've found Rust to work in that context.
The presentation slides are available at jon.thesquareplanet.com/slide... .
You can read more about Noria at pdos.csail.mit.edu/noria .

[@distrologic2925]

21:36 This is actually one of the best explanations of the rust ownership system I have heard yet.

[@juliavanderkris5156]

Damn yeah, that was great. I'm starting with learning Rust and the concept of borrowing seemed cool but also really confusing. This explanation made it so much easier to understand.

[@gloatsticks]

I'm slowly but surely learning Borrowing. The compiler is one of the most helpful with error messages!

[@someonlinevideos]

@@gloatsticks For some type T you have: Here is the text from that slide because Jon really does create a wonderfully succinct explanation of ownership in less words than this comment has used. T, &mut T, &T T is owned, &mut T is exclusive, &T is shared

[@davidkeller6334]

That explanation should be "borrowed" by every educator/communicator who wants to explain ownership

[@bdafeesh]

How you speak is perfectly paced and you hardly stutter/'um'. I'm always trying to improve my speaking abilities and you offer a great example to follow. Keep it up man

[@Codeaholic1]

Jon I really appreciate what you do. You inspired me to sit down and learn Rust. You've made a difference in my programming journey. Thanks

[@allesarfint]

Something tells me you liked the language

[@wrong1029]

He's extremely based. Thanks to him I doubled my salary and fell in love with a programming language.

[@johanhellberg9677]

Congratulations@@wrong1029 ! Good work. Would you like to share how that came about, and what made the previous difference?

[@aemmelpear5788]

I absolutely love how you can tell, how much you are excited about the Rust programming language. Always with a big beaming smile when talking about the nice features Rust has.

[@nickmaxwellambient6615]

Jon, just found your channel today. It's rare to find someone who both covers really in-depth topics and does so in an entertaining way. Fabulous job you are doing here.

[@plorio]

I used that exact same atomic counter technique to decide when to the other copy of data is safe to write to at work. Felt quite clever when I got it to work. Really cool to see the same thing in the wild.

[@karlwhitford7523]

Well done and don’t forget to debug

[@yashashav_dk3766]

This talk is worth it's weight in gold. Thank you kind, Jon!

[@MrGeissT]

Well done, Jon! Fantastic talk and really interesting to hear more about what you're doing when you're not Rusting on KZbin!

[@rajsahu4644]

Jon is inspiring in so many ways. The work (which he explains in the presentation) and the way he described every topic! Just awesome. I probably never watched any hour long talk in one go until this one!

[@flesnuk]

Thank you for uploading this, the second part of the talk is awesome and easy to follow.

[@LiamDennehy]

I'm so glad i found this presentation again. I'm really getting into Rust and even advocating for it in my work, and recalled this presentation does a great job introducing Rust and proving how it can solve problems.

[@simonray4713]

I pretty sure he love this language. Full of energy with a great talk

[@scriptozavr]

Enjoyed the talk, Jon is indeed a phenomenal speaker!

[@dsincl12]

Really enjoyed this talk. The epiphany for me when hitting the borrow checker wall was when I realized that it was helping me not penalizing me for the code I've written. This small mental shift made everything all the easier and suddenly Rust became a pure joy to work with. I have to admit I didn't read through the whole Rust book from start to finish before hitting the wall. I usually never do that when learning a new language but you are absolutely right. Reading the book from start to finish will save you a lot of pain and frustration with the language.

[@yurikhomyakov9826]

Thank you for an amazing talk. As soon as Rust 1.0 came out I knew something interesting was cooking!

Brilliant talk, thanks a lot! I really like how you've covered the ownership model, along with how you've implemented the lock free evmap datastructure and why that is safe. Well done.

This talk is amazing! You explain things in a very approachable way!

[@thisiswill]

You’re good at explaining this - and I’m very new to exploring Rust. Thanks for posting this keynote.

[@yurizappo5726]

Love the idea of double increment and check if number is even. Simple and effective.

[@ankurleonardo]

Every second in this video was worth watching. Awesome DS glimpse.

[@veramentegina]

wow, what a great talk. Passion and enthusiasm makes such a difference. It captivated me. Thank you so much.

[@mahmoudabuzamel7038]

Thank you very much Jon, this is a valuable presentation rich with guidance and recommendations. I learned a lot from it!

[@sgdfdsgs]

Your talk was fantastic! Your passion and delivery were both amazing, and it really got me excited about Rust. Thank you for sharing your knowledge and enthusiasm with us.

[@hanskessock3941]

Jon is a wonderful teacher, and inspirational.

[@samferrer]

Wonderful lecture ... and the switchable rw-lanes is a must takeaway... thank you!

[@JohnHAdams-vo2pk]

Brilliant explanation of ownership. Simple and to the point. Better than any book I've read. Thank you for this, now it's pretty solid in my head.

[@jonwise3419]

I have a difference experience with rapid prototyping. To me rapid prototyping is MUCH easier with a good powerful type system if you do it in the right way. I'm still learning Rust, so I'll use Haskell as an example, but I already see it will be the same in Rust. In Haskell rapid prototyping is a breeze. Remember, that you don't need to write implementations or correct code! In Haskell you can write "undefined" and compiler will shut up. I normally, have several shorthands like "u = undefined" and then write a type and `u` as an implementation, compile, write more code replacing `u` with a more concrete implementation (that still can have more `u`s inside it) and then slowly an implementation writes itself as you replace undefined parts of your code with more and more concrete implementations. I heard this being called as type-driven or hole-driven development, because you follow the type, put holes in your code and fill those holes. Also, remember that with type inference you can replace anything with `_` in Haskell or with a sham type in Rust (like u8 when you know something is not u8) to figure its type or what type you need to pass, because it will complain and show you exactly which type something needs. Rapid prototyping is all about speeding up the feedback cycle and code-reuse. The latter is about not writing code at all, because you already have a library somewhere. Both are enabled by a strong and flexible type system. Starting with feedback, in Python to detect that your code is not working you'll have to wait until runtime .In Haskell or Rust, you'll immediately get feedback the moment you press a key. In fact, I usually wrap a function into another function that will print some useful output and I have very fast key shortcuts to both compile and to show some runtime feedback as well (like to print a datastructure I'm working on or run just tests that matter, keeping it fast). Like there will usually be a predefined name for me in a file, like `c`, which I can use at any time when working on a file. If I want to print something I just put one line in a file like `c = my_function testArg` and this is what will start getting printed out as I'm working on that function because that `c` will be like a main for that file, while I'm working on it that will constantly print output, and a script will automatically import that file and run it as I'm working on it. Same for tracing statements, pretty printing, and other tools that you would normally use. I want all of them to be a short names that I can use anywhere as I'm rapidly typing and working on something, preferably as a Swiss knife personal library that you can import in a few keystrokes into any file. Using other libraries is easier in a languages with a good type system due to a) type inference b) faster feedback cycles c) confidence in that if it compiles it will work. So, you can again just put a bunch of undefines, type holes / sham types, and slowly figure out what do you really need to some interface of that library or how to get there. In Python you can pass invalid data and you'll have to wait until runtime to figure this out. In Haskell or Rust, you will get immediate feedback that something is wrong and you have type inference to figure out EXACTLY what data needs to be there or how it should be used, and by using type holes can try to arrive at that point in any way you want by splitting the problem into smaller parts and following the types. What Rust misses here is more libraries, but when it will have a lot of libraries, figuring out those libraries will be much easier for reasons I've mentioned. Prototyping in Python can be easier DESPITE the lack of type system because there are more libraries so a lot of code you don't have to write. But when I have to write it, it's more painful. Python also has some good tools to speed up feedback cycles, like IPython, so some people naturally use workflows with better feedback cycles. But REPL is really not necessary if you have a good workflow which imitates REPL automatically. In fact, REPL can''t MAXIMIZE speed. Like you don't want to be typing code in REPL and then also in your file. You want your file to automatically give you REPL-like feedback. There are several ways to do it, beyond having scrips and conventions to automatically run things. You can have one file that acts as a RELP, imports a library you are working on and gets continuously run as you work on that library. I just find that more direct approach is faster. I guess to summarize this, I would say that if you don't fight the compiler but optimize your workflow then rapid then prototyping is easier with a more powerful type system. Rust can have no type system at all if you don't want it to, because you can ignore any parts of the code with unimplemented or panic.

[@blo0mfilter868]

nice blog post

[@QmVuamFtaW4]

uhh can you explain all of it in under like 17 words? my brain hurts

[@julians.2597]

​@@QmVuamFtaW4 Grug like Rust. Rust compiler not quite REPL, but compiler know lots things. Compiler often tell Grug when mistake in thing Grug makes. Tells also where problem abd how to make mistake go away.

[@clinton9129]

Jon, thank you and your friends for digging deep and discovering the possibilities. Since our last discussion, I am now learning how to code in python and I am learning other languages also. But my idea behind RUST is low latency and Jupyter notebook style coding. In fact Jupyter notebook was also one of my crazy ideas. Now that I am learning how to code, my writings' will be phenomenal given my skills. Expect more movies, lyrics to songs, Comedy segments, etc. Anything is possible now as always.

[@bobwatson957]

Excellent description of some Rust features that will trip up the newb. Excellent video.

[@ozank7327]

great talk, rust looks amazing

[@MaxLambrecht]

Amazing presentation. Now I love Rust more.

[@flyingsquirrel3271]

Thanks for the very impressive and interesting talk! If I wouldn't love rust already, you would have convinced me to learn it ;)

[@sanderbos4243]

I adored this talk, thank you!

[@ZenTrickz]

49:16 "... your program will not compile unless everything is documented. It's fantastic!" You know you are a Computer Scientist when... 🤣

[@YoloMonstaaa]

@Ramon Oliveira you'll get used to it :)

[@ekremdincel1505]

@Ramon Oliveira not really.

[@vhiremath4]

This was an incredible talk. Thank you. I shared this with my team - I'd love for us to consider using Rust for some lower-level video stuff. :-)

[@MIHIRAMRELIA]

Love your work, enjoyed the session !

[@benoitranque7791]

This was fantastic, thank you

[@linuxsir-tw]

Great talk, many thanks for your sharing

[@BenjaminCronce]

Super-linear is real, up to a point. Good memory access patterns can cause different threads to share cached data, such that 2 threads is more than 2x faster than 1 and 4 threads is more than 2x faster than 2. But in my limited experience, I never noticed past 4-6 threads, but at least beyond that was essentially perfectly linear.

[@TheSunscratch]

This is a really great talk!

[@kompeterPC]

Really interesting talk, well done

[@marcusklaas4088]

Lovely talk. Thanks!

[@arhyth]

good stuff and great presentation!

[@thomascorbin5371]

This was a fantastic talk

[@chang8106]

Thanks for sharing. Appreciate it

[@serejkus]

Great talk, thank you!

[@darkenblade986]

awesome stuff. I learned so much

[@ddystopia8091]

It is just incredible talk

[@SahilDawka]

You are a genius! Thank you!

[@haystackdmilith]

Great stuff

[@shirshak6738]

thank you ! Looking forward for your new stream. Rust is the best programming language i have ever used :)

[@mykra2939]

Thanks a lot, great talk

[@mxoliv_]

amazing talk!

[@_akshaydeep]

Database talk start at 13:40

[@VishalAnand24]

Don't jump, this talk is awesome

[@eMbry00s]

@@VishalAnand24 if you're into Rust, though, you've already been introduced to the Rust basics in like fifty other talks. So the timestamp is useful for some of us!

[@VishalAnand24]

@@eMbry00s okay

[@steveharris3627]

Nice, sort of an advanced/smart “refresh ahead”, like caching systems do, that is built into the DB.

[@raysonlogin]

The 1 map for readers and another one for writer mechanism is like RCU - whenever the writer is ready then the pointer to the data structure is published such that the readers get the updated version.

[@jonhoo]

Yup, that's exactly right, except that in this scheme we only ever have two copies and switch between them. In RCU you create a new copy for each update.

[@creatcodebuild7180]

Hi Jon, this is a great talk and I am reading your paper as well. I found this video while searching on the topic of Read/Write separation in databases because Datomic is not open sourced thus I can't study its implementation. Have you ever evaluated Datomic? If so, what are your thoughts of Datomic and how different it is from Noria?

[@BohdanTrotsenko]

33:30 the datastructure suggested implements a "weaker map", meaning that writes are not seen.

[@corv882002]

Very interesting talk. I'm curious why you couldn't just store an atomic counter of how many readers are active per map instead of doing the whole epoch and ignore it if it's even thing.

[@psychicopus]

My inspiration

[@KFlorent13]

Thas was really really good.

[@prestigek1ngs]

I’m currently getting into rust with no basically no prior knowledge. It’s kind of a pain but it’s been fun. I’m on the fence still because I can’t find the best tutorials but I’m trying hard. I think it’s the future

[@sluongng]

at 52:00 he was describing that he had PTSD from fighting the Compiler ^_^

[@nextlifeonearth]

Or Stockholm syndrome.

[@manhquan5464]

Hi, thanks a lot

[@WolfBoy2700]

Really interesting! you did that very well! Thank you .

[@colinmaharaj]

16:40 That is spectacular, I like it, will try that. (Update the read caches on write)

[@emdeization]

Amazing

[@randomplan9537]

thank you for this great intro to noria. is there a way to try noria also ?

[@jonhoo]

github.com/mit-pdos/noria is what you'll want to look at :)

[@adesanoyesamson668]

2022 and I find this talk interesting

[@jwyliecullick8976]

Bravo.

[@batorshikh.baavgaikhuu]

Interesting!

[@danielfielding1938]

If he were creating a fairly normal relational DB in Rust, then it would be a cool way to see the advantages of Rust, but those advantages seem to be somewhat obscured by the unconventional design of his unusual database. In other words, maybe the radical design is what makes Noria cool, not Rust?

[@giganooz]

As for the database thing, how does it work when you have multiple queries that try to change data through addition at once. Wouldn't it read the same value multiple times first an then change that multiple times instead of going in parallel?

[@TimLF]

I was expecting "Multiversion concurrency control" of PostgreSQL, and CockroachDB to be compared, but then the video was over. I'll have to add that to my needs more research or testing list.

[@joshuayanovski5765]

MySQL also uses MVCC (albeit a slightly different variant of it, but one that actually makes reads by primary keys faster than Postgres at the cost of reads through secondary keys). The thing he's leaving out of this talk is that Noria does not fully support transactions, so it is not quite apples to apples; however, it is plausible that the queries he's testing are nonetheless transaction-safe (or have semantics that don't require full transactional isolation).

[@DavidBeaumont]

I'm interested in whether this approach is safe in shared memory multi-cpu systems. Aren't you relying on the order in which data pages are published to the different CPUs to be able to promise that an even count reflects the true state of the reader in another thread? Does Rust have some promises about the semantics of the memory architecture? (e.g. are you implicitly relying on the counter and the table pointers being in the same memory / same cache lines?)

[@jonhoo]

The epoch counters use atomic loads and stores (fech-add specifically) with well-defined memory orderings. Take a look at the AtomicUsize type in the Rust standard library :)

[@FrancescoCielo]

Isn't storing nodes of a graph in a list and then using indices basically just doing pointers while hiding it from the borrow checker so it doesn't bother you?

[@jonhoo]

Hehe, yes, pretty much!

[@dan-vw4ve]

Really awesome talk, one thing I don't get is the reasoning for an epoch counter instead of flipping between 0 and 1? edit: is it an ABA thing?

[@jonhoo]

So, you could get pretty far by just flipping between 0 and 1, but using an actual counter enables some cool optimizations. For example, in the real implementation, all writes are delayed until the _next_ swap, and only then are they applied to the "write" side of the map, which is then immediately made the "read" map and exposed to readers. This allows the swap to only block until the _previous_ pointer swap has been observed by all threads, at which point the "write" side is safe to modify. This, in turn, means that usually the swap doesn't have to wait _at_ _all_ , since in the time between the previous swap and the current swap, most reader threads will have finished any pending operation they had. _But_, this relies on being able to distinguish between a reader that is still doing the _same_ operation it was doing when the previous swap happened (and so hasn't seen the latest pointer swap) and one that is doing a _new_ operation (and so _has_ seen the latest pointer swap). And 0/1 wouldn't let you do that, so you would _always_ have to wait to observe a 0 from each thread.

[@passermelon7]

I have a question...When you exchange the reader/writer pointer, you need to ensure all the reader pointer number are even.What if you check the latter pointer, the former pointer's number change.It seems like we still need a mutex?

[@jonhoo]

Ah, no, it's sufficient to observe an even number for each reader _separately_. For any given reader, if you observe their epoch number being even, you know that _that_ reader has seen the updated pointer, and so you don't need to check it again until the next time you want to swap the pointer.

[@magnum789]

​@@jonhoo Hey Jon, very exciting talk! I'm new to Rust but have experience with Go. How does the writer reapply the writes it has done to map B into map A after swapping the pointers and revealing map B to readers and thus obtaining exclusive access to A? It seems like the writer must internally keep track of the missing writes (surely more than one missing write in map A due to possibly a reader being slower than the writer)? Another question: if the writer observes the even numbers separately for each reader, how does it prevent a reader from starting a long read just after the writer made the observations and just before it does the swap? Edit: I think I got the answers from reading the evmap docs now. There is a log, of course. The swap is just done at some unspecified point in time and the replay is done AFTER waiting for the epoches to get even. I'm thinking of trying to implement something like this in Go.

[@jonhoo]

Hey there! Almost missed this comment since it's under a relatively old one, and KZbin tends to hide those. Looks like you figured it out though! There is indeed an operation log so that the writer knows how to bring the "old" reader map up to speed when it becomes the "new" writer map. For the epochs, the way it works is that the writer will not start writing to what used to be the reader map until it has observed an even counter from every reader _since_ the swap. That ensures that any subsequent reader operation *must* be on the swapped map, and therefore there cannot be any readers left in the old reader map.

[@LaurentLaborde]

i came here mostly for the database part of the talk :)

[@bozhidaratanasov7800]

Do you think Rust has caught up to speed on async networking stuff? I am trying to follow the Rust trends, but I am not sure what the difference between now and 4 years ago is. Tokio seems to have been doing a fair bit of work there.

[@distrologic2925]

37:03 Instead of counting each read begin/finish, why not just keep a boolean value, which indicates whether the thread is currently reading or not. When starting a read it will set it to true, after finishing a read, it sets it back to false.

[@jonhoo]

If you did that, you might continuously read that readers are active if reads are sufficiently frequent and fast. That would block switching to the other map, even though it's safe to do so the moment all readers have finished at least one read.

[@davidboreham]

Quite surprised MySQL and Memcached are so fast by comparison. I'd expect more like O(10^2) difference.

[@paolophoenix]

We did reasearch on this topic 25 years ago. The approach fail on non monotone schemas. Sometimes you have to recompute all the db. Easy to demonstrate. I can write a db in wich if you add a record in a table (cache) that cause all other cache to empty. The future is in functional databases. Where tables are functions.

[@jonhoo]

It's true that there are cases like that, but the argument we're making here is that that is not true for *common* queries for web applications specifically. And even in those cases, if your application is read-heavy, it is far better to re-compute the query result only when the result changes, rather than on *every* read. If you have a particular problematic use-case in this domain in mind though, I'd be happy to take a look!

[@jonabirdd]

how can I upvote this talk twice?

[@KhoaNguyen96]

Create another account ... perhaps :)

[@junaid1464]

Can't postgres function idx do that?

[@haydarinda]

Thanks. Additionally, 0.8 is a reasonable speed for non-native speakers.

[@robheusd]

Isn't there a chance that just after the writer sees that all read counts are even and it is safe to do a write, and just before the writer/reader pointers get interchanged bfore the actual write, a reader starts a read, or what prevents the reader from doing that?

[@jonhoo]

The writer swaps the pointer *first*, then waits for all the counters to change, and the starts modifying the old pointer. If a reader starts doing another read, they'd follow the swapped in pointer, which points to the other copy (the one the writer isn't about to modify)

[@guibirow]

At 37:00, to check if there are no readers still reading or suspended before starting writing, he suggested to increment the counter twice. One on starting the read, the other when completed, then check if the counter is even before writing. With this approach, if there are two readers or any even number, will cause the writer to falsely assume there is no reader because the counter will be even. I would assume the right approach is increment on start and decrease when finished. Then, if the counter is 0, the writer can confidently guarantee no reader is pending finalization before start writing. Is there anything I am missing?

[@jonhoo]

Ah, no, each reader has its own counter :)

[@guibirow]

@@jonhoo Nice, I thought it was a global counter

[@Ethan-gu9hm]

The idea of caching query results cannot be that new or novel isn’t it? I recall that oracle db has the feature with the name “materialized view” for a long time now

[@jonhoo]

There's a little bit of discussion around this in response to www.reddit.com/r/rust/comments/acucrs/rust_at_speed_building_a_fast_concurrent_database/edbxqv8/. In general, materialized views in commercial relational databases are limited, both in terms of performance and flexibility. I'd recommend giving the Noria paper a read if you want more in-depth analysis, as we evaluate that there too!

[@panstromek]

The idea of it is just so straghtforward... Basically since I started working with DBs, the whole inefficiency of their work has been bugging me, especially considering many of them were built at times when computational resources were sparse. I just don't get it.. I am glad that someone just took this pretty logical idea and just built it. Great work and talk @@jonhoo ;)

[@robheusd]

If I'm correct, even before materialized views were added, queries, parsed queries and their result sets were chached and stored in the shared pool of the PGA.

[@maherkhalil007]

Can you make tutorial about using rust + databases + web assembly to produce MVC software?

[@markocvejic6416]

Link for voting on your youtube chanel, intro video is broken :)

[@micknamens8659]

What happens when a read client on the partial materialized view A has an odd number of points in his access accounting, and the associated client thread died or is in an endless loop and will never end its query, i.e. won't increment its counter?

[@jonhoo]

It's a good question! The current implementation has no story for that - it would cause the writer to wait indefinitely. If you use Rust with unwinding, the trick to pull would be a "panic guard": a type created just before the user code is called that, if dropped during a panic, will increment the epoch counter to restore parity. If you want to try to submit a PR to evmap with that change, I'd be happy to mentor and review it! If the reader runs indefinitely, that requires the writer to wait as well.

[@micknamens8659]

@@jonhoo I'm currently just "interested" in Rust, so no PR from me 😉 . What about a time-out for Read handles. It expires after x seconds. That's then part of the API contract. Could be a semantic extension of Rust language or a user-defined wrapper type: when the read client tries to call a function of this type after expiry time an exception is raised on client side. So the writer need not wait for an expired read handle.

[@jonhoo]

@@micknamens8659 Hehe, that's okay! I think getting the semantics of expiring read handles right would be quite a challenge. And especially because you'd need some way to preempt a running thread, which is far from trivial!

[@jonhoo]

Just implemented this in github.com/jonhoo/rust-evmap/commit/73a67292b5588d6109b6845f6e771d804a9ac906 !

[@micknamens8659]

@@jonhoo Another approach to overcome this exceptional situation could be to create a temporary third materialized view for writing.

[@piyushkatariya1040]

Old wine in a new bottle. This is very similar to following the even sourcing model with a materialized view (MV). or Simply use PostgreSQL 11 + CitusDB (distributed computing and sharding) + PipelineDB (continuous computation and MV)

[@jonhoo]

Noria is actually very different from both of your proposed systems, but this was a relatively high-level talk that didn't talk much about the actual contributions that Noria makes. If you're interested in a more technical discussion, I'd recommend giving the research paper from OSDI'18 a read: www.usenix.org/conference/osdi18/presentation/gjengset

[@jawad9757]

Interesting that you used the word similar

[@colinmaharaj]

7:56 hmmm, I rely on null pointers in C++ to know about the state of my objects.

[@dbaldwin2803]

Does anyone know if this has been abandoned? The GitHub repo hasn’t been updated in quite some time and I really want this to flourish.

[@jonhoo]

For Noria, the thing to watch is readyset.io/. For the data structure described in the talk, check out github.com/jonhoo/left-right/ !

[@dbaldwin2803]

@@jonhoo so some of it is open-source and some is proprietary?

[@jonhoo]

@@dbaldwin2803 All of Noria that was part of my research is open source on GitHub. However, since I graduated, it's not an active project, just the final state of the research prototype. ReadySet is building on top of the Noria code base and making it production ready. Some of that is proprietary, although much of it is also developed in the open over at github.com/readysettech/readyset

[@dbaldwin2803]

@@jonhoo can you elaborate on which parts are proprietary? I tried pitching Datomic to our CTO for a new product and it being proprietary was a non-starter.

[@jonhoo]

You'd be best off reaching out to them directly! While I'm a co-founder, I'm not involved with any of the ongoing work or plans :)

[@gregoryterzian8145]

Nice presentation, and I'm wondering if you've considered a potentially more simple solution to your problem: use multiple fine-grained locks, and use locks optimized for unfairness. It seems somewhat of a straw-man to compare a lock-free hashmap, with a non-concurrent hashmap protected by a single Rwlock, in the sense that one wouldn't expect the read throughput to scale linearly with the number of threads(beyond those few initial threads), when those would be contending on the same lock. Also I think you're taking a bit of a shortcut when saying that "with short critical sections, the lock acquisition/release becomes the bottle neck". With short critical sections, the point is doing work in parallel, and outside of the critical section. So if you compare the time spent acquiring and releasing a lock(entering/exiting the critical section), it should be very small compared to the time spent performing work outside of the critical section. The time spent in the critical section might be smaller than the time spent acquiring/releasing the lock, but that's besides the point, the real work is done outside of the critical section, and a thread should only enter the critical section to "publish" the result of parallel work for others to read(and take a snapshot to use in further parallel work?). ---- So I think there are two problems you're trying to solve, and going from "a single lock doesn't work", to "let's use lock-free algorithms", is a big jump from one side of the complexity spectrum to the other, one that is missing the low-hanging fruits in the middle: 1. When the individual threads in your workflow are making progress via lots of short-critical sections, you can optimize parallelism by allowing threads to re-acquire a lock they've just released, the so-called "unfairness" of the lock. 2. To improve your ability to scale the throughput of operations on the shared-data when those involve more than just a few threads, one could use many fine-grained locks, for example one around each bucket of data. Another benefit of actual locks, is that they can be paired with condvars for signalling, which can be useful if your business logic is not just about achieving lots of parallelism in reads or writes, but also cares about ordering or other types of logical synchronization between threads doing the reading or writing. I recommend the following articles: This one, which talks about optimizing locks for small size, cheap uncontended operations, and unfairness: webkit.org/blog/6161/locking-in-webkit/ This one, about locks, condvar, and concurrency in general(it has a nice paragraph on why one should avoid atomics as much as possible, and how locks can actually show better performance): www.chromium.org/developers/lock-and-condition-variable And off-course, in Rust, there is already a library offering you the kind of locks you would seem to need: docs.rs/parking_lot/

[@jonhoo]

Hi there! I am aware of all those options - I do research in this area. Fine grained locks do not work well when you have a skewed access pattern, because the threads still end up contending on the shared cache line for the lock semaphore. A shared reader-writer lock helps, but is harder to manage on top of a work stealing runtime where tasks may move between threads and cores decently often. As I say in the talk, this is a data structure specifically optimized for when you do not want stronger consistency, and that relaxed requirement allows you to perform additional optimizations that would otherwise be disallowed. That all said, I agree with you that this is not the right approach everywhere, and I also never claimed that it was. There was a lot of content to fit into a one hour talk, and some subtlety inevitably gets lost. I expect anyone with serious needs for concurrency to carefully weigh their options.

[@gregoryterzian8145]

@@jonhoo Ok I see, interesting. Have you tried this without any shared data? I can imagine having a "master thread" own the data, with reader/writers owning a local clone of the part of the data they're interested in. Then the master thread would receive streams of writes from writers, and propagate them as streams of updates to the readers? When a reader wants to access a different part of the data, it's a bit like subscribing to a new stream, where the first chunk is a clone of the initial data, followed by a stream of updates?

[@jonhoo]

You'd then up with many copies of the underlying data, which would be unfortunate. It would also require readers to synchronize with the writer to ensure they get the latest updates, which could be costly. It also adds significant costs if you have many reader threads accessing the same underlying data, which is often what happens when you have skewed access patterns. With the design I outline in this talk, you could have 16 threads all reading the same key at the same time with no contention among them at all.

[@gregoryterzian8145]

@@jonhoo I don't think you'd necessarily need the readers to synchronize with the writers, what I mean is that the writers publish updates to the "master thread", and the "master thread" then publishes updates to each readers(subscribed to updates for that particular piece of data). Or the readers pull updates, or a combination of both. Yes if all readers ask for an update to the same data from the master thread, that's going to end-up being serialized, and so would access to the same key on a concurrent map that offered strong consistency requirements. So looser consistency could also be encoded in either the behavior of the readers(don't ask for updates), or behavior of the master thread(pre-publish stale updates to meet request from readers). Optionally the writers could publish something to readers that "they've sent an update to the master thread", which would be a signal for readers to pull updates from it(or something like that). With regards to the data, a local (partial) "copy" also allows readers to read from it at full speed, since it's local data they own. So there is a trade-off there between copying data and avoiding shared-state(a concurrent map can be fast, but never as fast as a local map). That local copy is not going to be up to date, hence the option for readers to get up to speed by pulling updates from a master thread, or some other business logic. I understand there are lots of details and it's hard to argue this here, my main point is that you don't necessarily need to encode your parallel logic through a fast concurrent map. A fast concurrent map can be great, and it can be even better to simply have threads operate on local data in parallel, especially if you have loose consistency requirements. So when I read "16 threads all reading in parallel from the same key on a concurrent map" I think "16 threads all reading in parallel from local data, backed by some stream of updates implemented through message-passing(and hopefully doing some useful work besides reading from local data at a high-throughput)". Such a model also potentially allows for much more precise business logic in terms of requesting (consistent) updates, signalling that an update is available and so on...

[@jonhoo]

This discussion is very poorly suited for KZbin comments :) I agree there are alternative designs, I don't think the design you outline above has any advantages over the one I outline in the talk in the read-heavy workload context, and I don't think it is any simpler. It also comes with disadvantages, such as more cross-thread data movement and duplication. My guess is also that the version I outline will be faster since it allows more sharing of CPU cache state, though that's hard to say that for sure without empirical evidence. Local data is also not any "faster" if it is not concurrently modified, since the data will be marked as Shared in the CPUs, and they will all be allowed to read and cache it. I _also_ don't agree that it allows for "more precise business logic". They both present the same consistency guarantees for updates if you swap on every write. It's true that it buys you condvars/notifications, but if you don't need those, then that is just pure overhead. You are right that it's possible to model this in a sort of actor model with message-passing, but I don't think that is an obviously better design, it's just a different one, which presents a different point in the design space.

[@amdwi]

🤯

[@Kengur8]

Tea leafs on the stack.

[@thingsiplay]

40 years from now, no one will understand how someone could even write unsafe code. You can quote me, if I am wrong.

[@liquidmetal718]

38:00 hmm, sus....