"Distributed Ranges": Model for Building Distributed Data Structures, Algorithms & Views - Ben Brock

cppcon.org/
---
"Distributed Ranges": A Model for Building Distributed Data Structures, Algorithms, and Views - Benjamin Brock - CppCon 2023
github.com/CppCon/CppCon2023
Writing programs that run on clusters of computers is notoriously difficult. Distributed data structures can simplify this problem by automatically splitting up data across multiple memory spaces or nodes in a distributed memory program. Algorithms in most distributed data structures libraries, however, are not generic, but operate only on a specific data structure. In this work, we discuss how modern C++ features like concepts, customization points, and the ranges library can be used to support generic algorithms and views that can interoperate with multiple distributed data structures. We first discuss background in distributed computing and distributed data structures, including common techniques used to create high-level distributed data structures. We then show how customization point objects and concepts can be used to allow the implementation of generic algorithms that can be used with any distributed data structure as long as it fulfills a "distributed range" concept. We then show how to implement views that satisfy this concept, allowing algorithms to operate on lazily evaluated views of distributed data structures. Finally, we discuss two implementations of this model, for multi-GPU and multi-node programs, as well as performance considerations when running on state-of-the-art HPC systems.
---
Benjamin Brock
Benjamin Brock is a research scientist at Intel Labs, where he works on building libraries, tools, and algorithms for high-performance computing. In 2022, he completed a PhD in distributed data structures at UC Berkeley. His work focuses primarily on building cross-platform data structures for distributed applications, with applications in sparse linear algebra and graph algorithms. Ben is a member of the GraphBLAS Languages Committee and is involved in drafting the GraphBLAS C++ Specification.
__
Videos Filmed & Edited by Bash Films: www.BashFilms.com
KZbin Channel Managed by Digital Medium Ltd: events.digital-medium.co.uk
---
Registration for CppCon: cppcon.org/registration/
#cppcon #cppprogramming #cpp

[@nibrobb]

Very cool

[@wjrasmussen666]

This is going on the watchlist

[@Roibarkan]

11:03 Benjamins talk from 2021: kzbin.info/www/bejne/o4vahnpuqJiVZ5Y

[@RicardoCapurro]

Very interesting talk, but I believe this would never reach a production level code, mostly because the intention of ranges is to provide safety since iterators are not safe, and although iterators provided a nice separation between containers and algorithms they failed on safety. I believe distributed containers should be addressed on a different and more specific framework.

[@markusasennoptchevich2037]

Is that a trap in the first minute? Oh god...

[@douggale5962]

I am so tired of hearing how hard parallel programming is. It's not hard.

[@victoreijkhout7115]

Shared memory parallelism is not hard. Distributed? You ever written a sparse matrix-vector product? Scalable at a million cores? (And by scalable I don't mean "it runs" but: efficiency bounded from below.)

[@SimGunther]

So long as you have the exact same clock for every computer and no shared state is ever mutating, then it's 100% easy 😅