Fast and Efficient Log Processing with Wasm and eBPF - Michael Yuan, Second State

Don't miss out! Join us at our next Flagship Conference: KubeCon + CloudNativeCon North America in Salt Lake City from November 12 - 15, 2024. Connect with our current graduated, incubating, and sandbox projects as the community gathers to further the education and advancement of cloud native computing. Learn more at kubecon.io
Fast and Efficient Log Processing with Wasm and eBPF - Michael Yuan, Second State
Log processing is one of the most common cloud native services. Often times, the log processing tasks are handled by dedicated servers in the cluster with heavyweight software stacks (eg the JVM based ELK stack). With the advancement of lightweight sandboxes, such as Wasm, it is now feasible to have Wasm-based log processing applications, written in Rust or Go, directly embedded in the k8s pod. With WASI integration of eBPF, the sandboxed Wasm app can also manage eBPF agents on the host to collect and process kernel and network level logging data. In this talk, we will discuss the latest advances and tools for lightweight log processing. We will demonstrate a complete Wasm application to collect, filter, transform, and route log entries from a MySQL binlog to a Kafka queue. We will discuss how to add an eBPF agent to collect and filter networking data through the MySQL server socket to supplement the binlog data.

[@peacefuldragon1831]

Real use cases for WASM.