Bringing AI to the Heterogeneous Edge with WebAssembly & ONNX - Francisco Cabrera & Ralph Squillace

Bringing AI to the Heterogeneous Edge with WebAssembly and ONNX - Francisco Cabrera & Ralph Squillace, Microsoft
Workloads are moving closer to where the data is created. However, running critical training and inference apps on the edge presents multiple challenges, including the heterogeneous environment of devices (with different chips and operating systems), security concerns, and resource constraints. Moreover, the AI landscape is quite complex, with multiple SDKs, libraries, GPU acceleration technologies, and new model innovations emerging every week. This complexity makes it challenging to deploy AI workloads to multiple edge devices and locations. WebAssembly + Kubernetes is one of the most promising solutions for achieving simple multiplatform development and orchestration. ONNX is a open ecosystem providing an open-source format for AI models so AI developers can choose the right tools as their project evolves. This session will show how to develop ONNX models, compile them to WASM, and deploy them to multiple devices without the need for specific platform targeting.