Phi3 with ONNX Runtime at the edge

Use Olive and ONNX Runtime to efficiently inference Phi-3 model at the edge for mobile and Web applications
Part 1: Getting optimized models for mobile and web platforms Optimizing Phi-3 for chosen hardware with Olive​: github.com/mic...
Part 2: Running phi-3 on Android phone Android chat app with Phi-3 and ONNX Runtime Mobile: github.com/mic...
Part 3: Running phi-3 in the browser Web chat app with Phi-3 and ONNX Runtime Web: github.com/mic...
Blog: Enjoy the Power of Phi-3 with ONNX Runtime on your device​: huggingface.co...
Learn more
Olive: microsoft.gith...
ONNX Runtime: onnxruntime.ai/
ONNX Runtime Generate API: github.com/mic...

[@l.halawani]

Great demo, thank you!

[@cocooooooooo9150]

Is there an example of using phi3 vision model on Android?

[@ONNXRuntime]

It's coming!

[@MrBigdogtim69]

Great content and exciting to see! Is there an example using Maui?

[@ONNXRuntime]

here is a simple example of ORT mobile for Maui. github.com/microsoft/onnxruntime-inference-examples/tree/8dc4650a77b849fb706bee1bdaa4dda91a52511d/mobile/examples/Maui/MauiVisionSample

[@MrBigdogtim69]

@@ONNXRuntime Thank you!