Rust genai - Multi-AI Providers Client

[@squarecloud]

Jeremy, thank you for your contribution in bringing AI tooling into Rust. Right now I’m busy working on a project that isn’t related to AI, and it constantly feels like I’m missing on something more important but thanks to your videos and open source projects I know well where’s my entry on AI field! Amazing things, incredible flexibility in Rust!!! Thank you ❤

[@ruturajzadbuke9646]

Amazing! I was planning to write mine the coming weekend. Now I can use this one, at least as a starting point. Thanks Jeremy!

[@ShaunPrince]

This is the innovation that we need for AI apps.... The playing around and experimenting in python has hit a plateau, time to build some real apps now!3

[@yaanno]

Fantastic! Thank you for this yet another great series! I've been trying to learn Rust by building a client/consumer app utilizing AI tools to connect travel APIs and user queries by voice. Even though no success yet (just learning stuff) but the application of your library will be superb. Merci :)

[@JeremyChone]

Thanks, voice might come later in the context of chat api. If you need text to speech, the async-openai has a great support for open AI TTS.

[@norminemralino2260]

Another great tutorial Jeremy. I was able to follow along relatively easy. One small suggestion is to maybe use the dotenv create in your example. Anyway great job. I'd love to see a RAG example using Rust. I don't think I've seen one. I normally see RAG implementations in Python

[@JeremyChone]

Thanks, in the crate examples/ there is an example of Auth Resolver which allows providing a custom env name or even a resolver function. Yes, later I will be showing RAG in Rust. It might have SQLite for the content, lancedb/vectordb for the embeddings, and it will actually show that the embeddings/vector search can be optional when we do RAG.

[@JFaleiroJR]

Lower level that this, but I wonder what is in the making for a specialized machine language accelerator, friendlier than melior - I am considering putting a project together on that space.

[@ItaloMaiaTM]

Another nice video, Jeremy! Would you have any material to suggest on how to use genAI to solve objective problems?

[@JeremyChone]

Thanks, I am going to create more content with GenAI, and AwesomeApp will be a desktop app using it. In the enterprise/productivity app context, I created a high-level video UI to HI that outlines some of the opportunities. Function calling, which can translate a user request to a function name/params to be called by the application, can be very powerful. I will try to create some concrete examples of this.

[@user-zz6fk8bc8u]

I think some image things would be great. Especially stable Diffusion. Would love to use Rust for that instead of python or some GUIs

[@JeremyChone]

Yes, I am planning to add images as far it is supported by the chat apis.

[@froop2393]

No! Super cool 😊 I really like it!

[@TheMsksk]

Love it. Thanks!

[@ИванРагозин-я8я]

Would you please add Groq

[@JeremyChone]

Oh, yes, good one. I knew I had one important one left to do, but somehow I could not remember the name. Thanks, I will add Groq. Their API should be similar.

[@JeremyChone]

Groq has been added to the main branch. I am pretty amazed by their pricing, so cheap! Definitely a must-have for GenAI.

[@automatalearninglab]

Yessss, super neat! :)

[@ZiRo815]

What’s the thinking behind modelling the models as strings instead of an enum?

[@JeremyChone]

To avoid having a fixed list of model names that needs to be updated. This is especially important with ollama, which allows the installation of a wide variety of models. Eventually, I might have Model(Arc) type, but this will just be a wrapper for the string to make it a little more efficient with repeating the model name in the stream event (not present for now)

[@BrazenNL]

Any chance on Copilot?

[@JeremyChone]

I might have missed something, but I think Copilot is one level above the generative/AI provider. I believe they use the OpenAI model, probably wrapped in an Azure service. So, adding the "endpoint variants" such as Azure OpenAI endpoints, AWS Bedrock endpoints (for the open models and Anthropic/Cohere), and Google Vertex AI are part of the plan. The adapter infrastructure is now in place. Now most of the differences should be around the auth schemes.

[@BrazenNL]

@@JeremyChone Sounds like a plan! Looking forward to your thought process in the next video. Thanks!

[@meka4996]

Amazing! Thanks

[@cunningham.s_law]

not sure how I feel about the implicit key reading from ENV great crate tho

[@JeremyChone]

You can provide your own with resolver per adapter kind.

[@parabolicpanorama]

cool, but what if you want to roll your own model and not just ping an API for a response. You could do all of this in any language..