Rust Ollama By Examples - 4 Chapters (Full Tutorial - Rust AI)

[@JeremyChone]

IMPORTANT ollama-rs 0.1.6 introduced a breaking change (from 0.1.5) where stream.next() now returns a Result instead of a single GenerationResponse. The fix is straightforward. It has been implemented in the main branch and tagged as E01-01 ( github.com/jeremychone-channel/rust-xp-ollama/tree/E01-01 )

[@christerpherhunter930]

Please do not stop producing these high quality videos!

[@mr.daniish]

one of the top channels on the planet for rust and llms!

[@janiscakstins2846]

This is one of the best Rust channels out there. Keep making these videos!

[@MaxJM711]

Your videos are incredible Jeremy! I decided as one of my main New Years resolution to properly learn Rust this year and your channel has been nothing short of a miracle for me. Thank you so much for everything bro! I am doing my thesis next year and I wanted to make it about ML + Cybersecurity. I was thinking about making it all in Python (of course lol), but I may eventually use Rust for this in case the tooling is ready for something like that

[@romanstingler435]

"Wrong answer because we know Rust is the best one" YES I like that :)

[@GlebEagle]

WOW!!! That's very interesting topic! Thank you!

[@abdulcodes]

Absolutely adore your content! Could you whip up a dazzling video on grpc tonic? How about crafting a mesmerizing e-commerce backend with microservices? Picture this: a seamless connection between a user app and an admin app, all beautifully showcased in the same video. Heartfelt thanks for your incredible contributions to the Rust community!

[@JeremyChone]

Thanks for the kind words. On gRPC, I usually lean more towards json-rpc than grpc. I believe grpc serves its purpose in specific niches but has been overgeneralized. All the functionalities you described will work well with json-rpc too, offering increased versatility and virtually the same performance for those use cases. grpc becomes necessary when microsecond (not millisecond only) response times are needed, and when parsing speed is critical to the processing speed. For all other scenarios, json-rpc is more versatile that its overall value is higher for most use cases IMO, contrary to popular belief and momentum. Check out these videos: - Overview and insights: kzbin.info/www/bejne/fZSYf51rga-YZ7c - SQLBuilder / ModQL (related to json-rpc): kzbin.info/www/bejne/Y5WweWyLntx-p8k - Here's a mini high-level query spec I've developed for json-rpc: joql.org Now, if your team is really set on using grpc, then this might not be a discussion worth having, as grpc will get the job done, and with technologies like tonic and others, it could work ok.

[@abdulcodes]

Thanks a lot, will try to play with json-rpc.But for the front end part for the majority of frameworks , they do not have support for json-rpc currently than they have for grpc

[@maxali6591]

Continue to teach us. Great thank you!

[@prashlovessamosa]

Hey Jeremy thanks for making awesome stuff for us.

[@ItaloMaiaTM]

25:49 - Makefiles are also a nice option for that.

[@ItaloMaiaTM]

Really nice video, much appreciated!

[@brianlogan4740]

Ah yes thank you! Exactly what I needed!

[@sunitjoshi3573]

Thanks a lot for taking the time to put this out. Are you planning a Rust Web Dev course or a Learning Track? I would be very interested.

[@JeremyChone]

So there is couple of videos about web dev. The Rust full Axum course, and the Rust Web App production coding.

[@zeroows]

Thank you.

[@HagbardCelineOnRust]

Thanks for the video. I liked and subscribed 😛

[@mjcc_]

Hey, great video Jeremy ! Is it possible to make a video to how to create a plugin for obsidian with this rust-ollama project ? I think it can be a realy cool comunity project too 😉

[@ItaloMaiaTM]

== nevermind; saw your warning message; by the way, you could have used transpose to work around the api change. Why did you opt out of doing so? == At 15:16, `let res = res?;` worked here without the need to use map_err. Regardless, enjoying the video by buckets! I'm using llama2 model instead of the suggested one, so to save a bit in space. The docker Ollama docker container is working quite well too! Back to watching...

[@JeremyChone]

Ho, cool, you might be right, i will check the code.

[@JeremyChone]

Ok, yes, now we can use `let res_list = res?` without the `map_err`. It was leftover from ollam-rs `0.1.5` when it returned a single GenerationResponse and I think did not implement the standard output. I've updated the codebase. Also, I added a note because I think the handling of `Vec` might not be correct. It's a corner case, as the vector will mostly be of length 1, but it should still be fixed. I just need to understand what it means to have a vector length > 1. Here is the ticket I logged about the ollama-rs change: github.com/pepperoni21/ollama-rs/issues/20

[@abobunus]

Is there a way to make your own language model with Rust? Train it and then use it as you showed in this video.

[@백영승-h5d]

너 잘하네 재능있어 열심히해🎉

[@cathyaishu1179]

Does it support function calling?

[@JeremyChone]

I do not think it does yet. You can work around it with some instruction and clear schema (even TypeScript schema work). But dfferent flow.

[@eliasp.2759]

It does now

[@cthutu]

You will need an nVidia GPU to make use of the GPU acceleration. AMD is CPU-only :(

[@JeremyChone]

Works ok on my Mac apple silicon.

[@cthutu]

@@JeremyChone Unfortunately, my Linux-based PC with an AMD GPU (which is the best for Linux games) is not supported :( It falls back to CPU.

[@prashlovessamosa]

Firsttt🎉 finally