Demo: Rapid prototyping with Gemma and Llama.cpp

Рет қаралды 66,975

Күн бұрын

Learn how to run Gemma locally on your laptop using Llama.cpp and quantized models.
Checkout more videos of Gemma Developer Day 2024 → goo.gle/440EAIV
Subscribe to Google for Developers → goo.gle/develo...
#Gemma #GemmaDeveloperDay
Event: Gemma Developer Day 2024
Products Mentioned: Gemma

Пікірлер: 58

@ayoubachak01 5 ай бұрын

I've used gemma for a benchmark in a research project I'm working on, where I compared human results against AI, gemma was the closest after bloom 176B, followed by models like mistral instruct 7Band llama 34B, even the 2b version did pretty well, great work team 👏🏻

@polish4932 5 ай бұрын

Hi mate, if you'd like to compare diff models for the same question, you do so on Wordware. Highly recommending it! ;)

@ayoubachak01 5 ай бұрын

@@polish4932 thank you

@banzai316 5 ай бұрын

Very cool, thank you! I like this format with demos. We are developers!

@flynnmc9748 5 ай бұрын

This is a fantastic format for a talk, insightful and engaging for a viewer!!!

@GoogleDevelopers 5 ай бұрын

Glad you enjoyed this video! 😎

@MikaailMahammadsirac-q4y 5 ай бұрын

0:21 🎉🎉🎉🎉🎉🎉🎉🎉🎉🎉

@judevector 5 ай бұрын

Wow this is so cool 😎, developers changing the world

@arpitkumar4525 5 ай бұрын

Minimum System Requirements for running a model locally?

@kevinkawchak 4 ай бұрын

Thank you for the discussion.

@ser1ification 5 ай бұрын

Thanks for the demo!

@arpitkumar4525 5 ай бұрын

Really cool and simple to understand

@Daniel-zl7wf 5 ай бұрын

At 9:03, Gemma shows some solid satire

@forrestfeng1098 3 ай бұрын

Like it very good sharing.

@thesimplicitylifestyle 3 ай бұрын

I was looking for this! Thanks! 😎🤖

@voidan 5 ай бұрын

how do you connect the LM Studio to llama.cpp? you used a preset which was probably custom.

@johnkost2514 5 ай бұрын

Wrapped in the llamafile runtime it is an even better single file .. oh yes!

@zencephalon 5 ай бұрын

Good demo, nice tooling suggestions out of this

@parisneto 5 ай бұрын

CODE would be awesome, as well as knowing the SPEC of the notebook as it’s easy to buy a sub1k or 5K+ at apple store depending on so many factors…

@MyEthan1998 5 ай бұрын

If anyone faces an error on Mac about "network error: self signed certificate", close the app and use terminal, run "NODE_TLS_REJECT_UNAUTHORIZED=0 open -a "LM Studio" " This reopens the app and the error should go away. I have no idea where to put this info sooooo...

@TheOrator_Ese 5 ай бұрын

Very nice 👌 cool Demo

@svenkoesling 5 ай бұрын

Just my two cents: 1. No explanation on how to connect LM Studio to the Llama.cpp, 2. newest hardware required - at least it doesn't work on my M1 with eight performance cores and 32 GB Ram

@jadeonrails 15 күн бұрын

Oh, can you share more about "newest hardware required"? I'm thinking about trying on my Macbook Air (M3, 16GB), but I see your comment.

@MacGuffin1 5 ай бұрын

Great choice of demo app!!

@some1rational 4 ай бұрын

Has anyone else tried doing this? I tried following this exactly with LM Studio using the exact model and prompt but I am consistently getting atrocious outputs; the gemma model is just outputting gibberish or incorrectly formatted JSON. I wish there were more details on the presets used.

@monamibob 5 ай бұрын

Very interesting demo! What kind of extra work would be required to run this without LM Studio? Does Llama.cpp contain the necessarry functions to load models as servers you can interrogate?

@tonydevelopingstuff 5 ай бұрын

Very nice!!!!

@ChrisTrotter-oj9du 5 ай бұрын

good, thank you

@digisignD 5 ай бұрын

Cool. Will definitely use this soon

@takudzwamakusha5941 5 ай бұрын

This is so cool.

@andreawijayakusuma6008 2 ай бұрын

did gemma should use GPU ? so I wanna try to learn this model, but I didn't want to use GPU

@A032798 5 ай бұрын

How about windows environment? Is LMstudio/Ollama a better choice?

@bonadio60 5 ай бұрын

Very nice, but what is your computer spec? Memory and chip?

@darthvader4899 5 ай бұрын

Probably m3 max 128gb

@JJN631 5 ай бұрын

Gemma 7b can run on a rtx 4060 laptop

@airhead2741 5 ай бұрын

Is this meant to be super accessible? If I have an APU, on a laptop with no GPU or NPU(?), that means I can expect it to run fairly well? Also considerations for a lighter yet usable model?

@erickcarrasco1938 5 ай бұрын

I tried that in an old APU, very slow generations but the same result.

@郑枨玚 5 ай бұрын

Nah, Gemma is just a parot. It is released for fine-tuning, aka research purpose.

@KuldeepSingh-in6js 5 ай бұрын

cool demo

@indylawi5021 5 ай бұрын

Very cool demo 👍. Any chance we can get the source code 😀

@awakenwithoutcoffee 5 ай бұрын

where can we learn to set this up ?

@nayzawminnaing2562 5 ай бұрын

That's a lot of RAM to run this for me.

@deeplearningpartnership 5 ай бұрын

Awesome.

@devagarwal3250 5 ай бұрын

Pls provide source code also

@tandaramandaraba 5 ай бұрын

wow

@AIPeter-dd9hr 5 ай бұрын

game using lm studio, interesting.

@emmanuelokorafor1705 5 ай бұрын

It's cool now, but what if each application starts deploying local models. It'll turn our phones into what data centers were meant for thereby reducing costs for large corporations. Trading a few megabytes for faster and more expensive chips.

@cmoncmona959 5 ай бұрын

Please Elaborate. What were data centres meant for? Asides hardware to run inference of worldwide requests. If it’s done locally, surely it’s better for redundant tasks. Also, the data centres use a lot of megabytes and expensive chips.