AI Q&A with Falcon LLM on FREE Google Colab

[@Anzeljaeg]

omg WHAT a goooooood news and of course from our fav AI coder !!! ty bro

[@1littlecoder]

Thank you bro 😍

[@Imran-Alii]

@Your AI Q&A video on "Falcon LLM on FREE Google Colab" was outstanding. Thank you for simplifying complex concepts and providing valuable insights. Great job!!!!!

[@hitlab]

Before bothering to try the 7B model I wanted to see how good the 40B model was because if the 40B couldn't execute successfully on what I wanted it to do, I assumes that there was no possibility that the 7B model could do it. I tested Falcon 40B on Hugging face to see its capabilities in order to determine if it was worth the time to try and set up on Runpod and use for my POC. I have to admit that while there do seem to be a number of use cases where Falcon40/7B and other open source LLMs are quite impressive there still is at least one use case where every LLM I've tested so far including Falcon 40B fails where GPT absolutely crushes. This is a really simple game test I put together as a prompt. It goes like this: "Here is a simple game we will play. I will tell you what color square you are standing on and you will take an action based on these simple rules: if I say you are on a white square you will turn right and move 1 square forward. If you are on a black square you will turn left and move 1 square forward. After you take the action you will ask me what color square you are on and I will tell you and then you will take another action and so on. You will keep track of the number of colored squares you land on and report the tally after each action. If you land on red square you will encounter a wizard that will try to turn you into a mouse. When you encounter the wizard you must "roll", i.e. generate a random number between 1-10. If you get a number that is 2 or higher his spell will fail otherwise you will be turned into a mouse and the game ends. Do you understand? " GPT played the game flawlessly. I want to extend it into something way more complex and use the GPT API in a Unity based game to represent the actions as being taken by an agent in the game etc. I'd like to avoid using GPT however due the cost of using the API and instead use an open source model. But again, I have not found any model that can successfully execute the game task I outlined above. Does anyone have any suggestions? Maybe someone knows of one of the models discussed here or elsewhere that might match GPT in this regard. Thanks in advance.

[@ilianos]

Exciting use case! I'd be curious to know how your experience was (now, a few months later), when trying this with smaller open source models?

[@signupp9136]

Unfortunately, to date I have still not found any open source model at any size that can pass my reasoning questions. Many of the reasoning questions I see posed on KZbin channels out there are softball questions IMHO (and these benchmark websites are useless with respect to everyday applications/diverse use-cases). I think open source will eventually get there, but right now its a toy and a means for people to create KZbin followings by presenting these toy models in an entertaining way. Something to do in the meantime I suppose while we wait for an open source model that can compete with GPT4. I would love love love to be proven wrong. I've put this challenge out there on many forums and not once has anyone ever been able to suggest an open source model that can reason as well as GPT4. Let me be clear that my goal is to use a model with sufficient reasoning ability so that I can use it in production as in a SAAS business. As good as some of these open source models can be shown to be under the right conditions, i.e., right questions asked, right GPUs, right parameter count, right prompt engineering, they are still not reliable for production and therefore are still just toys, for-entertainment etc. If you want to make money at this point, you'll need to use OpenAI LLMs (maybe Claude) but if you are not really interested in that and want to play with opensource (that's fine. no judgements here) or engage with open source as entertainment and hope you're playing will possibly pay off commercially, then just be clear that you have a while to go before you'll be able to do anything production ready and reliable. One one gripe/judgement however is with these KZbinrs that I see as people selling shovels outside a fool's gold mine. I don't think they are being forthright. I include 1littlecoder, Mathew Berman, and others who are not being explicit about what these opensource models really are and telling people upfront that these are toys, don't get too excited and don't spin your wheels in opensource, if you want to start a business with LLMs, while people are riding the wave of success with OpenAI. @@ilianos

[@Mahmoudlk]

Thank you a lot for this video and the free tutorial. It was very helpful ! Keep up the good work man !

[@Endlessvoidsutidos]

been trying to get this model to work for days on colab thanks so much for your awsome VIDDD :D

[@1littlecoder]

Glad I could help!

[@curiousstreamer4062]

hey can you explain how to make LLMs do arithmetic (Lang chain connected with ripple enviroment)

[@wa3044]

Nice work, I tried your Colab, and set temperature=0.01, the answer was "Nile River", and since its from UAE I thought it will support Arabic language, but it does not.

[@blisphul8084]

The fact that this is not multilingual makes this far behind the proprietary models. Of the fast models, I grade performance as Bing Chat Balanced (GPT-4 Ultra-turbo)>GPT-3.5 Turbo>Claude-instant>Bard>All Open Source models I've tried. Every single proprietary model I've listed does multilingual reasonably well, though Bard is pretty bad overall.

[@blisphul8084]

I will need to see where ChatRWKV is at, since that was the most promising model from my testing.

[@1littlecoder]

I liked Raven (ChatRKWV)

[@lakshsinghania]

sir, i dont understand this one thing fine tuning means we change something in that pre-trained model and inference means just using that model right ? because when i surfed the web page " fine tuning a pretrained model" on hugging face website i dont understand the trainer api and other stuff, pls can u make a video on this

[@1littlecoder]

Yes that's correct. Check this out kzbin.info/www/bejne/oJ-zi3SmrNl4aqs

[@jeffwads6158]

This model is painfully slow unless you have a few A-100's at your disposal. Hopefully they can rectify that soon, but I wouldn't hold my breath.

[@1littlecoder]

Did you try on T4 or some other machine?

[@nickjain4992]

very nice video !!! I was wondering if we can you use this model with llama and langchain. I tried to use it but keep running into problems. It will be great if you could look into it

[@1littlecoder]

You honestly read my mind and here is the video - kzbin.info/www/bejne/o3LSf3SEhbSmmpI

[@ppbroAI]

Trainable with qlora?

[@MasterBrain182]

Astonishing content Man 🔥🔥🔥 🚀

[@machinepola6246]

What is the temperature set to? If the temperature set to more randomness then these answers would come. The model is not good incase it was deterministic.

[@1littlecoder]

You are right, there is one more sub who checked with zero temp and the answer is accurate

[@spinninglink]

Can you use this method for any model on huggingface?

[@JainmiahSk]

what is difference between falcon 7b and falcon 7b instruct?

[@prakashj3436]

Sometimes or Always Falcon 7b Instruct model is gonna work well compared to Falcon 7b, falcon 7b is kinda a raw model. Whereas, Falcon 7b Instruct is instructed to jobs perfectly.

[@epotnwarlock]

how can you train the falcon llm?

[@manon-francais]

Apart from OpenAI, open source LLMs are still in the back... They are better, but just fine, finer.. More and higher quality training is needed... But this is how usually everything in life builds up, hopefully it doesn't take a long time to see a real competitor to GPT-4 (or 5 lol)

[@VincentNeemie]

GPT-3 has 175 billion parameters, you are comparing it with a LLM that has 7B. Most people now day are trying to compress and be efficient with less.

[@KevinKreger]

1 Little Coder rules the AI knowosphere 😍

[@1littlecoder]

Thank you Kevin

[@jekkleegrace]

Awesome video such great news

[@1littlecoder]

Thanks for the feedback!

[@byspec3239]

Hello man, love your videos! Can you help me with one thing? One video of yours developing a chatbot with hugging face + llama-index really helped me to achieve something, but when I try to work with LOTS of text it doesn't work and I can't figure out why. Would love if you have any hints in how to make it work if its even possible

[@debalinaghosh6404]

thank you

[@1littlecoder]

You're welcome

[@muhamadabdallah7960]

an easy tutorial about dubbing a video with facebook's NLLB-200 and MMS PLEASE ! thank you .

[@1littlecoder]

It's definitely in my backlog

[@doords]

We need an API that we can connect to this from other applications

[@shekharkumar1902]

It seems the model is performing bad.why to use it then ? But nice explanation LilCo

[@1littlecoder]

The idea is primarily to identify what kind of tasks this is good at. Look at more narrow use-cases than generalized solution. Also to help provide a boilerplate code for larger size model.

[@MythicMinds537]

there was a warning about this model isn't available for text generation it still Runs HOW?. 😂

[@Chemisynthstrucplexifyimunosys]

Thanks

[@satyamraj2779]

It is not Yellow river, its nile......

[@1littlecoder]

Correct!

[@NetanyaAvabel]

i cant. it errors

[@1littlecoder]

What errors?

[@manuelrech1262]

this guy is too fun

[@1littlecoder]

Thanks

[@starbuck1002]

the yellow river, lmao

[@1littlecoder]

I checked after the video. It seems it's the 2nd longest

[@starbuck1002]

@@1littlecoder It's apparently the second longest in China, but tbh at first I thought this one was completely ridiculous :D

[@1littlecoder]

@@starbuck1002 Same here. I was like Yellow river. Does that even exist.

[@swapnilchavan7076]

Moon is not a star. It's a earth satellite. But Elon Musk don't know this. That's the joke.

[@1littlecoder]

Got it!

[@epictetus__]

Elon Musk is the Rakhi Sawant of tech world

[@surajkhan5834]

Hi i use to run falcon 7b if i run this below code pipeline = transformers.pipeline( "text-generation", model=model, tokenizer=tokenizer, torch_dtype=torch.bfloat16, trust_remote_code=True, device_map="auto", ) it almost took 40gb + disk space now i am almost full of disk space how to delete that?

[@greenTech88]

Falcon LLM

[@joanapaulasoliveira]

🙏👍

[@1littlecoder]

🙏🏽

[@DevasheeshMishra]

first comment

[@1littlecoder]

Thanks!

[@LearnWithBahman]

Yellow river is not even longest in China. So use open ai API , not these models.

[@gurekodok]

When creating the pipeline, I got : "ValueError: Could not load model tiiuae/falcon-7b-instruct with any of the following classes: (, )."

[@1littlecoder]

Where did you create? On colab ?

[@gurekodok]

@@1littlecoder yes from the collab link at the description

[@1littlecoder]

Can you try this and see if this works? kzbin.info/www/bejne/o3LSf3SEhbSmmpI