Meta Llama 3.1 405B Released! Did it Pass the Coding Test?

[@proflead]

Thanks for the tests! :)

[@Max_Moura]

Hey Mervin, thanks for sharing your insights on the new Meta Llama 3.1 model! 🤖 I'm blown away by its capabilities, especially with the context length of 128,000 tokens and the fact that it can perform multitasking with logical and reasoning questions. I'm also excited about the potential applications of this model in real-world scenarios. One thing I'd like to suggest is exploring the use of this model in conjunction with other AI tools, such as computer vision models, to create even more powerful and integrated AI systems. Keep up the great work, and I'm looking forward to seeing more videos like this! 👍 (This comment was generated by Llama 3.1 70b. I hope you like it)

[@davidtindell950]

Thank You. Great Review and Impressive Tests !!!

[@h.h.c466]

What is the benefit of using the multiple agents in task like 11:56? I see that encapsulation make sense so that no crosstalk happens , but this could have been done in like a succession of chat entries as well. Do you use specialized/Fine-tuned LLM for agents?

[@lalamax3d]

bhai, bahut aala hai..........

[@wardehaj]

Great video again, thanks! Can you please make an instruction video about installing routeLLM on an Android phone using termux and using the llama3.1 8b local and groq/chatgpt/claude through internet?

[@sundarrajendiran2722]

Hi Mervin, I have tried extracting fields from pdf document using both llama3.1 70b and 405b models using toolsconfig(in toolsconfig I am passing schema to get json response) sometimes I am getting incomplete response. Meaning out of 40 fields sometimes I am getting less than half of the fields. Note: I am accessing models via AWS bedrock. With this context do you have any suggestion that I can try?

[@miguelsalcedo01]

ok i stand corrected maybe i'm not supposed to create a virtual environment for pip installs with brackets? I just deactivated the virtual env and ran the pip install directly and it worked no errors and loaded site. I digress

[@loryo80]

i have a lot of problems when i launch praisonai ui :It appears you've shared some log messages and warnings from a system or application startup. These messages indicate a few things: The system is using gRPC with some experimental features enabled. There are several warnings about an SDK being disabled. There are multiple warnings about translation files for "fr-FR" (French) not being found, and the system is defaulting to "en-US" (English) translations. There are warnings about API keys not being in a valid OpenAI format. There are multiple warnings about no default IOStream being set, defaulting to IOConsole. These messages suggest that the system is starting up with some configuration issues, particularly related to localization (French translations not found) and API key formatting. The SDK being disabled could also impact functionality any help

[@MervinPraison]

Which OS are you using ?

[@loryo80]

@@MervinPraison I'm using windows 11

[@mohamedkeddache4202]

what are PC requirements to run the 405b version locally with ollama?

[@GoysForGiza]

a spaceship

[@JoanApita]

thats a good questions. The 70B models cant even move my mouse when i tried to run it.

[@GusRJ70]

In other vídeo, this guy told that it do not run even with his 96gb of RAM.

[@dosky5w7]

about 256GB RAM on q4 and don't expect more than token a minute. super low quant would run on 128GB RAM but would most likely considerably degrade quality while still taking forever to output and ollama doesn't even offer such option. You might want to use mistral large 2 instead

[@syedabdul4515]

Does installing praisonai , expose my api keys or any codebase that i upload.

[@miguelsalcedo01]

maybe I'm doing something wrong but it's really frustrating when you create a virtual env the go to pip install and you get red all over your screen with dependency issues. It's my understanding that pip tools will allow developers to lock down exact package requirements needed to run so we don't have to try and fix all the dependency issues. It's like a freaking rabbit hole, fixing one package break another fixing that breaks another.... and really just drives people away from the good work people are doing. Just really hair pulling frustration.

[@MeinDeutschkurs]

First of all, I appreciate you returned to your initial style. Please let sponsors on your platform. Maybe after the first third of your video, and everything is fine. Now to the topic: it should be good on large context summaries. But it isn‘t, 70.000 tokens to summary fails on M2 Ultra 192GB Ram. (About 150 GB useable VRAM). It just outputs jibberish. 8B unquantized, also with Q4. Most of the creators are celebrating the 3.1 version as the open source competitor to GPT4o or Anthropic‘s Claude 3.5 sonnet. It isn‘t. If so, then in particular/certain tasks. Nothing else. And geoq is restricting the context window to 16000 tokens, so it isn‘t comparable. I wrote to groq, and they said that it is not possible to provide that context window on their large scale. So what? Kismet, bad model.

[@bgriffin5447]

Best open source title for 24 hours .. mistral 2 got no chill

[@john_blues]

You lost me 1 minute in. You are reading those charts wrong. 3.1 is only better in 7/15 evaluations, not "most". And you're reading the Human Evaluation bar chart wrong. 3.1 loses more than it wins against Gpt 4o and 4-0125.

[@ZuckFukerberg]

Nice video! You seem very knowledgeable on the matter

[@mrinalraj4801]

Thanks for the video 🙏 It's highly informative. Just one thing please remove the intro sound effect. My headphones exploded

[@MervinPraison]

Sorry for that I decreased the volume a little for the blast. Is this better now ? kzbin.info/www/bejne/bIGYkIqmq7Zpf6s

[@emmanuelkolawole6720]

Llama 3.1 8b parameters failed my data analytics test, Mistral nemo passed all my data analytics test

[@morespinach9832]

@@emmanuelkolawole6720 what do you do in this test

[@commoncats5437]

bro create a tamil dataset and best llm for tamil

[@pratikkumar939]

🎉

[@cloudshoring]

"Amazing Mervin" - Please add some sound/ music in between the scenarios when you transition.

[@neoglacius]

no

[@jackflash6377]

Why? We don't need music, we need knowledge.

[@mrd6869]

Go eat cheeseburgers.

[@fkxfkx]

I'm getting ready to drop that annoying second L

[@saabirmohamed636]

This is called "metai" (the indian sweet) and everyone loves metai

[@madushandissanayake96]

However 405b version fails on answering following types of tricky questions. Q :- In 2023 Tony sold all of his vehicles and decided to not to buy single one again. However in 2016 he purchased two cars worth 9000$ and 7000$ each. In each year the price of has fallen by 10% of its value. However on July this year(2024) he was short of 7000$ to buy a home. Can he buy the house by selling his cars?

[@morespinach9832]

It also fails on very basic things like data extraction from unstructured docs. Uselsss marketing.

[@mrd6869]

​@@morespinach9832.sounds like a user problem buddy.you realize prompt engineering is a thing.

[@mrd6869]

​@@morespinach9832.sounds like a user issue.

[@john_blues]

That's a badly written word problem.

[@madushandissanayake96]

@@john_blues That is the point. Claude 3.5 sonnet can answer this problem without any issues.