NEW WizardCoder-34B - THE BEST CODING LLM

Well, less than 48 hours since the release of Codellama, there is a new model WizardCoder-Python-34B that beats all previous models on the HumanEval benchmark. We will also cover the Phind model that was able to be the second best model.
#wizardcoder #codellama #llama2
▬▬▬▬▬▬▬▬▬▬▬▬▬▬ CONNECT ▬▬▬▬▬▬▬▬▬▬▬
☕ Buy me a Coffee: ko-fi.com/promptengineering
|🔴 Support my work on Patreon: Patreon.com/PromptEngineering
🦾 Discord: / discord
▶️️ Subscribe: www.youtube.com/@engineerprom...
📧 Business Contact: [e-mail]
💼Consulting: calendly.com/engineerprompt/c...
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
LINKS:
WizardCoder: github.com/nlpxucan/WizardLM/...
Phind Model: www.phind.com/blog/code-llama...
Try it yourself: 47.103.63.15:50085/
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
All Interesting Videos:
Everything LangChain: • LangChain
Everything LLM: • Large Language Models
Everything Midjourney: • MidJourney Tutorials
AI Image Generation: • AI Image Generation Tu...

[@engineerprompt]

Want to connect? 💼Consulting: calendly.com/engineerprompt/consulting-call 🦾 Discord: discord.com/invite/t4eYQRUcXB ☕ Buy me a Coffee: ko-fi.com/promptengineering |🔴 Join Patreon: Patreon.com/PromptEngineering

[@OwnOpinions]

"Training Llama2 with Function Calling Capabilities"... I am daily checking your channel if this video gets uploaded. Thanks for all the great content.

[@chaithanyachaganti4305]

Been following your channel lately, very informative and insightful analysis give by you on llms. Keep up good work

[@vincentjean6756]

Yes, amazing. 🎉

[@vishnuitsrocking]

Super excited to see the progress in open-source model! Soon everyone will be a programmer!

[@trpultz]

Thanks for the great video! Really appreciate the thoroughness (especially mentioning the concerns surrounding possibility of overfitting to the HumanEval test set) and you actually putting the models to the test yourself! Looking forward to seeing where these models are at by the end of the year and looking forward to your future videos on them. 😀

[@williammixson2541]

Outstanding content. Instant sub!

[@engineerprompt]

Thank you 🙏

[@sveindanielsolvenus]

Awesome stuff! Where is the paper with the updated GPT-4 humaneval scores? I can't find that.

[@Yewbzee]

Very good point regarding HumanEval overfitting

[@paraconscious790]

great video as always, thanks! can you please make a video for installing this model locally on Mac? thanks! the Vicuna video may not work as the text-generation-webui is changed since then.

[@Nick_With_A_Stick]

I agree with that. I don’t have proof but I have a feeling a lot of these models are writing good code due to good data in the original training, but they don’t understand the code they are writing. A more complex benchmark would help diagnose that, and I think benchmarks should be remade and harder specifically since we are starting to get close to 100% for some of them. God just seeing how good the 34b coder does makes me wonder how good the unreleased regular llama 34 could’ve done if trained with orca. Such a shame.

[@NakedSageAstrology]

Wonderful video sir thank you, could you please do an instructional video tutorial on how to install this on a Windows PC?

[@Artorias920]

amazing updates! End of year prediction: GPT-4 level open source models running on blockchain-esque publicly distributed systems

[@jonsantos6056]

what would that mean for us - free?

[@Artorias920]

@@jonsantos6056 probably something close to it. Something akin to renting your GPU to the network and earning credits that can be redeemed for inference

[@AlexanderBukh]

No need for blockchain, too much overhead and only needed when everyone wants to steal or manipulate data. As for the distributed inference, they already do that, see Petals. Actually, llama.cpp also has means for distributed computational (MPI)

[@STDFme]

I am looking for a llm supporting C/C++, is it suited, too?

[@bmoore813]

Do you have any example of how I could run the WizardCoder-Python-34B-V1.0 on my a100? I ask cause that site is so slow and its takes three minutes to get a reply

[@3lbios]

is the non-python version going to be quantized?

[@theresalwaysanotherway3996]

interestingly enough, people are taking the 34B coding model and fine-tuning it for other purposes like RP or tool usage now (airoboros-34B and samantha-34B being the first ones). It may be worth testing out the 34Bs ability to solve non coding logic problems, to see if the coding knowledge generalises to better overall logic.

[@VR_Wizard]

Very interesting. I woulf love to see the code interpreter compared to the open source .odels and to GPT-4 base model. Havent seen these comparisonst. I havent used code unterpreter yet so i am not sure it is usefull for creating code but I thought thats one of its strenghts so should be interesting to see how much better code interpreter is.

[@engineerprompt]

That’s actually a good point. Let me do a comparison

[@sunfisoft]

does it recognize my project files and respond accordingly and is it necessary to have gpu i dont have gpu

[@SlyNine]

33 billion fits on my 4090 34 normally doesn't launch.

[@user-fm5yy2sx2p]

What is web-ui? is text-generation-webui?

[@lsp0]

Why are none of these papers touching the HumanEval+ benchmark?

[@temp911Luke]

Has anyone tried this in koboldCPP or gpt4all using CPU only in terms of speed ?

[@erikjohnson9112]

5:53 Isn't this a fail because it uses .sort()? That would be an advanced function/methods. Only direct looping and conditionals are allowed.

[@engineerprompt]

You are right, that’s a great catch. Should have caught that.

[@chirwatra]

Do you have a video about how to build or deploy WizardCoder-34B app?

[@engineerprompt]

I don't but you could use streamlit or check out databutton.com. Its a good platform for quickly deploying streamlit apps

[@BrandosLounge]

anywhere we can try this model online? Don't hvae the resources to run locally

[@engineerprompt]

Yes, there is a link in the video description. Responses are slow though

[@jonsantos6056]

can i dl this onto local machine?

[@vincentjean6756]

Yes.

[@bradgalaxy8419]

phind, since its a search engine its likely pronounced "find"

[@Jorsten]

My chatGPT 3.5 solved both of them at first try... We need something harder.

[@rickhoro]

Does anyone know if this would be reasonable to run on a Linux machine with the following configuration: Intel® Core™ i7-7820X CPU @ 3.60GHz × 16 NVIDIA Corporation GP106 [GeForce GTX 1060 3GB] 64 GB RAM Thanks. I'm new to the whole Python and AI world. Help would be very much apprec!

[@engineerprompt]

I suspect, you will not be able to run it. To just run a 4bit version of 34B model, you need at least 16GB RAM. see here discord.com/channels/1104593592229572640/1134401808425549824/1141459504421146777

[@rickhoro]

@@engineerprompt Thanks for your reply. Do you mean 16GB of main memory RAM or GPU RAM? Because I do have 64GB of main memory RAM.

[@pocketsfullofdynamite]

@@rickhoro 16GB installed RAM is good to start depending on how much dependencies are required to run on your local machine. Also check how much VRAM is available. These models take huge amount of VRAM considering your display monitor also uses VRAM.

[@rickhoro]

@@pocketsfullofdynamite Well, as I said, I have 64GB of DRAM (main memory) but only 3GB of GPU/VRAM. What do you think?

[@morsmagne]

I’m skeptical of all those assessments: when it comes to coding what you need is a very large token limit - like Claude 2 with 100,000 tokens. This is because you want the AI to understand all the code that’s written and remember it for the rest of the discussion. It’s no good having a clever AI that’s got the memory of someone living with Alzheimer’s! Because that’s exactly how it feels like trying to do any kind of code work with GPT-4 etc.

[@hemeleh8683]

Exactly! Which model is your preference? Gpt 4 32k is getting quite expensive on my end, would be happy to hear alternatives😅

[@temp911Luke]

I spoke to someone from WIzardCoder and they say the token limit is same as in CodeLLama so the token window size might actually be potentially 100k.

[@morsmagne]

@@hemeleh8683 Personally, I use GPT-Engineer using GPT-3.5-turbo (only because I don't have access to GPT-4 API) then I use Claude 2 (free version) to review the code and make suggestions. However, I'm lucky enough to live in the UK where Claude 2 is free. Therefore, you might need a VPN to access it. You're welecome!

[@morsmagne]

And also, I start a new Claude 2 chat for every change I intend to make because even 100,000 tokens doesn't go very far.