Deploy LLMs using Serverless vLLM on RunPod in 5 Minutes

[@Bluedrake42]

Finally a tutorial that isn't awful. Thank you for existing.

[@pagadalasumanth7969]

Oh bro I feel you so much !

[@udaykiran2053]

as you are using the llama model , what is the need for OpenAI installed to check it in the colab Notebook , can you explain

[@nishalk781]

I think he's using openai model for its functions, like that module has stream which will make things easier for you if u need to receive text has chunks, instead of entire text.

[@matthewchung74]

Serverless on runpod with a bigger model, like llama70b on multiple gpus would be awesome!

[@AIAnytime]

Coming soon 🔜

[@renwar_G]

Great video G

[@AIAnytime]

Appreciate it

[@jamesalxl3636]

im trying to run a 70B uncensored model, will this be possible with this method?

[@SohamBasu-b1x]

can we set automated pause and resume in runpod endpoints ? like I want it to run for 3 hours per day in the morning? Can I set that up?

[@premierleaguehighlights9061]

Can i use deepfacelab on runpod?

[@frag_it]

Bro do one for azure Kubernetes with vllm

[@AIAnytime]

Coming soon

[@frag_it]

@@AIAnytime make sure you do a in depth guide would be awesome to learn and apply the llama 3.1 405 B on it. You can even make it a longer playlist ppl would go crazy over it

[@shekharkumar1902]

Sounds like a web promotion. Please create video with agentic based use case example with free of cost llms in local computer

[@GodFearingPookie]

Nothing is free. Money has to come in