Fine-tune GPT-3.5-TURBO - A Crash Course

[@Writingsfromred]

Bro I've been bingeing all your videos theyre so good keep it up the views will come

[@codu]

Beast! Nice one Ronan. 🙌

[@EntreEden]

very helpful, thank you

[@MohammedAbdulatef]

just pefect!

[@rbrowne4255]

great job on this video!!! Is it possible to do inferencing or fine-tuning on a system with multiple GPUs ? if so do you have to use a specific library or framework?

[@TrelisResearch]

thanks! do you mean inferencing/fine-tuning of an open source model? Say Llama 2? For inferencing, text-generation-inference automatically uses multiple GPUs, if available, demo here: kzbin.info/www/bejne/iKDVY6uFhKZgjKc For fine-tuning, I haven't done it yet with multiple GPUs. Sagemaker (AWS) is probably one way to do it. The HuggingFace/transformers are not as mature for doing multiple GPUs.

[@rbrowne4255]

thanks!! I recently came across frameworks like Ray or Deepspeed which could help with Multi-GPU inferencing, given as you stated most of the transformers are not mature for multiple GPUs, does this mean that there will be a trend towards smaller models...my primary concern is scaling and user access concurrency

[@TrelisResearch]

@@rbrowne4255 seems to me that both small and very large will be important. Actually the transformers support for inferencing on multiple GPUs is not bad, support for training on multiple GPUs is poorer, specifically for the transformers library. Sagemaker I believe have lots of options out of the box.