Fine-tune my Coding-LLM w/ PEFT LoRA Quantization

Coding-LLM are trained on old data. Even the latest GPT-4 Turbo Code Interpreter (CI) has a knowledge cut-off at April 2023. All AI research from the last 7 moths are not in the training data of commercial coding LLMs. And RAG lines of code do not help at all, given the complex interdependencies of code libs.
Therefore an elegant solution for AI researcher is to fine-tune your own Coding-LLM on the latest GitHub repos and coding data. Which is exactly the content of this video: How to fine-tune your personal coding-LLM (or a Co-pilot like Microsoft's GitHub co-pilot or any CODE-LLM like StarCoder).
#ai
#coding
#pythonprogramming

[@javiergimenezmoya86]

This channel has supreme quality.

[@sup5356]

Outstanding content, beautifully conveyed

[@FaizanKhan-x1g9u]

Thanks for sharing the knowledge. ❤

[@unclecode]

I am at 00:02:40, already super excited for what you decided to create a video. Amazing. Thanks

[@unclecode]

I finished the video, extremely appreciate, the way you put all these valuable information together. Looking forward for part 2.

[@shayanshamsi7540]

Thankyou for this amazing video! Could you please make a coding walkthrough for soft prompts?

[@daryladhityahenry]

Hi! I have a question about deepspeed or accelerate library from huggingface. It said that can make a portion of the model to be trained on, so it can be used across multiple GPU, then.... Can it also become a scheduler? I mean, if yes then we can just using 1 GPU, just train for a longer time for many sequence of training batch that should be spread across GPUs. I hope I can use my own GPU to train @_@.

[@idoronen9497]

Appreciate the video! Can I apply the same technique to fine-tune a model like LLaVA?