What is Quantization? - LLM Concepts ( EP - 3 )

[@iFastee]

when will you show all this process in code bro?

[@AkhilSharmaTech]

Hi bro, mistral 8X7B running on colab video I've made. It uses a quantized model and uses parameter offloading Before I release that, i thought why not explain these concepts Otherwise that video won't make sense

[@AkhilSharmaTech]

Actually let me do one thing. Today's video is about post training quantization . Let me show you some code examples in that

[@iFastee]

@@AkhilSharmaTech nice... i'm going through some simple raw implementations of RAG with quantization so then I will revisit you to check the usages of stuff that is not raw in that process like ollama, langchain, hugginface etc... whatever u will use