Get started with Microsoft's Phi-2 on your laptop

In this video we explore how to run phi-2 on a local dev box. We try the raw model and a DPO fine-tune from TheBloke.
Raw model on HuggingFace - huggingface.co...
DPO fine-tune by TheBloke - huggingface.co...
Llama.cpp - github.com/gge...
Airtrain.ai - airtrain.ai
#microsoft #artificialintelligence #llms #phi2 #chatgpt #gpt4

[@ffox3123]

Thank you for making this tutorial! It is very apprecaited. Can you recommend a good discord server to ask questions on topics related to this? Also, do you know if there is a way to send prompts via API? So I can make a PySimpleGUI UI for this?

[@AirtrainAI]

Here are a few Discord/Slack servers: - LLMOps.Space discord.gg/aAxBAZYA - Airtrain Slack join.slack.com/t/airtrain-ai/shared_invite/zt-252ipfh5j-ljAxJm4P0XCLAZXpOyKH1A Regarding a Phi-2 API. I don't know one off the top of my head, but we might offer it soon on Airtrain.ai. Stay tuned.

[@khushidhar1619]

I did the entire process but it's saying phi-2 not found

[@divyateja939]

Thank you for the Video. Could you please help with the hardware requirements(CPU, GPU etc) to run Phi-2.

[@AirtrainAI]

With only 2.7 billion parameters, Phi-2 is a small model. If we want to load its parameters as fp16 (no quantization), we need at least 5.4 GB (2 GB per billion fp16 parameters) of GPU VRAM. If you do not have this available, you can try one of the quantizations offered by TheBloke: huggingface.co/TheBloke/phi-2-GGUF/tree/main

[@ajayjasperj]

getting same response everytime

[@AirtrainAI]

Hmm, strange. Did you use the raw model or the DPO fine-tune?

[@ajayjasperj]

@@AirtrainAI lm studio gguf