Run phi-v2 Mac | step by step | m1 | rust | cargo | candle

Hi everyone, I recently saw microsoft's new phi v2 model. and instantly I went out on internet looking on how to run it on my mac m1. After few hours on internet I found solution and I summed up everything into 5 minute video on installing and running phi-v2 on your machine locally.
It is about running quantized gguf model on machine with metal support.
Command 1: "curl sh.rustup.rs -sSf | sh"
Command 2: "git clone github.com/hug..."
Command 3: "cargo build --example phi --release --features metal"
Command 4: "./phi --prompt "hello, how are you? Assistant: " --model 2 --quantized"
Instruction Format for phi-v2
Instruct: {prompt}
Output:
Follow video step by step along with above mentioned commands.
Connect with me:
🔹 FaceBook : / proogramminghub
🔹 Twitter : / programming_hut
🔹 Github : github.com/Paw...
🔹 Discord : / discord
🔹 LinkedIn : / programminghut
🔹 KZbin : / programminghutofficial
phi,phi v2,phi-2,install,mac os,gguf,quantized,metal,rust,cargo,step by step,mac m1,mac,m2,arm,silicon

[@alexpro5778]

Also, it's not clear how to load another model. For example, I want to use a 8-bit quantized model instead of 4-bit.

[@alexpro5778]

Hi bro. Thank you for the detailed instruction. It was very helpful. Can you please explain how to run this model similar to LLAMA CPP in the instruct (chat) mode? It would very useful. And one more time, thank you. There is not enough information on the subject.

[@programming_hut]

Llama.cpp has support for phi-2 now! Check out there pull request section I think it is not merged yet. Also there is separate branch with name of gg/phi-2 you can checkout that to run phi on llama.cpp

[@alexpro5778]

@@programming_hut Thank you for the suggestion. Yes, I managed to run the model with LLAMA CPP, but the output is a garbage. Unfortunately ;-(

[@programming_hut]

i ran it this morning, it was working fine! Try using proper prompt, or use ./server

[@somalingakoushikmatheeswar]

Hi I am getting an error while running the PHI command: invalid peer certificate: UnknownIssuer

[@subhangmokkarala]

hi bro discord link expired