Large Language Models on Raspberry Pi

This video was made using zoom & the summary used zoom AI.
This video explores using AI (Artificial Intelligence) on the computer, Raspberrry Pi, Model 5. I explore Large Language Models and focus on Microsoft;s Phi2.
Music:
Composition: AVAI: AI music
Arrangement: Philip using Ableton Live
Here is its AI summary:
Philip discussed the potential commercialization of large language models, specifically Phi 2 and GPT-4, and their performance when compared to Microsoft's model. He also explored the use of AI accelerators, the importance of quantization, and the potential of using models in the GGUF format. Additionally, he shared his experiences with running these models on a Raspberry Pi, the limitations of their current model, and his progress on running models on specific questions.
Next steps
• Philip to test and compare the performance of different large language models on Raspberry Pi devices.
• Philip to evaluate the impact of different precision levels (e.g., 8-bit vs. 2-bit) on model performance and interaction quality.
• Philip to conduct tests using various Raspberry Pi models with different RAM configurations (4GB and 8GB) to assess compatibility and performance.
• Philip to explore and test additional models in the GGUF format from Hugging Face for potential use on Raspberry Pi.
• Philip to compare the performance of Microsoft's Phi-2 model with other models like Cognitive Common on Mistral and DeepSeek.
• Philip to investigate if larger models (e.g., 7.3GB and 6.1GB) can run on a Raspberry Pi with 16GB of RAM.
• Philip to continue evaluating model performance using human evaluation methods for various tasks (e.g., Q&A, coding, mathematical problems).
Ref:
ggerganov/llama.cpp: ( github.com/gge... )
Deploy and run LLM on Raspberry Pi 5 vs Raspberry Pi 4B (LLaMA, LLaMA2, Phi-2, Mixtral-MOE, etc): ( www.dfrobot.co... )
Phi-2: The surprising power of small language models: (www.microsoft.... )
Raspberry Pi Model 5 ( www.seeedstudi... )

[@rohitkumar2306]

Getting lost of idea from this video.

[@dsrc12]

thanks for the feedback. More to come including a $20 computer with audio input and output that chats to openAI