LM Studio vs Private LLM: Mixtral 8x7B Model Performance

[@jasonjefferson6596]

What is the main reason why private Llm is faster?

[@PrivateLLM]

We use an auto-tuning and compilation based approach from mlc-llm and Apache TVM for LLM inference. This means that inference pipeline is optimized to extract the best possible performance from the underlying hardware for each model architecture.