User Experiences with Phi-3 Mini 128K Instruct Language Model: What You Need to Know

Microsoft recently introduced their new Phi-3 LLMs, which quickly outperformed the Llama 3 models despite their release less than a week prior. The Phi-3 models, particularly the Phi-3-mini, have demonstrated remarkable efficiency. Despite having only 3.8 billion parameters — less than half the size of the Llama 3 8B—and being trained on significantly fewer tokens (3.3 trillion compared to 15 trillion), Phi-3-Mini-128K-Instruct surpasses the performance benchmarks of Llama 3 8B.

Remarkably, the Phi-3-mini can be deployed on mobile devices 💪, requiring only about 1.8GB of memory when quantized to 4-bits. This model also matches the performance of larger models like Mixtral 8x7B and GPT-3.5. The technical report credits the success of Phi-3 to the quality of its training data, which consists of heavily filtered web data and synthetic data. Additionally, there is a phi-3-mini-128K variant capable of handling up to 128,000 tokens, leveraging the same tokenizer used by Llama 2, with a vocabulary size of 32,064.

We’ve gathered feedback from users who have experimented with the Phi-3-Mini-128K-Instruct model and here are some key insights. The AI community is enthusiastic about having such a compact model that can be operated directly from mobile devices. Users are particularly impressed with its balance of speed and quality, likening it to the capabilities of older 7B models but with significantly better mathematical abilities.

One standout feature of Phi-3 is its MIT license, offering more flexibility in usage compared to other models with stricter licensing. Users have noted that the Phi-3-Mini-128K-Instruct excels at logic questions and storytelling, performing almost on par with a 7B model, yet it falls short of the performance claimed against larger models like Mixtral or ChatGPT 3.5. However, it does manage to outperform the Llama 3 8B in solving logic riddles.

In terms of functionalities, the Phi-3-Mini-128K-Instruct has been praised for its summarization, text analysis, search capabilities, and knowledge graph construction. While its translation capabilities aren’t perfect, it’s noted to be exceptional for a model of its size, particularly in smaller languages such as Danish, Finnish, and Swedish. Interestingly, it even performs better in coding tasks than the Llama 3.

A critical point of feedback is the model’s heavy censorship, which users find too restrictive compared to other models, limiting its use in generating content that might be considered sensitive.

The Phi-3-Mini-128K-Instruct model shows remarkable efficiency in academic-style queries that one might find in school textbooks but struggles with pop culture content, leading to varied accuracy. This aligns with its training on a more controlled dataset as indicated in its technical profile.

An interesting user story highlighted the model's utility in geographical queries. When tasked with providing details about a small, lesser-known town, the Phi-3-Mini-128K-Instruct not only pinpointed the country, province, and region but also provided near-accurate GPS coordinates.

Overall, while the Phi-3-Mini-128K-Instruct model has its limitations, particularly in handling diverse and complex cultural content, its strengths in educational and technical tasks make it a valuable tool for those needing a compact yet powerful AI assistant.

You can find all the language models from the Phi-3 family here.