Recently, smaller language models (e.g., Mistral 7B, phi-2, etc.) presented as alternatives to large language models (LLMs) have been gaining popularity. Their advantages in terms of cost-effectiveness and performance are frequently mentioned. Does this rise of smaller models threaten the dominance of large language models, or do they complement them? In your opinion, which areas of use will they be more suitable for in the future? Let's discuss!
Are Small Language Models a Threat to Their Dominance?
👁️ 8 views💬 1 replies❤️ 0 likes
1 Replies
I've been working with both model classes for half a year now: locally, I run phi-2 for document summarization within my research team, while for complex coding questions, I occasionally fall back on LLMs like Llama 3.
For me, the answer is clear regionally: small models like phi-2 are perfectly adequate when the primary goals are speed and low infrastructure costs—everything runs on an old NVIDIA 3080 here without any cloud expenses. But as soon as we're dealing with context length (over 4k tokens) or niche knowledge, we regularly hit limits: phi-2 suddenly forgets details in long conversation threads that Llama 3 handles with ease. My compromise: we fine-tune phi-2 with specific domain data, and then it almost reaches the quality of large models—but the 7-billion-parameter ceiling is still noticeable.
For your use case, I'd recommend: start with a prototype using a 7B model like Mistral. If the accuracy is right, you can always scale later—but skip proprietary LLMs from the start if you're dealing with sensitive data. Small models already cover 80% of typical use cases well enough today.