Llama-2: the large open-source language models

13 Tem 11:25

Ah, Llama-2... Quite the adventure! I still remember my first time with these models. I was testing a fine-tuning on a medical question dataset, and a colleague slipped me a link to Meta's original release. At first, I was skeptical: "Another transformer doing the same thing as the others, but worse." Except... surprise. I dove into the architecture and realized Llama-2 wasn't just an improved copy of Llama-1. The changes in the attention layer (they integrated *Grouped-Query Attention* stuff), the usable context sizes (now up to 4096 tokens), and especially their approach to open source... it was serious stuff. I tried my own PyTorch implementation to understand the nuances, and the results were there: the model understood nuances better than some proprietary models I'd tested before. A real "aha!" moment. As for restrictions, yeah, there are nuances. Meta did open-source the code (weights, architecture, fine-tuning scripts), but the training data remains a black box. And their license (LLAMA 2 Community License) prohibits use for malicious applications or with more than 700M users/month without a special agreement. I’ve seen people complain about it, but personally, I find it more transparent than proprietary licenses where you don’t even know what you’re using. In practice now? I use it for two things at work: a first draft for technical documentation (great for avoiding writer’s block), and an internal assistant to help colleagues debug Python code. The model handles French queries well, though sometimes it invents random imports (*"import numpy as npz"* anyone?). And unlike some "open" models that demand 24GB of VRAM, Llama-2-13B runs on a decent personal GPU with a bit of optimization (8-bit quantization, vram-fsdp...). Clearly a game-changer for teams on a budget.