Which path for learning LLM dev in 2024?

14 Tem 20:33

I started learning LLM dev last year by diving straight into pre-built frameworks like Hugging Face's Transformers—that’s how most people were getting results quickly back then. First project was a sentiment analysis model with fine-tuned BERT, and honestly, seeing it work in just a few days felt unreal. The problem came when I tried tweaking hyperparameters or adding custom layers—I had no idea what was happening under the hood, so debugging became a nightmare. The error logs might as well have been hieroglyphics at first. Later, I switched to building a tiny transformer from scratch (yes, even the matrix multiplications for self-attention). It took forever to get right, but suddenly things like positional encodings or layer normalization weren’t just magical functions anymore—they were tools I understood how to use. For anyone just starting, frameworks will get you results fast, but if you skip the fundamentals, you’ll hit a wall eventually. My suggestion? Do both in parallel: use frameworks for quick wins, but always take time to dissect how they actually work. That way you get both productivity *and* depth.

14 Tem 21:21

Yeah, I'm still trying to figure out why "Hello World" takes 3 hours when I forget to install Python 😅 Built my first tiny model and spent more time debugging typos than actual training—turns out semicolons aren’t just ornaments! Anyway, I guess starting with frameworks is my guilty pleasure for now... maybe later I’ll graduate to "from scratch" like a real dev.

14 Tem 21:58

Начал с фреймворков пару лет назад — взял Hugging Face Transformers, сляпал пару простых чат-ботов на основе готовых моделей. За две недели уже что-то бегло работало, но толком не понимал, что под капотом. Как только пришлось модифицировать архитектуру под специфичный датасет, начались проблемы: размерность ошибок, GPU memory bottlenecks, инференс нестабильность… Пришлось резать слои с исходного кода, и вот тогда понял — без фундамента никуда. Сейчас бы сначала потратил месяц на реализацию трансформера с нуля, используя PyTorch. Хоть и дольше, зато на быстрых примерах усвоил бы, почему dropout слои, как работает multi-head attention, и где кроются грабли в оптимизации. Рамки ускоряют прототипирование, но без базы рискуешь всю жизнь тыкать наугад в “правильные” настройки, не понимая почему что-то ломается.