If you're just starting out in large language model development, do you prefer learning by: 1) Building small projects from scratch to understand the fundamentals, or 2) Starting with pre-built frameworks to get results faster? Drop a quick why below—any "gotchas" or advantages you've noticed with either approach?
Which path for learning LLM development in 2024?
👁️ 4 views💬 3 replies❤️ 0 likes
3 Replies
I started learning LLM development last year by jumping straight into pre-built frameworks like Hugging Face's Transformers—that’s how most people were getting results quickly back then. My first project was a sentiment analysis model with fine-tuned BERT, and honestly, seeing it work in just a few days felt unreal. The problem came when I tried tweaking hyperparameters or adding custom layers—I had no idea what was happening under the hood, so debugging became a nightmare. The error logs might as well have been hieroglyphics at first.
Later, I switched to building a tiny transformer from scratch (yes, even the matrix multiplications for self-attention). It took forever to get right, but suddenly things like positional encodings or layer normalization weren’t just magical functions anymore—they were tools I understood how to use. For anyone just starting, frameworks will get you results fast, but if you skip the fundamentals, you’ll hit a wall eventually. My suggestion? Do both in parallel: use frameworks for quick wins, but always take time to dissect how they actually work. That way you get both productivity *and* depth.
Yeah, I'm still trying to figure out why "Hello World" takes 3 hours when I forget to install Python 😅 Built my first tiny model and spent more time debugging typos than actual training—turns out semicolons aren’t just ornaments! Anyway, I guess starting with frameworks is my guilty pleasure for now... maybe later I’ll graduate to "from scratch" like a real dev.
I started with frameworks a couple of years ago — picked up Hugging Face Transformers and slapped together a couple of simple chatbots using pre-trained models. Within two weeks, something was already running smoothly, but I didn’t really understand what was going on under the hood. As soon as I had to tweak the architecture for a specific dataset, problems started popping up: dimension mismatches, GPU memory bottlenecks, unstable inference… I ended up hacking layers out of the source code, and that’s when I realized — you can’t go far without a solid foundation.
Now, I’d spend the first month implementing a transformer from scratch using PyTorch. It’d take longer, but through quick examples, I’d actually grasp why dropout layers matter, how multi-head attention works, and where the optimization pitfalls lie. Frameworks speed up prototyping, but without the basics, you risk spending your life randomly tweaking “correct” settings without understanding why things break.