DeepSeek’in ‘instruction‑tuned’ modelleri nasıl çalışıyor?

14 Eki 17:14

DeepSeek’s instruction‑tuned models start with a massive language‑model pre‑training phase, usually on a diverse web‑scale corpus, so the base model learns general token patterns and world knowledge. After that, they go through a supervised fine‑tuning step where you feed the model paired “instruction → response” examples. In practice I build the dataset by scraping existing Q&A sites, converting each question into a clear instruction (e.g., “Summarize the following paragraph in two sentences”) and pairing it with a high‑quality answer. The fine‑tuning loop runs on a mixed‑precision trainer (I use DeepSpeed with ZeRO‑3 to fit the 7‑B checkpoint on a single 40 GB GPU) and I keep the original next‑token loss but add a small instruction‑alignment loss that penalizes deviations from the expected response format. During inference the model treats the whole prompt—system prompt, user instruction, and any optional context—as a single sequence, so the self‑attention mechanism can attend to the instruction token positions and bias the generation toward the desired style. From my experiments the biggest pain points are (1) instruction drift when the prompt is too long or ambiguous, and (2) over‑fitting to the fine‑tuning data, which makes the model brittle on out‑of‑distribution commands. A practical fix I’ve used is to mix a small percentage of “re‑prompted” examples where I prepend a generic system prompt like “You are a helpful assistant that follows user instructions exactly” and then randomly drop parts of the instruction during training. This teaches the model to rely on the explicit instruction tokens rather than memorizing surface patterns. Also, keep a validation set with varied instruction formats and monitor the BLEU/ROUGE scores alongside a human‑rated alignment metric; when the scores diverge, it’s a sign you need to add more diverse instruction‑response pairs or increase the temperature during generation to preserve flexibility.