Stable Diffusion’da metin‑görsel eşleştirmesi nasıl gerçekleşiyor?

04 Eki 09:06

Stable Diffusion takes your prompt, tokenises it with CLIP‑text and turns those embeddings into a conditioning vector that is injected at every denoising step. The core loop is a reverse diffusion process: you start from pure Gaussian noise in the latent space, then the UNet predicts the noise component for the current timestep and subtracts it, gradually moving the latent toward a clean representation that matches the conditioning. Classifier‑free guidance (CFG) simply runs the UNet twice—once with the text conditioning and once with an “empty” conditioning—and blends the two predictions (usually `pred_guided = pred_uncond + cfg_scale * (pred_cond – pred_uncond)`). This amplifies the prompt signal while keeping the underlying diffusion dynamics intact, which is why higher CFG scales give sharper, more prompt‑faithful images but can also suppress diversity. In training, the model sees billions of image‑text pairs across many domains, and that variety directly translates into its ability to generalise to unseen concepts. If the dataset is skewed toward a particular style or subject, you’ll notice the outputs gravitating toward that bias. From my own experiments, a couple of tricks help tighten creative control: (1) use a moderate CFG (≈7–9) and then post‑process with a low‑strength img2img pass to nudge details without re‑introducing too much noise; (2) prepend or append “style‑specific” tokens (e.g., “in the style of Studio Ghibli”) to steer the latent early on; and (3) fine‑tune a lightweight LoRA on a curated subset of images that capture the exact aesthetic you want. These methods let you keep the broad creativity of the base model while pulling the result toward a more predictable look.