Image Generation & Diffusion

Question 1

What single task is a diffusion model trained on?

Accepted Answer

Predicting the noise that was added to an image — Images are noised by random amounts; the network learns to predict that noise. Generation is just this skill applied repeatedly in reverse.

Question 2

Generation starts from ___ and proceeds by ___.

Accepted Answer

pure Gaussian noise / iteratively subtracting predicted noise — Start from static; each step predicts the noise and removes a portion — the image emerges from the process.

Question 3

During denoising, what emerges first and what arrives last?

Accepted Answer

Structure/layout first, fine detail last — Early steps decide composition (sky vs hill); late steps refine texture — coarse-to-fine is intrinsic to the process.

Question 4

How does the text prompt influence generation?

Accepted Answer

Via cross-attention to the prompt embedding at every denoising step — A text encoder embeds the prompt and each of the 20-50 denoising passes attends to it — steering the whole trajectory.

Question 5

Why does latent diffusion use a VAE?

Accepted Answer

To run the expensive loop in a ~48× smaller compressed space, decoding to pixels once — Denoising 1024² pixels 30 times is brutal; loop in ~128² latents (the standard VAE downsamples 8× per dimension) and decode at the end — the compression made diffusion shippable.

Question 6

The three trained components of a Stable-Diffusion-style system are:

Accepted Answer

Text encoder, denoiser (U-Net/DiT), and VAE decoder — Prompt understanding, iterative denoising, and latent↔pixel translation — three models in a trench coat.

Question 7

Classifier-free guidance (CFG) works by:

Accepted Answer

Exaggerating the difference between prompt-conditioned and unconditioned noise predictions — Predict noise with and without the prompt, then amplify "what the prompt adds" — a faithfulness dial applied at every step.

Question 8

Setting CFG very high (15+) typically produces:

Accepted Answer

Over-saturated, high-contrast "fried" images with visible artifacts — You are amplifying a signal past what the image can absorb — adherence rises but colors burn and artifacts appear.

Question 9

Negative prompts work by:

Accepted Answer

Steering the same guidance mechanism away from the negative embedding — The unconditional side of guidance is replaced with the negative prompt — the trajectory is pushed away from it each step.

Question 10

Why is step count the main cost dial?

Accepted Answer

Every step is a full denoiser network pass, scaling compute linearly with steps — 30 steps = 30 forward passes of the U-Net/DiT — compute scales linearly with steps at fixed resolution.

Key insight

Generate an image by denoising

Test your knowledge

What single task is a diffusion model trained on?

The short version

In more depth

Image Generation & Diffusion

The loop

The three-model architecture ("Stable Diffusion is three models in a trench coat")

Engineering dials

Interview Questions

References & Further Reading

Related topics

Love this visualization?