Why AI Image Models Keep Getting Smaller and Better

Last reviewed August 2026

AI image models are getting smaller because of three converging techniques: knowledge distillation (training small models to mimic large ones), quantisation (reducing numerical precision from 32-bit to 4-bit without visible quality loss), and architectural efficiency improvements in latent diffusion. The result is that four-step generation on a consumer GPU in 2026 produces output that required fifty steps on a data centre card in 2023. Distillation is the single most important technical development for consumer AI because it moved photorealistic generation from server farms onto desktop hardware.

What Is Knowledge Distillation and Why Does It Matter?

Knowledge distillation is a training technique where a large, slow “teacher” model trains a smaller, faster “student” model to replicate its outputs. The student learns not just the training data but the teacher’s learned representations — effectively inheriting years of expensive training in a fraction of the compute. Applied to image generation, distillation produces models that generate in 1-4 steps what the original required 20-50 steps to produce.

Stability AI’s SDXL Turbo, released in November 2023, demonstrated this dramatically. A standard SDXL generation at 1024×1024 required 30-50 diffusion steps (approximately 8-15 seconds on a consumer GPU). SDXL Turbo, distilled using Adversarial Diffusion Distillation, produced comparable images in a single step — under one second. According to Stability AI’s technical report, SDXL Turbo achieved equivalent FID scores to the full model at 1/30th the compute cost.

Latent Consistency Models (LCM), published by researchers at Tsinghua University in October 2023, took a different distillation approach. Rather than adversarial training, LCM uses consistency distillation to teach the student to predict the final image from any intermediate noise level. The result is 2-4 step generation with quality that sits between SDXL Turbo’s speed and full SDXL’s fidelity — a practical middle ground that most users find sufficient.

How Does Quantisation Shrink Models Without Destroying Quality?

Quantisation reduces the numerical precision of model weights. A standard diffusion model stores each parameter as a 32-bit floating point number (FP32). Quantisation converts these to lower precision: FP16 (half the memory), FP8 (quarter), INT8, or even INT4 (one-eighth the original size). The insight is that neural networks are remarkably tolerant of reduced precision — most parameters do not need 32 bits of accuracy to produce good output.

For free AI image tools running locally, quantisation is the difference between needing a £1,000 GPU and a £200 one. A full-precision FLUX.1 Dev model requires approximately 23 GB — more than any consumer GPU’s VRAM. Quantised to FP8, it fits in 12 GB. Quantised to NF4 (a format developed for the GGUF standard), it runs on 8 GB cards. Visual quality loss at FP8 is imperceptible in blind testing; at NF4, minor degradation appears in fine textures but overall composition and coherence remain intact.

The GGUF quantisation format, originally developed by Georgi Gerganov for language models, has become the de facto standard for distributing quantised diffusion models. Sites like Hugging Face and CivitAI now host GGUF variants of major checkpoints alongside full-precision versions, making low-VRAM generation accessible without any technical knowledge beyond downloading the correct file.

What Architectural Changes Made Diffusion Models More Efficient?

Beyond distillation and quantisation, the underlying architecture of diffusion models has improved. Three changes have had the largest impact on efficiency:

Latent diffusion (the “L” in LDM) was the foundational efficiency gain. Instead of running diffusion in pixel space (generating a 1024×1024 image = processing over 3 million values per step), latent diffusion operates in a compressed latent space — typically 128×128 or 64×64 — and uses a separate decoder to expand the result to full resolution. This 64x reduction in per-step computation is why Stable Diffusion can run on consumer hardware at all.

Transformer-based architectures replaced the U-Net backbone in newer models. FLUX.1 uses a Multimodal Diffusion Transformer (MMDiT) that processes text and image tokens in a unified attention mechanism. This produces better prompt adherence and composition than U-Net architectures, while being more amenable to optimisation techniques like flash attention. According to Black Forest Labs, FLUX.1’s transformer architecture achieves higher quality at equivalent compute compared to SDXL’s U-Net.

Rectified flow matching, used in FLUX.1 and Stable Diffusion 3, replaces the traditional noise-prediction objective with a straighter path between noise and image. Straighter paths require fewer steps to traverse, which is why FLUX Schnell produces usable output in 1-4 steps without explicit distillation — the architecture itself is more step-efficient than its predecessors.

How Much Smaller Are Current Models Compared to Two Years Ago?

The practical improvement is dramatic. A comparable generation task — 1024×1024 photorealistic image from a text prompt — has moved from requiring a 48 GB A6000 in early 2024 to running on an 8 GB consumer card in 2026, while maintaining equivalent output quality.

Does Smaller Always Mean Worse Quality?

Not at the model architecture level, and only marginally at the quantisation level. Distilled models like SDXL Turbo sacrifice diversity (fewer unique outputs for the same prompt) rather than peak quality. A distilled model’s best output matches its teacher; its average output is slightly less varied. For practical use — generating a specific image for a specific purpose — this tradeoff is invisible.

Quantisation introduces measurable but often imperceptible degradation. Academic benchmarks report FID score increases of 0.5-2.0 points when moving from FP16 to FP8, and 2-5 points at NF4. In practical terms, human evaluators in blind comparisons published by the ComfyUI community cannot reliably distinguish FP8 output from FP16 output. NF4 becomes noticeable primarily in fine hair detail, complex fabric textures, and small text — areas where the precision loss compounds across multiple transformer attention layers.

The direction is clear: each generation of models produces better output at lower compute cost. FLUX Schnell in 4 steps at NF4 quantisation produces noticeably better images than full-precision SDXL at 50 steps, despite using a fraction of the compute. Progress in architecture design consistently outpaces the quality loss from compression.

What Does This Trend Mean for the Next Two Years?

If the current trajectory holds, photorealistic image generation will run on integrated GPUs (Intel Arc, AMD RDNA 3 iGPU, Apple M-series) at usable speeds by late 2027. The combination of further distillation, hardware-specific quantisation kernels, and architecture improvements points toward 2-4 GB model sizes producing current-FLUX-level quality. Stability AI’s roadmap mentions on-device generation as a priority; Black Forest Labs has stated that future FLUX variants will target mobile-class hardware.

For consumers, the takeaway is straightforward: the hardware required for state-of-the-art local AI image generation drops by roughly half every 12-18 months. A machine purchased today for local generation will run models two generations ahead of current quality within its useful hardware lifetime.

For related reading, see open-source rendering versus paid tools and running AI on integrated GPUs.

Sources

• SDXL Turbo technical report: stability.ai/research/adversarial-diffusion-distillation
• Latent Consistency Models paper: arxiv.org/abs/2310.04378
• FLUX.1 model architecture: blackforestlabs.ai
• GGUF quantisation format: github.com/ggerganov/ggml
• Stable Diffusion 3 Medium technical details: stability.ai/news/stable-diffusion-3-medium