Ideogram 4: structured JSON prompting, hands-on, on a single RTX 4090

Ideogram has been the "text-in-images" lab since its first model, and Ideogram 4 is its first open-weight release. It is not a fine-tune of an existing checkpoint — it's a 9.3B-parameter foundation model trained from scratch, aimed squarely at design work: typography, layout, posters, diagrams, brand color.

The receipts are public. On Design Arena's image Elo board it is the top-ranked open-weight model, trailing only proprietary GPT and Gemini image models; filtered to open weights it leads by a commanding margin. In ContraLabs' blind typography evaluation, ten professional designers picked it first 47.9% of the time — ahead of Nano Banana 2 (30.0%), FLUX.2 [max] (15.5%) and Grok Imagine (15.0%).

Parameter efficiency is the other headline: at 9.3B it beats much larger open models on text rendering — Qwen-Image (20B), FLUX.2 [dev] (32B), even HunyuanImage 3.0's 80B MoE.

Three design decisions define the model. First, it's a fully single-stream Diffusion Transformer: text tokens and image latent tokens are concatenated into one sequence and processed by the same 34 layers — no separate branches, so text and pixels interact at every depth. Positioning uses 3D multimodal RoPE so both modalities share one coordinate space.

Second, the text encoder is not CLIP or T5 — it's Qwen3-VL-8B-Instruct, a full vision-language model run in text-only mode. Hidden states are extracted from 13 layers (0, 3, 6 … 35) and concatenated, handing the DiT a multi-scale representation from surface tokens to deep semantics. I suspect this is a big part of why JSON captions work so well: a VLM has seen structured descriptions of images.

Third, generation is flow matching: the network predicts a velocity field and an Euler sampler integrates from noise to image. Classifier-free guidance is dual-branch with two separate transformers — and the negative branch is asymmetric, processing image tokens only, which makes the unconditional pass cheaper than the conditional one.

The weights ship in two quantizations on Hugging Face (gated — accept the license and set HF_TOKEN). Everything in this article ran nf4.

Sampling is controlled by named presets — same model, different step budgets. All three share one clever recipe: most steps run at guidance 7 for adherence, then the last few drop to guidance 3 — gentle "polish" passes that clean artifacts right before the image resolves.

The entire registry is ~25 lines of source — with one gotcha worth knowing:

# src/ideogram4/sampler_configs.py
# guidance_schedule is in loop-INDEX order:
# index 0 is the LAST (polish) step.
PRESETS = {
  "V4_QUALITY_48": SamplerParameters(
    num_steps=48,
    guidance_schedule=(3.0,) * 3 + (7.0,) * 45,
    mu=0.0, std=1.5,
  ),
  "V4_DEFAULT_20": SamplerParameters(
    num_steps=20,
    guidance_schedule=(3.0,) * 2 + (7.0,) * 18,
    mu=0.0, std=1.75,
  ),
  "V4_TURBO_12": SamplerParameters(
    num_steps=12,
    guidance_schedule=(3.0,) * 1 + (7.0,) * 11,
    mu=0.5, std=1.75,
  ),
}

Ideogram 4 was trained on structured JSON captions, not free text. Plain sentences work, but a schema-compliant JSON object unlocks the controllability this model is famous for. Three top-level keys:

• high_level_description — one summary sentence.
• style_description — aesthetics, lighting, medium, an optional hex color_palette, and exactly one of photo (photographic) or art_style (everything else).
• compositional_deconstruction — a background plus an elements list. Each element is obj or text, with a description, an exact text string when applicable, and an optional bbox as [ymin, xmin, ymax, xmax] in a 0–1000 space independent of output resolution.

Here's a real caption from my library, and the image it produced — every service icon landed inside its declared box:

Hand-writing captions is powerful but slow, so the repo ships magic prompt: an LLM expands your one-liner into a full schema-compliant caption, aspect-ratio aware. Backends: Ideogram's hosted endpoint (free with an IDEOGRAM_API_KEY), Claude Opus/Sonnet via OpenRouter using the open-source system prompt that ships in the repo, and a Gemini Flash-Lite backend I added — fractions of a cent per expansion.

I wrote a library of 14 hand-crafted JSON captions across every style I could think of and generated them all locally (V4_DEFAULT_20, seed 11). Each result below shows the source, the output, and an honest verdict. We start with photorealism.

The portrait asked for a single Rembrandt key light, an 85mm f/2 look, the navy cap and cream wool sweater — and delivered pore-level skin detail, white stubble, and catchlights in pale blue eyes with zero uncanny artifacts. Only miss: the requested "hint of a smile" came out neutral. The snow leopard ran the full pipeline — one plain sentence → my Gemini backend → JSON → native 2048², no upscaler.

Switching photo → art_style turns the model into a painter with deep art-history vocabulary: "sfumato blending, fine glazing layers, visible craquelure" produced exactly those. The one interesting miss of the whole experiment is the fox: every prop arrived (goggles, tool belt, one-eyed robot) but the requested flat TV-cartoon cel shading drifted toward painterly storybook. Style adherence is strong but not absolute — register can drift toward the model's design-forward house taste.

This is where bounding boxes earn their keep — and where the model's one systematic weakness shows. Structure: flawless. Small labels: occasionally wobbly.

The infographic delivered its headline, both stat callouts (2.25 billion cups / 25 million growers), the ascending bars, and the source credit — but the numbers inside the chart corrupted ("652,00"). Lesson: words render better than digits. The magazine cover is near print-ready; I asked for the robot's head to partially overlap the masthead for depth and got exactly that layering. One small teaser line wobbled.

I saved Arabic for last deliberately, because the story is mixed and worth telling straight. Structurally these are impressive: the infographic flows right-to-left as Arabic must, text blocks are right-aligned, the three tip cards sit on their bounding boxes, and it even used proper Arabic-Indic numerals (١ ٢ ٣). The magazine's Andalusian-gate photography is stunning and the gold-on-green palette followed my hex codes exactly.

Each generation scored on four axes: adherence (did everything in the caption appear?), layout (were bboxes respected?), text (fidelity of rendered strings) and craft (quality of the medium itself). Set average: 8.6 / 10.

Three presets × seven aspect ratios, fixed JSON caption and seed, one warmup excluded, torch.cuda.synchronize() around every run. Each timing is the complete pipeline: text encode, all sampling steps, VAE decode.

The three scaling laws

• Linear in steps. Per-step cost is constant per resolution across presets (1024²: 1.74 → 1.59 → 1.50 s/step). Quality ≈ 2.3× Default; Turbo saves ~35%.
• Super-linear in pixels. 4× the area costs ~6.3× the time — quadratic attention over 16k image tokens at 2048². The log-log curve sits between linear and quadratic.
• VRAM follows resolution only, never steps: 17.2 → 23.3 GB, kissing the 4090's ceiling at 2048². Orientation is free — portrait ≡ landscape to the decimal.

The practical recipe: draft in Turbo at 1024², work in Default at ≤ 2 MP, and reserve hi-res Quality renders for finals (and a coffee break).