Five generation modes & nine diagram styles, end-to-end on a single GPU

HiDream-O1-Image is an 8B-parameter image-generation model released in May 2026 by HiDream-ai. Architecturally it is a Pixel-level Unified Transformer (UiT): instead of stacking a separate text encoder, VAE, and denoiser the way diffusion models like FLUX or SD3 do, it encodes raw pixels, text, and task-specific conditions in a single shared token space. One end-to-end model handles five generation modes natively — text-to-image, instruction editing, subject-driven personalization, bbox-layout conditioning, and skeleton/openpose conditioning — at resolutions up to 2048×2048.

The open weights ship in two variants:

The Artificial Analysis text-to-image leaderboard ranks a HiDream-O1 entry (“Peanut”) at #8 — but that entry is the unreleased 200B+ Pro variant. The open 8B is what we work with here. It does not match the Pro’s photorealism or its identity preservation in IP mode, but for free local generation on a single consumer GPU it’s remarkable.

What it’s good at: photographic compositions, long-text rendering, multi-region layout, infographic and architecture-diagram aesthetics, editorial portraits, instruction-based editing with identity preservation.

Where it struggles: dense short technical labels (>6 elements at high resolution), sequence-diagram arrows, period-faithful photo restoration, identity preservation when the prompt and reference image disagree on physical traits.

Target hardware: NVIDIA RTX 4090 (24 GB VRAM), Linux (Ubuntu/Pop!_OS), CUDA 12.8 or newer driver. Python 3.12 in a dedicated venv. Disk: ~7 GB for the venv + ~35 GB for both BF16 model weights.

1. Clone the upstream repo

git clone https://github.com/HiDream-ai/HiDream-O1-Image.git
cd HiDream-O1-Image

2. Create a dedicated Python 3.12 venv

python3.12 -m venv venv
source venv/bin/activate
pip install --upgrade pip

3. Install torch (pin to <2.9)

PyTorch 2.9.x has a Qwen3-VL regression (QwenLM/Qwen3-VL#1811) that the upstream README explicitly warns about. Pin below 2.9:

pip install "torch<2.9" --index-url https://download.pytorch.org/whl/cu128

4. Install the upstream requirements

pip install -r requirements.txt
pip install "huggingface-hub>=0.24" safetensors accelerate

5. (Optional) flash-attn

Flash attention is preferred but optional. On CUDA 13 hosts it often fails to build; the pipeline falls back to standard attention with a small (~15%) speed penalty.

pip install flash-attn --no-build-isolation \
  || sed -i 's/"use_flash_attn": True/"use_flash_attn": False/' models/pipeline.py

6. Download BF16 weights

Use drbaph’s republished BF16 weights — about 17.6 GB per variant. Pick a cache root with plenty of free space (the model is too big for a home directory on most laptops):

export HF_HOME=/big/disk/hugging_face_cache
huggingface-cli download drbaph/HiDream-O1-Image-Dev-BF16
huggingface-cli download drbaph/HiDream-O1-Image-BF16

7. Verify CUDA

python -c "import torch; assert torch.cuda.is_available(); print(torch.cuda.get_device_name(0))"

8. Run inference

The upstream inference.py is an argparse wrapper around models.pipeline.generate_image(). You can use it directly:

python inference.py \
  --model_path /big/disk/hugging_face_cache/hub/models--drbaph--HiDream-O1-Image-Dev-BF16/snapshots/<rev>/ \
  --model_type dev \
  --prompt "a photo of a cat on a wooden floor" \
  --output_image cat.png

For editing or IP generation, swap --model_type full and pass --ref_images path1 path2 .... The upstream README lists complete examples for all five modes.

Text-to-image is the default mode. A prompt goes in, a 2048×2048 (or aspect-bucket-snapped) image comes out. HiDream-O1 is unusually good at photographic scenes and long-text rendering compared to other 8B-class models. Use the Dev variant for general T2I — it’s about 2× faster than Full and produces high-quality results at 28 inference steps.

Edit mode takes a source image plus a text instruction and produces a modified version. Identity, composition, lighting, and unaffected regions are preserved; only the instruction-specified change is applied. The upstream README recommends the Full variant for editing, and that’s our default. Pair edit mode with --keep-original-aspect so the model uses the input’s exact dimensions instead of snapping to a fixed bucket.

Subject-driven personalization — also called IP reference — places a known subject into a new scene. The model takes one or more reference images plus a prompt describing the new context. Two design rules apply: (1) the model was trained on multi-reference inputs (the README’s example uses ten), so passing two genuinely different views (e.g. face crop + full body) outperforms a single image or duplicated copies; (2) when the prompt and reference disagree on physical traits (hair, beard color, age), the model follows the prompt. Drop conflicting descriptors and the reference dominates.

Bbox-layout conditioning constrains where named regions appear in the frame. The layout JSON uses [x1, x2, y1, y2] normalized coordinates (xxyy, not xyxy) and accepts either a raw JSON string or a JSON file path. The Full variant runs at 50 steps with CFG 5.0; we drop shift to 1.0 per the upstream README’s Mode 5 example.

Skeleton / openpose-conditioned mode uses a pose-skeleton image (named with a .openpose.* suffix) to drive the body posture of the generated subject. Optional appearance references can be added alongside the pose image. An *.openpose.jpg file is typically pre-extracted by a separate preprocessor (e.g. controlnet-aux’s OpenposeDetector). We don’t showcase skeleton mode here — it’s functional but needs pose extraction outside the scope of this article.

HiDream’s combination of long-text rendering plus multi-region layout makes it surprisingly useful for cloud architecture, software architecture, flowchart, and editorial-infographic generation — provided you respect its density ceiling.

The empirically-tuned recipe: use the Full variant (the Dev variant’s low-step / no-CFG recipe consistently hallucinates short technical labels). Cap diagram complexity at five to seven labeled main elements per render — beyond that, per-element typography degrades. Use --size M, not the largest size; smaller canvases make labels proportionally larger relative to the model’s working resolution. Always quote your exact label strings in the prompt: tile labeled ‘API Gateway’ is reliable; the gateway service gets paraphrased.

Each example below shows the rendered diagram, the prompt that produced it, and a candid note on what worked and what didn’t.

• Upstream repo: github.com/HiDream-ai/HiDream-O1-Image — reference inference, web demo (app.py), prompt agent.
• BF16 weights (Full): huggingface.co/drbaph/HiDream-O1-Image-BF16
• BF16 weights (Dev): huggingface.co/drbaph/HiDream-O1-Image-Dev-BF16
• Technical report: arXiv:2605.11061 — HiDream-O1-Image (Pixel-level Unified Transformer)
• Community ComfyUI node: github.com/Saganaki22/HiDream_O1-ComfyUI — required if you want to use the FP8 weights.
• Hugging Face Spaces demos: HiDream-O1-Image, HiDream-O1-Image-Dev

All renders in this article were generated locally on an RTX 4090 with weights cached on a secondary drive. Total wall-clock for the full gallery: about 90 minutes including diagram regenerations after each tuning pass.