Today we are excited to introduce GLM-Image, the first open-source, industrial-grade discrete auto-regressive image generation model. GLM-Image adopts a hybrid architecture combining an auto-regressive module with a diffusion decoder. The auto-regressive part is partially based on, and initialized from, [GLM-4-9B-0414][1] with 9 billion parameters, while the diffusion decoder follows [CogView4][2] to adopt a single-stream DiT structure with 7 billion parameters. In general image generation quality, GLM-Image aligns with mainstream latent diffusion approaches, but it shows significant advantages in text-rendering and knowledge-intensive generation scenarios. It performs especially well in tasks requiring precise semantic understanding and complex information expression, while maintaining strong capabilities in high-fidelity and fine-grained detail generation. In addition to text-to-image generation, GLM-Image also supports a rich set of image-to-image tasks including image editing, style transfer, identity-preserving generation, and multi-subject consistency.

Background: In recent years, diffusion models have become the mainstream in image generation for their training stability and strong generalization capabilities. Yet even with substantial improvements in diffusion modeling and VAE formulation[3][4][5], etc., end-to-end diffusion models still have shortcomings in complex instruction following and knowledge-intensive scenarios, often falling short in both information expression and semantic alignment. At the same time, some newly released high-quality image generation models have demonstrated outstanding performance in such knowledge-dense cases, producing visually rich detail while exhibiting auto-regressive modeling characteristics. Drawing inspiration from these developments, GLM-Image was designed from the beginning with two decoupled objectives: robust understanding of complex information and the ability to produce high-quality image details. In our approach, the auto-regressive generator produces tokens with low-frequency semantic signals, while the diffusion decoder refines high-frequency details to deliver the final image. This hybrid architecture not only performs reliably in general image generation tasks, but also presents noticeable advantages in creative work that demands intricate knowledge representation, pushing image generation toward a new stage that combines artistic aesthetics with precision in conveying information.

Techniques

Visual Token Selection

In previous visual auto-regressive generation models, the token types used have typically fallen into three categories:

• Visual codes obtained via discrete reconstruction training (VQVAE[6])
• Visual codes obtained via discrete semantic training (semantic-VQ[7])
• Statistical semantic features extracted from 1D vectors (as in DALLE2[8])

These approaches rank from high to low in the order above from an information completeness standpoint, whereas their semantic relevance tends to increase in the reverse order. For visual generation models, the correlation between tokens (or patches) is a crucial factor influencing both model convergence and the final output quality. In latent diffusion models, works as VAVAE[5] and SSVAE[9] have demonstrated the significance. While for auto-regressive generation, training loss comparison shows a clear different magnitude (~7 vs. ~3) for tokens derived from VQVAE and semantic-VQ with similar codebook size, suggesting that modeling with semantic tokens offers superior convergence properties for visual generation training. On the other hand, 1D vectors suffer from insufficient information completeness and correspondence towards a specific image, and are more commonly used in subsequent works for tasks as subject consistency generation (e.g., FLUX.1 Redux[10]).

Building on these conclusions and observations, GLM-Image adopts semantic-VQ as its primary tokenization strategy. To be specific, we implemented the tokenizer scheme from XOmni for better semantic correlation during token modeling, combined with a diffusion decoder subsequently decoding from these tokens to produce the final image outputs.

Auto-regressive Pre-training

The auto-regressive part of GLM-Image initializes from GLM-4-9B-0414 and implements combinitorial training of text-to-image generation and image-to-image generation. We freeze the text word embedding layer of the model while enabling other parts for training, appending an extra vision word embedding layer for vision token projection and replacing the original LM head with a vision LM head for the new task. We implement MRoPE as the positional embedding for the circumstance of interleaving images and texts from both generation tasks of text-to-image and image-to-image, as illustrated in the picture.

We train the model with multiple resolution stages including 256px, 512px and a mixed-resolution training stage spanning from 512px to 1024px. The tokenizer from XOmni patchifies the image with a 16× compression ratio, which means the token count per sample is 256, 1024 and 1024-to-4096, respectively for the three training stages. Given that we set the upscaling factor of our diffusion decoder’s final output to 32, the resulting image resolution ranges from 1024 px to 2048 px.

In the initial 256-token stage of training, we implemented a straightforward raster scan order for token generation strategy. However, as we advanced to higher-resolution training stages, we observed a drop in controllability of model outputs when applying the same generation approach. To address this, we adopted a progressive generation strategy[11]: before generating high-resolution image tokens, we first generate approximately 256 tokens with the same aspect ratio, obtained by tokenizing a down-sampled version of the target image. Considering that these preliminary tokens largely determine the final image layout, but due to their small number they might receive insufficient attention, we increased their training weight in subsequent stages, which effectively improved the overall generation quality.

Decoder Formulation

The diffusion decoder receives semantic-VQ tokens generated by the auto-regressive model as conditional inputs to reconstruct the target image. While semantic-VQ tokens carry rich semantic information, they discard high-frequency image details and primarily present relatively low-frequency image layout information. As a result, the diffusion decoder must retain a certain generative capacity to synthesize and recover the missing fine-grained details.

For the backbone design, we follow CogView4 to adopt a single-stream DiT architecture. The decoder employs flow matching as its diffusion scheduling strategy, ensuring stable training and efficient convergence for high-fidelity image generation. For integration, the semantic-VQ tokens are first passed through a projection layer and then concatenated with the VAE latent representation along the channel dimension. This preserves the input sequence length and incurs almost no extra computational overhead. Since the semantic-VQ tokens already provide sufficient semantic information, we remove the prompt input from the decoder’s conditioning. This design eliminates the need for a large-parameter text encoder, thereby reducing both memory usage and computational cost. To strengthen the decoder’s ability to render complex textual content—particularly Chinese characters—we introduce a lightweight Glyph-byT5[12] model that performs character-level encoding for rendered text regions. The resulting glyph embeddings are concatenated with the vision embeddings along the sequence dimension.

For image editing tasks, it is often critical to preserve the high-frequency details present in the reference images. The semantic information provided solely by semantic-VQ tokens is insufficient for modeling fine-grained detail preservation. Therefore, in GLM-Image we use both the semantic-VQ tokens and the VAE latents of the reference images as additional conditioning inputs for the diffusion decoder, as illustrated in Figure4. Unlike concurrent image editing models such as Qwen-Image-Edit[13], which apply full attention between reference images and the generated image, we adopt a block-causal attention mechanism between the reference and the generated image. This follows the attention design pattern used in ControlNet-Reference-Only[14]. The block causal attention can significantly reduce the computation overhead on the reference image tokens by kvcache while keeping competitive detailed preservation.

disentangled Reward for AR+Diffusion Post-training

In the post-training stage, GLM-Image employs a decoupled reinforcement learning strategy to separately optimize its auto-regressive generator and diffusion decoder, enabling improvements to both semantic alignment and visual detail quality. Both modules are trained with GRPO[15] optimization. For the diffusion decoder specifically, GLM-Image adopts flow-GRPO[16], a variant of the standard LLM GRPO adapted for diffusion models.

The auto-regressive module focuses on low-frequency rewards that guide semantic consistency and aesthetics, thereby improving instruction following and artistic expressiveness. It combines multiple reward sources, including HPSv3[17] for aesthetic scoring, OCR for enhancing text rendering accuracy and VLM for overall semantic correctness of generated content. The decoder module targets high-frequency rewards to refine fine-detail fidelity and text precision. It leverages LPIPS[18] to improve perceptual texture and detail similarity, integrates OCR signals to further boost text accuracy, and employs a dedicated hand-scoring model to enhance the correctness of generated hands.

Eval metrics

Text-rendering benchmarks

CVTG-2k

LongText-Bench

General benchmarks

OneIG_EN

OneIG_ZH

DPG Bench

TIFF Bench

Reference

[1] https://huggingface.co/zai-org/GLM-4-9B-0414
[2] https://huggingface.co/zai-org/CogView4-6B
[3] Liu, Xingchao, Chengyue Gong, and Qiang Liu. "Flow straight and fast: Learning to generate and transfer data with rectified flow." arXiv preprint arXiv:2209.03003 (2022).
[4] Yu, Sihyun, et al. "Representation alignment for generation: Training diffusion transformers is easier than you think." arXiv preprint arXiv:2410.06940 (2024).
[5] Yao, Jingfeng, Bin Yang, and Xinggang Wang. "Reconstruction vs. generation: Taming optimization dilemma in latent diffusion models." Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.
[6] Van Den Oord, Aaron, and Oriol Vinyals. "Neural discrete representation learning." Advances in neural information processing systems 30 (2017).
[7] Geng, Zigang, et al. "X-omni: Reinforcement learning makes discrete autoregressive image generative models great again." arXiv preprint arXiv:2507.22058 (2025).
[8] Ramesh, Aditya, et al. "Hierarchical text-conditional image generation with clip latents." arXiv preprint arXiv:2204.06125 1.2 (2022): 3.
[9] Liu, Shizhan, et al. "Delving into Latent Spectral Biasing of Video VAEs for Superior Diffusability." arXiv preprint arXiv:2512.05394 (2025).
[10] https://huggingface.co/black-forest-labs/FLUX.1-Redux-dev
[11] Zheng, Wendi, et al. "Cogview3: Finer and faster text-to-image generation via relay diffusion." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.
[12] Liu, Zeyu, et al. "Glyph-byt5: A customized text encoder for accurate visual text rendering." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.
[13]Wu, Chenfei, et al. "Qwen-image technical report." arXiv preprint arXiv:2508.02324 (2025).
[14]Zhang, Lvmin, Anyi Rao, and Maneesh Agrawala. "Adding conditional control to text-to-image diffusion models." Proceedings of the IEEE/CVF international conference on computer vision. 2023.
[15] Shao, Zhihong, et al. "Deepseekmath: Pushing the limits of mathematical reasoning in open language models." arXiv preprint arXiv:2402.03300 (2024).
[16] Liu, Jie, et al. "Flow-grpo: Training flow matching models via online rl." arXiv preprint arXiv:2505.05470 (2025).
[17] Ma, Yuhang, et al. "Hpsv3: Towards wide-spectrum human preference score." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2025.
[18] Zhang, Richard, et al. "The unreasonable effectiveness of deep features as a perceptual metric." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.