Intern Lumina U2: multi-codebook diffusion LLM for unified vision and image generation
The paper introduces Intern Lumina U2, which the authors describe as the first diffusion language model to represent multi-codebook visual tokens in a fully discrete space, where each visual position is encoded by a group of codes drawn from complementary codebooks to raise representational capacity while staying compatible with discrete text modeling. At the input the embeddings are concatenated per position, while at the output a lightweight multi-codebook autoregressive head predicts codes within each position sequentially and the diffusion backbone processes all positions in parallel. Built on AToken's unified 4D representation, it handles image, video and 3D inputs in a single backbone and, according to the authors, achieves leading results among unified models in visual understanding, image generation and image editing; a stability-aware sampling strategy combining confidence-plateau-guided unmasking with drift-triggered cache refreshing is reported to speed up image generation by 4.45x. The authors also say they built a full-stack training, inference and evaluation system on both Ascend and NVIDIA platforms to verify cross-platform effectiveness.
The work tries to place high-capacity visual representations and text in one discrete optimization space, offering a possible route toward unified multimodal diffusion models.