Diffusion Transformer (DiT)

Headlines · 1

1. 6d
   AReA: Training-Free Attention That Boosts High-Resolution Diffusion Transformers

   Researchers posted AReA, a training-free adaptive receptive-field attention mechanism for high-resolution image generation with Diffusion Transformers (DiTs). The paper argues uniform full attention is the common cause of both the quadratic cost and the structural distortions and lost detail seen at higher resolutions, because attending to all tokens at every step pushes the model beyond the token counts and relative distances seen in training. AReA instead lets the effective receptive field evolve from coarse to fine along the denoising trajectory: early steps apply cross-scale coarse attention over downsampled keys and values to capture global context, while later steps apply local attention within progressively shrinking windows. The authors report that, with no extra parameters or training, AReA consistently beats state-of-the-art training-free baselines across target resolutions and models while also being faster and more efficient.

   Hugging Face · Papers · 🔥 0

Experience and discussion from the community

Share my Diffusion Transformer (DiT) experienceAsk about Diffusion Transformer (DiT)