Preprint reports zero-shot steering transfer between text and image generation
An arXiv preprint reports that steering vectors derived from one modality can transfer zero-shot to another in several unified multimodal models. The authors report bidirectional transfers between text-to-image and image-to-text generation across concepts including emotion and object size. They argue that steering provides a causal test beyond correlational alignment such as representational similarity, and that highly transferable vectors tend to point in similar directions for the same concept despite a gap between text and image representations. The preprint includes links to code, data and a demo; the supplied report does not identify the evaluated models.
The reported transfers suggest that some controllable concepts may be shared across text and image modalities in unified multimodal models, offering evidence about how vision and language interact during generation.
Computer Science > Computer Vision and Pattern Recognition
arXiv:2610.36269 (cs)
[Submitted on 26 Oct 2026]
Title:Cross-modal Steering Vectors Exist Across Both Image and Text Generation
Authors:Donghao Zeng (1), Nishant Subramani (2), Tamar Rott Shaham (3), Benno Krojer (1 and 4) ((1) Mila, McGill University, (2) Carnegie Mellon University, (3) MIT CSAIL, (4) Northeastern University)
View a PDF of the paper titled Cross-modal Steering Vectors Exist Across Both Image and Text Generation, by Donghao Zeng (1) and 7 other authors
#Mila#McGill University#Carnegie Mellon University#MIT CSAIL#Northeastern University