Study finds mixed-modality retrievers favour text, proposes Trident
An arXiv paper systematically evaluates dense retrievers across architectures and finds their performance is highly sensitive to modality composition, following a pronounced V-shaped curve as image documents are progressively replaced with semantically corresponding text: strong on single-modality corpora but substantially degraded when modalities coexist. The authors call the finding "Chaos in the Text," noting that irrelevant text causes more severe degradation than an equal number of irrelevant images, because text representations receive systematically higher similarity scores and can outrank relevant images. They propose Trident, which builds text, image and fused text-image views of each document as co-equal positives and jointly optimizes relevance discrimination and positive-view balance via Multi-Positive View InfoNCE.
If this modality preference is widespread, ranking in multimodal RAG and retrieval systems could be skewed by irrelevant text; Trident offers one possible mitigation.
Computer Science > Information Retrieval
arXiv:2610.11816 (cs)
[Submitted on 8 Oct 2026]
Title:Chaos in the Text: Revealing the Modality Preference in Mixed-Modality Retrievers
Authors:Yubo Sun, Chunyi Peng, Yukun Yan, Zhenghao Liu, Zhipeng Xu, Sen Mei, Linlin Xin, Zheni Zeng, Maosong Sun
View a PDF of the paper titled Chaos in the Text: Revealing the Modality Preference in Mixed-Modality Retrievers, by Yubo Sun and 8 other authors