CLIP

Headlines · 2

1. 3d
   JEPA Paper Proposes a Platonic Representation Theorem

   Randall Balestriero and Yann LeCun submitted “Proving and Improving the Platonic Representation Hypothesis with JEPAs” to arXiv on October 29, 2026. The authors say their theorem applies to separately trained JEPA encoders whose positive pairs are two observations of the same scene, with misalignment bounded otherwise, and that the encoders learn equivalent representations up to one rotation. They report experiments ranging from 145,000 captions to text associated with 940 million web images. Their reported results include requiring 42 times fewer training samples than next-token prediction for the same alignment, 64% top-5 accuracy on ImageNet-100 for a 109-million-parameter text JEPA, and a 0.838 CIDEr score on COCO after applying one rotation.

   Hugging Face · Papers · 🔥 0
2. Oct 7
   Study finds mixed-modality retrievers favour text, proposes Trident

   An arXiv paper systematically evaluates dense retrievers across architectures and finds their performance is highly sensitive to modality composition, following a pronounced V-shaped curve as image documents are progressively replaced with semantically corresponding text: strong on single-modality corpora but substantially degraded when modalities coexist. The authors call the finding "Chaos in the Text," noting that irrelevant text causes more severe degradation than an equal number of irrelevant images, because text representations receive systematically higher similarity scores and can outrank relevant images. They propose Trident, which builds text, image and fused text-image views of each document as co-equal positives and jointly optimizes relevance discrimination and positive-view balance via Multi-Positive View InfoNCE.

   Hugging Face · Papers · 🔥 0

Experience and discussion from the community

Share my CLIP experienceAsk about CLIP