Study reveals vision and language models develop similar internal structure but organize it differently

Researchers analyzing cross-modal alignment between pretrained vision and language encoders discovered they develop comparable manifold complexity yet organize representations fundamentally differently, creating a "spectral complexity-orientation gap." The finding suggests independent training converges models on what to represent but diverges on how, informing design of better multimodal alignment methods.