Harnessing the Universal Geometry of Embeddings

This academic paper presents vec2vec, a novel method for translating text embeddings between different models without requiring paired data or prior knowledge of the encoders. The authors demonstrate that this unsupervised technique successfully aligns embeddings from various models into a universal latent space, preserving the geometric structure and semantics of the original data. They show that these translated embeddings can then be used to extract sensitive information from documents, even when only the embedding vectors are available, highlighting potential security implications for vector databases. Experiments on diverse datasets and model pairs, including cross-modal translations with CLIP, reveal that vec2vec significantly outperforms baseline methods and provides strong evidence for a "Strong Platonic Representation Hypothesis" in text embeddings.