Natural language processing techniques, such as Word2Vec, have demonstrated exceptional capabilities in capturing semantic and syntactic relationships of text through vector embeddings. Inspired by this technique, we propose CSI2Vec, a self-supervised framework for generating universal and robust channel state information (CSI) representations tailored to CSI-based positioning (POS) and channel charting (CC). CSI2Vec learns compact vector embeddings across various wireless scenarios, capturing spatial relationships between user equipment positions without relying on CSI reconstruction or ground-truth position information. We implement CSI2Vec as a neural network that is trained across various deployment setups (i.e., the spatial arrangement of radio equipment and scatterers) and radio setups (RSs) (i.e., the specific hardware used), ensuring robustness to aspects such as differences in the environment, the number of used antennas, or allocated set of subcarriers. CSI2Vec abstracts the RS by generating compact vector embeddings that capture essential spatial information, avoiding the need for full CSI transmission or reconstruction while also reducing complexity and improving processing efficiency of downstream tasks. Simulations with ray-tracing and real-world CSI datasets demonstrate CSI2Vec's effectiveness in maintaining excellent POS and CC performance while reducing computational demands and storage.