Spectral signatures are crucial in the era of large X-ray surveys. Automatic machine learning methods have proven useful in this respect, but so far they have not been applied to large spectral datasets, such as the Chandra Source Catalog (CSC). This work aims to develop a compact and physically meaningful representation of Chandra X-ray spectra using deep learning. To verify that the learned representation captures relevant information, we evaluate it through classification, regression, and interpretability analyses, and measure the mutual information between spectral and time-domain properties of these sources, aiding in the future identification of transient events. We use a transformer-based autoencoder to compress X-ray spectra into representations in an 8-dimensional latent space. Astrophysical source types and physical summary statistics are compiled from external catalogs. We evaluate the learned representation in terms of spectral reconstruction accuracy, clustering performance on 8 known astrophysical source classes, and correlation with physical quantities such as hardness ratios and hydrogen column densities ($N_H$). Upon reconstruction, clustering in the latent space yields a balanced classification accuracy of $\sim$40% across the 8 source classes, increasing to $\sim$69% when restricted to AGNs and stellar-mass compact objects exclusively. Moreover, latent features correlate with spectral and temporal properties, suggesting that the compressed representation captures physically relevant information. Features learned directly from X-ray spectra capture relevant physical information as effectively as human-extracted features that require additional computations. They can be used for both classification and regression in large surveys, and also share mutual information with time-domain properties. The method can be adapted to existing and upcoming X-ray catalogs.