In this letter, we propose a vector quantized-variational autoencoder (VQ-VAE)-based feedback scheme for robust precoder design in multi-user frequency division duplex (FDD) systems. We demonstrate how the VQ-VAE can be tailored to specific propagation environments, focusing on systems with low pilot overhead, which is crucial in massive multiple-input multiple-output (MIMO). Extensive simulations with real-world measurement data show that our proposed feedback scheme outperforms state-of-the-art autoencoder (AE)-based compression schemes and conventional Discrete Fourier transform (DFT) codebook-based schemes. These improvements enable the deployment of systems with fewer feedback bits or pilots.