With the use of primaries which have increasingly narrow bandwidths in modern displays, observer metameric breakdown is becoming a significant factor. This can lead to discrepancies in the perceived color between different observers. If the spectral sensitivity of a user's eyes could be easily measured, next generation displays would be able to adjust the display content to ensure that the colors are perceived as intended by a given observer. We present a mathematical framework for calculating spectral sensitivities of a given human observer using a color matching experiment that could be done on a mobile phone display. This forgoes the need for expensive in-person experiments and allows system designers to easily calibrate displays to match the user's vision, in-the-wild. We show how to use sRGB pixel values along with a simple display model to calculate plausible color matching functions (CMFs) for the users of a given display device (e.g., a mobile phone). We evaluate the effect of different regularization functions on the shape of the calculated CMFs and the results show that a sum of squares regularizer is able to predict smooth and qualitatively realistic CMFs.