User experience (UX) evaluation is gaining increased interest lately, both from academia and industry. In this paper we argue that UX evaluation needs to fulfill two important requirements: scalability, i.e. the ability to provide useful feedback in different stages of the design, and diversity, i.e. the ability to reflect the di-versity of opinions that may exist in different users. We promote the use of the Repertory Grid Technique as a promising UX evaluation technique and discuss some of our concerns regarding the quantitative side of its use.