An architectural framework, based on collaborative filtering using K-nearest neighbor and cosine similarity, was developed and implemented to fit the requirements for the company DecorRaid. The aim of the paper is to test different evaluation techniques within the environment to research the recommender systems performance. Three perspectives were found relevant for evaluating a recommender system in the specific environment, namely dataset, system and user perspective. With these perspectives it was possible to gain a broader view of the recommender systems performance. Online A/B split testing was conducted to compare the performance of small adjustments to the RS and to test the relevance of the evaluation techniques. Key factors are solving the sparsity and cold start problem, where the suggestion is to research a hybrid RS combining Content-based and CF based techniques.