Typically, recommender systems from any domain, be it movies, music, restaurants, etc., are organized in a centralized fashion. The service provider holds all the data, biases in the recommender algorithms are not transparent to the user, and the service providers often create lock-in effects making it inconvenient for the user to switch providers. In this paper, we argue that the user's smartphone already holds a lot of the data that feeds into typical recommender systems for movies, music, or POIs. With the ubiquity of the smartphone and other users in proximity in public places or public transportation, data can be exchanged directly between users in a device-to-device manner. This way, each smartphone can build its own database and calculate its own recommendations. One of the benefits of such a system is that it is not restricted to recommendations for just one user - ad-hoc group recommendations are also possible. While the infrastructure for such a platform already exists - the smartphones already in the palms of the users - there are challenges both with respect to the mobile recommender system platform as well as to its recommender algorithms. In this paper, we present a mobile architecture for the described system - consisting of data collection, data exchange, and recommender system - and highlight its challenges and opportunities.