Naturally, university courses should be designed to attract students, engaging them to achieve learning goals. Toward this end, the use of Serious Games has been proposed in the literature. To address positive effects, such as content memorability and attendance rates, we propose Ambient Serious Games as games embedded in a computer-enriched environment, which is only partially perceived mentally by players. In this paper, we describe five technological key challenges that must be overcome to seamlessly and beneficially integrate an Ambient Serious Game into teaching. These challenges, derived from a scenario, focus on the technological provision and conduct of such games based on a software platform. They include (1) the integration of physical smart learning objects in heterogeneous environments under dynamic constraints, (2) the representation of abstract subject matter using smart learning objects, (3) the guided or automatic connection of all involved components, (4) the explanation of the components, their interaction, as well as the serious game itself, and (5) feedback on the game state.