Evolutionary Computation has been successfully used to synthesise controllers for embodied agents and multi-agent systems in general. Notwithstanding this, continuous on-line adaptation by the means of evolutionary algorithms is still under-explored, especially outside the evolutionary robotics domain. In this paper, we present an on-line evolutionary programming algorithm that searches in the agent design space for the appropriate behavioural policies to cope with the underlying environment. We discuss the current problems of continuous agent adaptation, present our on-line evolution testbed for evolutionary simulation.