The design (shape) of a robot is usually decided before the control is implemented. This might limit how well the design is adapted to a task, as the suitability of the design is given by how well the robot performs in the task, which requires both a design and a controller. The co-optimization or simultaneous optimization of the design and control of robots addresses this limitation by producing a design and control that are both adapted to the task. This paper investigates some of the challenges inherent in the co-optimization of design and control in simulation. The results show that reducing how well the controllers are trained during the co-optimization process significantly improves the robot's performance when considering a second phase in which the controller for the best design is retrained with additional resources. In addition, the results demonstrate that the computation budget allocated to training the controller for each design influences design complexity, with simpler designs associated with lower training budgets. This paper experimentally studies key questions discussed in other works in the literature on the co-optimization of design and control of robots in simulation in four different co-optimization problems.