This paper focuses on the signal design in a Gaussian amplify-and-forward (AF) relay system with superposition coding (SC) being applied at the relay, which allows the source and relay to transmit their own information within two time slots. Practical quadrature amplitude modulation (QAM) constellations are adopted at the source and relay. To improve the system error performance, we optimize the weight coefficients adopted at the source and relay to maximize the minimum Euclidean distance of the received composite constellation, subject to their individual average power constraints. The formulated optimization problem is shown to be a mixed continuous-discrete one that is non-trivial to resolve in general. By resorting to the punched Farey sequence, we manage to obtain the optimal solution to the formulated problem by first partitioning the entire feasible region into a finite number of sub-intervals and then taking the maximum over all the possible sub-intervals. Simulation results are provided to demonstrate the superior performance of our proposed design based on SC over that using conventional time division multiple access (TDMA).