Constant envelope (CE) precoding is an appealing transmission technique, which enables highly efficient power amplification, and is realizable with a single radio frequency (RF) chain at the multi-antenna transmitter. In this paper, we study the transceiver design for a point-to-point multiple-input multiple-output (MIMO) system with CE precoding. Both single-stream transmission (i.e., beamforming) and multi-stream transmission (i.e., spatial multiplexing) are considered. For single-stream transmission, we optimize the receive beamforming vector to minimize the symbol error rate (SER) for any given channel realization and desired constellation at the combiner output. By reformulating the problem as an equivalent quadratically constrained quadratic program (QCQP), we propose an efficient semi-definite relaxation (SDR) based algorithm to find an approximate solution. Next, for multi-stream transmission, we propose a new scheme based on antenna grouping at the transmitter and minimum mean squared error (MMSE) or zero-forcing (ZF) based beamforming at the receiver. The transmit antenna grouping and receive beamforming vectors are then jointly designed to minimize the maximum SER over all data streams. Finally, the error-rate performance of single- versus multi-stream transmission is compared via simulations under different setups.