In a frequency division duplexing (FDD) massive multiple-input multiple-output (MIMO) system, the acquisition of downlink channel state information (CSI) at base station (BS) is a very challenging task due to the overwhelming overheads required for downlink training and uplink feedback. In this paper, we reveal a deterministic uplink-to-downlink mapping function when the position-to-channel mapping is bijective. Motivated by the universal approximation theorem, we then propose a sparse complex-valued neural network (SCNet) to approximate the uplink-to-downlink mapping function. Different from general deep networks that operate in the real domain, the SCNet is constructed in the complex domain and is able to learn the complex-valued mapping function by off-line training. After training, the SCNet is used to directly predict the downlink CSI based on the estimated uplink CSI without the need of either downlink training or uplink feedback. Numerical results show that the SCNet achieves better performance than general deep networks in terms of prediction accuracy and exhibits remarkable robustness over complicated wireless channels, demonstrating its great potential for practical deployments.