This paper considers the transceiver design for uplink massive multiple-input multiple-output (MIMO) systems with channel sparsity in the angular domain. Recent progress has shown that sparsity-learning-based blind signal detection is able to retrieve the channel and data by using massage-passing based sparse matrix factorization methods. Short pilots sequences are inserted into user packets to eliminate the so-called phase and permutation ambiguities inherent in sparse matrix factorization. In this paper, to exploit the knowledge of these short pilot sequences more efficiently, we propose a semi-blind channel-and-signal estimation (SCSE) scheme in which the knowledge of the pilot sequences are integrated into the message passing algorithm for sparse matrix factorization. The SCSE algorithm involves enumeration over all possible user permutations, and so is time-consuming when the number of users is relatively large. To reduce complexity, we further develop the simplified SCSE (S-SCSE) to accommodate systems with a large number of users. We show that our semi-blind signal detection scheme substantially outperforms the state-of-the-art blind detection and training-based schemes in the short-pilot regime.