False information detection on social media is challenging as it commonly requires tedious evidence-collecting but lacks available comparative information. Clues mined from user comments, as the wisdom of crowds, could be of considerable benefit to this task. However, it is non-trivial to capture the complex semantics from the contents and comments in consideration of their implicit correlations. Although deep neural networks have good expressive power, one major drawback is the lack of explainability. In this paper, we focus on how to learn from the post contents and related comments in social media to understand and detect the false information more effectively, with explainability. We thus propose a Quantum-probability based Signed Attention Network (QSAN) that integrates the quantum-driven text encoding and a novel signed attention mechanism in a unified framework. QSAN is not only able to distinguish important comments from the others, but also can exploit the conflicting social viewpoints in the comments to facilitate the detection. Moreover, QSAN is advantageous with its explainability in terms of transparency due to quantum physics meanings and the attention weights. Extensive experiments on real-world datasets show that our approach outperforms state-of-the-art baselines and can provide different kinds of user comments to explain why a piece of information is detected as false.