Recent years have witnessed a growing focus on automated software vulnerability detection. Notably, deep learning (DL)-based methods, which employ source code for the implicit acquisition of vulnerability patterns, have demonstrated superior performance compared to other approaches. However, the DL-based approaches are still hard to capture the vulnerability-related information from the whole code snippet, since the vulnerable parts usually account for only a small proportion. As evidenced by our experiments, the approaches tend to excessively emphasize semantic information, potentially leading to limited vulnerability detection performance in practical scenarios. First, they cannot well distinguish between the code snippets before (i.e., vulnerable code) and after (i.e., non-vulnerable code) developers' fixes due to the minimal code changes. Besides, substituting user-defined identifiers with placeholders (e.g., "VAR1" and "FUN1") in obvious performance degradation at up to 14.53% with respect to the F1 score. To mitigate these issues, we propose to leverage the vulnerable and corresponding fixed code snippets, in which the minimal changes can provide hints about semantic-agnostic features for vulnerability detection. In this paper, we propose a software vulneRability dEteCtion framework with zerO-sum game and prototype learNing, named RECON. In RECON, we propose a zero-sum game construction module. Distinguishing the vulnerable code from the corresponding fixed code is regarded as one player (i.e. Calibrator), while the conventional vulnerability detection is another player (i.e. Detector) in the zero-sum game. The goal is to capture the semantic-agnostic features of the first player for enhancing the second player's performance for vulnerability detection. Experiments on the public benchmark dataset show that RECON outperforms the state-of-the-art baseline by 6.29% in F1 score.