We develop an analogue of probability theory for probabilities taking values in topological groups. We generalize Kolmogorov's method of axiomatization of probability theory: main distinguishing features of frequency probabilities are taken as axioms in the measure-theoretic approach. We also present a review of non-Kolmogorovian probabilistic models including models with negative, complex, and $p$-adic valued probabilities. The latter model is discussed in details. The introduction of $p$-adic (as well as more general non-Archimedean) probabilities is one of the main motivations for consideration of generalized probabilities taking values in topological groups which are distinct from the field of real numbers. We discuss applications of non-Kolmogorovian models in physics and cognitive sciences. An important part of this paper is devoted to statistical interpretation of probabilities taking values in topological groups (and in particular in non-Archimedean fields).