Switching is an operation on a graph that does not change the spectrum of the adjacency matrix, thus producing cospectral graphs. An important activity in the field of spectral graph theory is the characterization of graphs by their spectrum. Hence, switching provides a tool for disproving the existence of such a characterization. This paper presents a general framework for counting the number of graphs that have a non-isomorphic cospectral graph through a switching method, expanding on the work by Haemers and Spence [European Journal of Combinatorics, 2004]. Our framework is based on a different counting approach, which allows it to be used for all known switching methods for the adjacency matrix. From this, we derive asymptotic results, which we complement with computer enumeration results for graphs up to $10$ vertices.