This paper presents a new technique for providing the analysis and comparison of wiretap codes in the small blocklength regime over the binary erasure wiretap channel. A major result is the development of Monte Carlo strategies for quantifying a code's equivocation, which mirrors techniques used to analyze normal error correcting codes. For this paper, we limit our analysis to coset-based wiretap codes, and make several comparisons of different code families at small and medium blocklengths. Our results indicate that there are security advantages to using specific codes when using small to medium blocklengths.