Based on two involutions and a bijection, we completely determine the difference between the number of $\ell$-regular partitions of $n$ into an even number of parts and into an odd number of parts for all positive integers $n$ and $\ell>1$, which extends two recent results due to Ballantine and Merca. As an application, we provide a combinatorial proof of Hickerson's identity on the number of partitions into an even and odd number of parts.