In 1997, Van Hamme proposed 13 supercongruences on truncated hypergeometric series. Van Hamme's (B.2) supercongruence was first confirmed by Mortenson and received a WZ proof by Zudilin later. In 2012, using the WZ method again, Sun extended Van Hamme's (B.2) supercongruence to the modulus $p^4$ case, where $p$ is an odd prime. In this paper, by using a more general WZ pair, we generalize Hamme's (E.2) and (F.2) supercongruences, as well as two supercongruences by Swisher, to the modulus $p^4$ case. Our generalizations of these supercongruences are related to Euler polynomials. We also put forward a relevant conjecture on $q$-congruences for further study.