In this paper, we obtain a number of new infinite families of Hadamard matrices. Our constructions are based on four new constructions of difference families with four or eight blocks. By applying the Wallis-Whiteman array or the Kharaghani array to the difference families constructed, we obtain new Hadamard matrices of order $4(uv+1)$ for $u=2$ and $v\in Φ_1\cup Φ_2 \cup Φ_3 \cup Φ_4$; and for $u\in \{3,5\}$ and $v\in Φ_1\cup Φ_2 \cup Φ_3$. Here, $Φ_1=\{q^2:q\equiv 1\pmod{4}\mbox{ is a prime power}\}$, $Φ_2=\{n^4\in \mathbb{N}:n\equiv 1\pmod{2}\} \cup \{9n^4\in \mathbb{N}:n\equiv 1\pmod{2}\}$, $Φ_3=\{5\}$ and $Φ_4=\{13,37\}$. Moreover, our construction also yields new Hadamard matrices of order $8(uv+1)$ for any $u\in Φ_1\cup Φ_2$ and $v\in Φ_1\cup Φ_2 \cup Φ_3$.