In this article, we study $2$-$(v,k,λ)$ designs $\mathcal{D}$ with $λ$ prime admitting flag-transitive and point-primitive almost simple automorphism groups $G$ with socle $T$ a finite exceptional simple group or a sporadic simple groups. If the socle of $G$ is a finite exceptional simple group, then we prove that $\mathcal{D}$ is isomorphic to one of two infinite families of $2$-designs with point-primitive automorphism groups, one is the Suzuki-Tits ovoid design with parameter set $(v,b,r,k,λ)=(q^{2}+1,q^{2}(q^{2}+1)/(q-1),q^{2},q,q-1)$ design, where $q-1$ is a Mersenne prime, and the other is newly constructed in this paper and has parameter set $(v,b,r,k,λ)=(q^{3}(q^{3}-1)/2,(q+1)(q^{6}-1),(q+1)(q^{3}+1),q^{3}/2,q+1)$, where $q+1$ a Fermat prime. If $T$ is a sporadic simple group, then we show that $\mathcal{D}$ is isomorphic to a unique design admitting a point-primitive automorphism group with parameter set $(v,b,r,k,λ)=(176,1100,50,2)$, $(12,22,11,6,5)$ or $(22,77,21,6,5)$.