In 2017, Giudici, Li and the third author constructed the first known family of vertex-primitive $2$-arc-transitive digraphs of valency at least $2$. The smallest digraph in this family admits $\mathrm{PSL}_3(49)$ acting $2$-arc-transitively with vertex-stabilizer $\mathrm{A}_6$ and hence has $30758154560$ vertices. In this paper, we prove that this digraph is the vertex-primitive $2$-arc-transitive digraph of valency at least $2$ with fewest vertices.