Let $X=\{X(t),t\in {\mathbb{R}}^N\}$ be a centered Gaussian random field with stationary increments and $X(0)=0$. For any compact rectangle $T\subset {\mathbb{R}}^N$ and $u\in {\mathbb{R}}$, denote by $A_u=\{t\in T:X(t)\geq u\}$ the excursion set. Under $X(\cdot)\in C^2({\mathbb{R}}^N)$ and certain regularity conditions, the mean Euler characteristic of $A_u$, denoted by ${\mathbb{E}}\{\varphi(A_u)\}$, is derived. By applying the Rice method, it is shown that, as $u\to\infty$, the excursion probability ${\mathbb{P}}\{\sup_{t\in T}X(t)\geq u\}$ can be approximated by ${\mathbb{E}}\{\varphi(A_u)\}$ such that the error is exponentially smaller than ${\mathbb{E}}\{\varphi(A_u)\}$. This verifies the expected Euler characteristic heuristic for a large class of Gaussian random fields with stationary increments.