Consider a random walk $S_n=\sum_{i=1}^n X_i$ with independent and identically distributed real-valued increments with zero mean, finite variance and moment of order $2 + δ$ for some $δ>0$. For any starting point $x\in \mathbb R$, let $τ_x = \inf \left\{ k\geq 1: x+S_{k} < 0 \right\}$ denote the first time when the random walk $x+S_n$ exits the half-line $[0,\infty)$. We investigate the uniform asymptotic behavior over $x\in \mathbb R$ of the persistence probability $\mathbb P (τ_x >n)$ and the joint distribution $\mathbb{P} \left( x + S_n \leq u, τ_x > n \right)$, for $u\geq 0$, as $n \to \infty$. New limit theorems for these probabilities are established based on the heat kernel approximations. Additionally, we evaluate the rate of convergence by proving Berry-Esseen type bounds.