We study infinite analogues of expander graphs, namely graphs where subgraphs weighted by heat kernels form an expander family. Our main result is that there does not exist any infinite expander in this sense. This proves the analogue for random walks of Benjamini's conjecture that there is no infinite graph whose metric balls are uniformly expander. The proof relies on a study of stationary random graphs, in particular proving non-expansion of heat kernels in that setting. A key result is that any stationary random graph is stationary hyperfinite, which is potentially of independent interest.