We prove a characterization of the amenability of countable Borel equivalence relations in terms of the uniform Liouville property for group actions on their classes. Furthermore, inspired by a well-known amenability criterion for locally compact groups due to Kesten, we study return probabilities for random walks, and in particular a limiting condition that we call Kesten's property, on general topological groups. We show that every amenable topological group with small invariant neighborhoods indeed has Kesten's property. For measurable lamplighter groups associated with countable Borel equivalence relations, we establish a connection between Kesten's property and anti-concentration inequalities for the inverted orbits of random walks on the equivalence classes. This allows us to construct an amenable contractible Polish group without Kesten's property.