We survey the known group properties that a sequence of finite groups or group actions needs to satisfy to admit subsets of bounded cardinality producing expander Cayley or Schreier graphs. We prove that an infinite amenable group and solvable groups of bounded derived length do not produce expander Schreier graphs, generalizing with easier proofs results of Lubotzky and Weiss for Cayley graphs. In particular, the poor expansion properties of a group action cannot in general be detected by looking at the abelian sections or at the representations above the stabilizer of a point.