We explore the novel connection between rook placements on collections of cells, also known as pruned chessboards, and the algebraic properties of ideals generated by $2$-minors. We design an algorithm to compute the switching rook polynomial of a collection of cells and show that it coincides with the $h$-polynomial of the associated coordinate ring for all collections up to rank 10 and polyominoes up to rank 12. Motivated by this evidence, we conjecture that the correspondence holds in general, and we prove it for certain convex collections of cells by algebraic tools.