Given two polynomials $p(x), q(x)$ of degree $d$, we give a combinatorial formula for the finite free cumulants of $p(x)\boxtimes_d q(x)$. We show that this formula admits a topological expansion in terms of non-crossing multi-annular permutations on surfaces of different genera. This topological expansion, on the one hand, deepens the connection between the theories of finite free probability and free probability, and in particular proves that $\boxtimes_d$ converges to $\boxtimes$ as $d$ goes to infinity. On the other hand, borrowing tools from the theory of second order freeness, we use our expansion to study the infinitesimal distribution of certain families of polynomials which include Hermite and Laguerre, and draw some connections with the theory of infinitesimal distributions for real random matrices. Finally, building off our results we give a new short and conceptual proof of a recent result [Steinerberger (2020), Hoskins and Kabluchko (2020)] that connects root distributions of polynomial derivatives with free fractional convolution powers.