For a given permutation or set partition there is a natural way to assign a genus. Counting all permutations or partitions of a fixed genus according to cycle lengths or block sizes, respectively, is the main content of this article. After a variable transformation, the generating series are rational functions with poles located at the ramification points in the new variable. The generating series for any genus is given explicitly for permutations and up to genus 2 for set partitions. Extending the topological structure not just by the genus but also by adding more boundaries, we derive the generating series of non-crossing partitions on the cylinder from known results of non-crossing permutations on the cylinder. Most, but not all, outcomes of this article are special cases of already known results, however they are not represented in this way in the literature, which however seems to be the canonical way. To make the article as accessible as possible, we avoid going into details into the explicit connections to Topological Recursion and Free Probability Theory, where the original motivation came from.