We propose a general framework for the study of the genealogy of neutral discrete-time populations. We remove the standard assumption of exchangeability of offspring distributions appearing in Cannings' models, and replace it by a less restrictive condition of non-heritability of reproductive success. We provide a general criterion for the weak convergence of their genealogies to $Ξ$-coalescents, and apply it to a simple parametrization of our scenario (which, under mild conditions, we also prove to essentially include the general case). We provide examples for such populations, including models with highly-asymmetric offspring distributions and populations undergoing random but recurrent bottlenecks. Finally we study the limit genealogy of a new exponential model which, as previously shown for related models and in spite of its built in (fitness) inheritance mechanism, can be brought into our setting.