We study the Ordered k-Median problem, in which the solution is evaluated by first sorting the client connection costs and then multiplying them with a predefined non-increasing weight vector (higher connection costs are taken with larger weights). Since the 1990s, this problem has been studied extensively in the discrete optimization and operations research communities and has emerged as a framework unifying many fundamental clustering and location problems such as k-Median and k-Center. This generality, however, renders the problem intriguing from the algorithmic perspective and obtaining non-trivial approximation algorithms was an open problem even for simple topologies such as trees. Recently, Aouad and Segev were able to obtain an O(log n) approximation algorithm for Ordered k-Median using a sophisticated local-search approach and the concept of surrogate models thereby extending the result by Tamir (2001) for the case of a rectangular weight vector, also known as k-Facility p-Centrum. In this paper, we provide an LP-rounding constant-factor approximation algorithm for the Ordered k-Median problem. We first provide a new analysis of the rounding process by Charikar and Li (2012) for k-Median, when applied to a fractional solution obtained from solving an LP relaxation over a non-metric, truncated cost vector, resulting in an elegant 15-approximation for the rectangular case. Then, we show that a simple weight bucketing can be applied to the general case resulting in O(log n) rectangles and hence in a constant-factor approximation in quasi-polynomial time. Finally, we show that also the clever distance bucketing by Aouad and Segev can be combined with the objective-oblivious version of our LP-rounding for the rectangular case, and that it results in a true, polynomial time, constant-factor approximation algorithm.