In this work, we study the extension of two variants of the facility location problem (FL) to make them robust towards a few distantly located clients. First, $k$-facility location problem ($k$FL), a common generalization of FL and $k$ median problems, is a well studied problem in literature. In the second variant, lower bounded facility location (LBFL), we are given a bound on the minimum number of clients that an opened facility must serve. Lower bounds are required in many applications like profitability in commerce and load balancing in transportation problem. In both the cases, the cost of the solution may be increased grossly by a few distantly located clients, called the outliers. Thus, in this work, we extend $k$FL and LBFL to make them robust towards the outliers. For $k$FL with outliers ($k$FLO) we present the first (constant) factor approximation violating the cardinality requirement by +1. As a by-product, we also obtain the first approximation for FLO based on LP-rounding. For LBFLO, we present a tri-criteria solution with a trade-off between the violations in lower bounds and the number of outliers. With a violation of $1/2$ in lower bounds, we get a violation of $2$ in outliers.