The Uncapacitated Facility Location (UFL) problem is one of the most fundamental clustering problems: Given a set of clients $C$ and a set of facilities $F$ in a metric space $(C \cup F, dist)$ with facility costs $open : F \to \mathbb{R}^+$, the goal is to find a set of facilities $S \subseteq F$ to minimize the sum of the opening cost $open(S)$ and the connection cost $d(S) := \sum_{p \in C} \min_{c \in S} dist(p, c)$. An algorithm for UFL is called a Lagrangian Multiplier Preserving (LMP) $α$ approximation if it outputs a solution $S\subseteq F$ satisfying $open(S) + d(S) \leq open(S^*) + αd(S^*)$ for any $S^* \subseteq F$. The best-known LMP approximation ratio for UFL is at most $2$ by the JMS algorithm of Jain, Mahdian, and Saberi based on the Dual-Fitting technique. We present a (slightly) improved LMP approximation algorithm for UFL. This is achieved by combining the Dual-Fitting technique with Local Search, another popular technique to address clustering problems. From a conceptual viewpoint, our result gives a theoretical evidence that local search can be enhanced so as to avoid bad local optima by choosing the initial feasible solution with LP-based techniques. Using the framework of bipoint solutions, our result directly implies a (slightly) improved approximation for the $k$-Median problem from 2.6742 to 2.67059.