We study minimum cost constraint satisfaction problems (MinCostCSP) through the algebraic lens. We show that for any constraint language $Γ$ which has the dual discriminator operation as a polymorphism, there exists a $|D|$-approximation algorithm for MinCostCSP$(Γ)$ where $D$ is the domain. Complementing our algorithmic result, we show that any constraint language $Γ$ where MinCostCSP$(Γ)$ admits a constant-factor approximation must have a \emph{near-unanimity} (NU) polymorphism unless P = NP, extending a similar result by Dalmau et al. on MinCSPs. These results imply a dichotomy of constant-factor approximability for constraint languages that contain all permutation relations (a natural generalization for Boolean CSPs that allow variable negation): either MinCostCSP$(Γ)$ has an NU polymorphism and is $|D|$-approximable, or it does not have any NU polymorphism and is NP-hard to approximate within any constant factor. Finally, we present a constraint language which has a majority polymorphism, but is nonetheless NP-hard to approximate within any constant factor assuming the Unique Games Conjecture, showing that the condition of having an NU polymorphism is in general not sufficient unless UGC fails.