A finite Coxeter group $W$ has a natural metric $d$ and if $\mathcal{M}$ is a subset of $W$, then for each $u\in W$, there is $q\in \mathcal{M}$ such that $d(u,q)=d(u,\mathcal{M})$. Such $q$ is not unique in general but if $\mathcal{M}$ is a Coxeter matroid, then it is unique, and we define a retraction $\mathcal{R}^m_{\mathcal{M}}\colon W\to \mathcal{M}\subset W$ so that $\mathcal{R}^m_{\mathcal{M}}(u)=q$. The $T$-fixed point set $Y^T$ of a $T$-orbit closure $Y$ in a flag variety $G/B$ is a Coxeter matroid, where $G$ is a semisimple algebraic group, $B$ is a Borel subgroup, and $T$ is a maximal torus of $G$ contained in $B$. We define a retraction $\mathcal{R}^g_{Y}\colon W\to Y^T\subset W$ geometrically, where $W$ is the Weyl group of $G$, and show that $\mathcal{R}^g_{Y}=\mathcal{R}^m_{Y^T}$. We introduce another retraction $\mathcal{R}^a_{\mathcal{M}}\colon W\to \mathcal{M}\subset W$ algebraically for an arbitrary subset $\mathcal{M}$ of $W$ when $W$ is a Weyl group of classical Lie type, and show that $\mathcal{R}^a_{\mathcal{M}}=\mathcal{R}^m_{\mathcal{M}}$ when $\mathcal{M}$ is a Coxeter matroid.