The study of Boolean functions with low $c$-differential uniformity has become recently an important topic of research. However, in odd characteristic case, there are not many results on the ($c$-)differential uniformity of functions that are not power functions. In this paper, we investigate the $c$-differential uniformity of the swapped inverse functions in odd characteristic, and show that their $c$-differential uniformities are at most 6 except for some special case.