Statistical inference as a formal scientific method to covert experience to knowledge has proven to be elusively difficult. While frequentist and Bayesian methodologies have been accepted in the contemporary era as two dominant schools of thought, it has been a good part of the last hundred years to see growing interests in development of more sound methods, both philosophically, in terms of scientific meaning of inference, and mathematically, in terms of exactness and efficiency. These include Fisher's fiducial argument, the Dempster-Shafe theory of belief functions, generalized fiducial, Confidence Distributions, and the most recently proposed inferential framework, called Inferential Models. Since it is notoriously challenging to make exact and efficient inference about the Cauchy distribution, this article takes it as an example to elucidate different schools of thought on statistical inference. It is shown that the standard approach of Inferential Models produces exact and efficient prior-free probabilistic inference on the location and scale parameters of the Cauchy distribution, whereas all other existing methods suffer from various difficulties.