The maximum likelihood principle is widely used in statistics, and the associated estimators often display good properties. indeed maximum likelihood estimators are guaranteed to be asymptotically efficient under mild conditions. However in some settings, one has too few samples to get a good estimation. It then becomes desirable to take into account prior information about the distribution one wants to estimate. One possible approach is to extend the maximum likelihood principle in a bayesian context, which then becomes a maximum a posteriori estimate; however this requires a distribution model on the distribution parameters. We shall therefore concentrate on the alternative approach of regularized estimators in this paper; we will show how they can be naturally introduced in the framework of maximum likelihood estimation, and how they can be extended to form robust estimators which can reject outliers.