MLE's bias pathology, Model Updated Maximum Likelihood Estimates and
Wallace's Minimum Message Length method
The inherent bias pathology of the maximum likelihood (ML) estimation method is confirmed for models with unknown parameters and when MLE is function of MLE To reduce 's bias the likelihood equation to be solved for is updated using the model for the data in it. Model updated (MU) MLE, often reduces either totally or partially 's bias when estimating shape parameter For the Pareto model reduces also 's variance. The results explain the difference that puzzled R. A. Fisher, between biased and the unbiased estimate he obtained for two models with the "2-stage procedure". MUMLE's implementation is equivalent to the abandoned 2-stage procedure thus justifying its use. MUMLE and Firth's bias correcting likelihood are also obtained with the Minimum Message Length method thus motivating its use in frequentist inference and, more generally, model updating with a prior distribution.
View on arXiv