mle.tools (version 1.0.0)

mle.tools-package: Overview of the ``mle.tools'' Package

Description

The current version of the mle.tools package has implemented three functions which are of great interest in maximum likelihood estimation. These functions calculates the expected /observed Fisher information and the bias-corrected maximum likelihood estimate(s) using the bias formula introduced by Cox and Snell (1968). They can be applied to any probability density function whose terms are available in the derivatives table of D function (see ``deriv.c'' source code for further details). Integrals, when required, are computed numerically via integrate function. Below are some mathematical details of how the returned values are calculated. Let \(X_{1},\ldots ,X_{n}\) be i.i.d. random variables with probability density functions \(f(x_{i}\mid \bold{\theta })\) depending on a \(p\)-dimensional parameter vector \(\bold{\theta } = (\theta_1,\ldots,\theta_p)\). The (j,k)-th element of the observed, \(H_{jk}\), and expected, \(I_{jk}\), Fisher information are calculated, respectively, as $$H_{jk} =\left. {-\sum\limits_{i=1}^{n}\frac{% \partial ^{2}}{\partial \theta _{j}\partial \theta _{k}}\log f\left( x_{i}\mid {\bold{\theta} }\right) }\right\vert _{\bold{\theta }=\widehat{\bold{% \theta }}}$$ and $$I_{jk}=-n\times E\left( \frac{\partial ^{2}}{\partial \theta _{j}\partial \theta _{k}}\log f\left( x\mid \bold{\theta }\right) \right) =\left. -n\times \int\limits_{\mathcal{X} }\frac{\partial ^{2}}{\partial \theta _{j}\partial \theta _{k}}\log f\left( x\mid \bold{\theta }\right) \times f\left( x\mid \bold{\theta }\right) dx\right\vert _{\bold{\theta }=\widehat{\bold{% \theta }}}$$ where \((j,k=1,\ldots,p)\), \(\bold{\widehat{\theta}}\) is the maximum likelihood estimate of \(\bold{\theta}\) and \(\mathcal{X}\) denotes the support of the random variable \(X\). The observed.varcov function returns the inputted maximum likelihood estimate(s) and the inverse of \(\bold{H}\) while the expected.varcov function returns the inputted maximum likelihood estimate(s) and the inverse of \(\bold{I}\). If \(\bold{H}\) and/or \(\bold{I}\) are singular an error message is returned. Furthermore, the bias corrected maximum likelihood estimate of \(\theta_s\) (\(s=1,\ldots,p)\), denoted by \(\widetilde{\theta_s}\), is calculated as \(\widetilde{\theta_s} = \widehat{\theta} - \widehat{Bias}(\widehat{\theta}_s)\), where \(\widehat{\theta}_s\) is the maximum likelihood estimate of \({\theta}_s\) and $${\widehat{Bias}\left( {\widehat{\theta }}_{s}\right) =}\left. {% \sum\limits_{j=1}^{p}\sum\limits_{k=1}^{p}\sum\limits_{l=1}^{p}\kappa ^{sj}\kappa ^{kl}\left[ 0.5\kappa _{{jkl}}+\kappa _{{jk,l}}\right] }% \right\vert _{\bold{\theta }=\widehat{\bold{\theta }}}$$ where \(\kappa ^{jk}\) is the (j,k)-th element of the inverse of the expected Fisher information, \({\kappa_{jkl}=} n\times E\left( \frac{\partial ^{3}}{\partial \theta _{j}\partial {{\theta}}_{k}{\theta }_{l}}\log f\left( x\mid \bold{\theta }\right) \right)\) and \(\kappa_{jk,l}= n \times E\left( \frac{\partial ^{2}}{\partial \theta _{j}\partial \theta_{k}}\log f\left( x\mid\bold{\theta }\right) \times \frac{\partial }{{\theta }_{l}}\log f\left( x\mid\bold{\theta }\right) \right) \). The bias-corrected maximum likelihood estimate(s) and some other quantities are calculated via coxsnell.bc function. If the numerical integration fails and/or \(\bold{I}\) is singular an error message is returned. It is noteworthy that for a series of probability distributions it is possible, after extensive algebra, to obtain the analytical expressions for \(Bias({\widehat{\theta}_s)}\). In Stosic and Cordeiro (2009) are the analytic expressions for 22 two-parameter continuous probability distributions. They also present the Maple and Mathematica scripts used to obtain all analytic expressions (see Cordeiro and Cribari-Neto 2014 for further details).

Arguments

References

Azzalini, A. (1996). Statistical Inference: Based on the Likelihood. London: Chapman and Hall. Cordeiro, G. M. and Cribari-Neto, F., (2014). An introduction to Bartlett correction and bias reduction. SpringerBriefs in Statistics, New-York. Cordeiro, G. M. and McCullagh, P., (1991). Bias correction in generalized linear models. Journal of the Royal Statistical Society, Series B, 53, 3, 629--643. Cox, D. R. and Hinkley, D. V. (1974). Theoretical Statistics. London: Chapman and Hall. Cox, D. R. and Snell, E. J., (1968). A general definition of residuals (with discussion). Journal of the Royal Statistical Society, Series B, 30, 2, 24--275. Efron, B. and Hinkley, D. V. (1978). Assessing the accuracy of the maximum likelihood estimator: Observed versus expected Fisher information. Biometrika, 65, 3, 457--482. Pawitan, Y. (2001). In All Likelihood: Statistical Modelling and Inference Using Likelihood. Oxford: Oxford University Press. Stosic, B. D. and Cordeiro, G. M., (2009). Using Maple and Mathematica to derive bias corrections for two parameter distributions. Journal of Statistical Computation and Simulation, 79, 6, 751--767.