The functions allow for the numerical evaluation of some commonly used entropy measures, such as Shannon entropy, Rényi entropy, Havrda and Charvat entropy, and Arimoto entropy, at selected parametric values from several well-known and widely used probability distributions. Moreover, the functions also compute the relative loss of these entropies using the truncated distributions. Let \(X\) be an absolutely continuous random variable having the probability density function \(f(x)\). Then, the Shahnon entropy is as follows: $$ H(X)=-\intop_{-\infty}^{+\infty}f(x)\log f(x)dx. $$ The Rényi entropy is as follows: $$ H_{\delta}(X)=\frac{1}{1-\delta}\log\intop_{-\infty}^{+\infty}f(x)^{\delta}dx;\qquad\delta>0,\delta\ne1. $$ The Havrda and Charvat entropy is as follows: $$ H_{\delta}(X)=\frac{1}{2^{1-\delta}-1}\left(\intop_{-\infty}^{+\infty}f(x)^{\delta}dx-1\right);\qquad\delta>0,\delta\ne1. $$ The Arimoto entropy is as follows: $$ H_{\delta}(X)=\frac{\delta}{1-\delta}\left[\left(\intop_{-\infty}^{+\infty}f(x)^{\delta}dx\right)^{\frac{1}{\delta}}-1\right];\qquad\delta>0,\delta\ne1. $$
Let \(D(X)\) be an entropy, and \(D_p(X)\) be its truncated integral version at \(p\), i.e., defined with the truncated version of \(f(x)\) over the interval \((-\infty,p)\). Then we define the corresponding relative loss entropy is defined by
$$ S_D(p)= \frac{D(X)-D_p(X)}{D(X)}. $$
Muhammad Imran imranshakoor84@yahoo.com, Christophe Chesneau christophe.chesneau@unicaen.fr and Farrukh Jamal farrukh.jamal@iub.edu.pk.
Muhammad Imran <imranshakoor84@yahoo.com>
| Package: | shannon |
| Type: | Package |
| Version: | 0.2.0 |
| Date: | 2024-08-21 |
| License: | GPL-2 |
Shannon, C. E. (1948). A mathematical theory of communication. The Bell system technical journal, 27(3), 379-423.
Rényi, A. (1961). On measures of entropy and information, Hungarian Academy of Sciences, Budapest, Hungary, 547- 561.
Havrda, J., & Charvat, F. (1967). Quantification method of classification processes. Concept of structural \(\alpha\)-entropy. Kybernetika, 3(1), 30-35.
Arimoto, S. (1971). Information-theoretical considerations on estimation problems. Information and control, 19(3), 181-194.
Awad, A. M., & Alawneh, A. J. (1987). Application of entropy to a life-time model. IMA Journal of Mathematical Control and Information, 4(2), 143-148.