Kernel smoothing for data from 1- to 6-dimensions.

There are three main types of functions in this package:

computing kernel estimators - these function names begin with `k'

computing bandwidth selectors - these begin with `h' (1-d) or `H' (>1-d)

displaying kernel estimators - these begin with `plot'.

The kernel used throughout is the normal (Gaussian) kernel \(K\). For 1-d data, the bandwidth \(h\) is the standard deviation of the normal kernel, whereas for multivariate data, the bandwidth matrix \(\bold{{\rm H}}\) is the variance matrix.

--For kernel density estimation, `kde`

computes
$$\hat{f}(\bold{x}) = n^{-1} \sum_{i=1}^n K_{\bold{{\rm H}}} (\bold{x} - \bold{X}_i).$$

The bandwidth matrix \(\bold{{\rm H}}\) is a matrix of smoothing
parameters and its choice is crucial for the performance of kernel
estimators. For display, its `plot`

method calls `plot.kde`

.

--For kernel density estimation, there are several varieties of bandwidth selectors

least squares (or unbiased) cross validation (LSCV or UCV)

`hlscv`

(1-d);`Hlscv`

,`Hlscv.diag`

(2- to 6-d)smoothed cross validation (SCV)

`hscv`

(1-d);`Hscv`

,`Hscv.diag`

(2- to 6-d)

--For kernel density derivative estimation, the main function is `kdde`

$${\sf D}^{\otimes r}\hat{f}(\bold{x}) = n^{-1} \sum_{i=1}^n {\sf
D}^{\otimes r}K_{\bold{{\rm H}}} (\bold{x} -
\bold{X}_i).$$
The bandwidth selectors are a modified subset of those for
`kde`

, i.e. `Hlscv`

, `Hns`

, `Hpi`

, `Hscv`

with `deriv.order>0`

.
Its `plot`

method is `plot.kdde`

for plotting each
partial derivative singly.

--For kernel discriminant analysis, the main function is
`kda`

which computes density estimates for each the
groups in the training data, and the discriminant surface.
Its `plot`

method is `plot.kda`

. The wrapper function
`hkda`

, `Hkda`

computes
bandwidths for each group in the training data for `kde`

,
e.g. `hpi`

, `Hpi`

.

--For kernel functional estimation, the main function is
`kfe`

which computes the \(r\)-th order integrated density functional
$$\hat{{\bold \psi}}_r = n^{-2} \sum_{i=1}^n \sum_{j=1}^n {\sf D}^{\otimes r}K_{\bold{{\rm H}}}(\bold{X}_i-\bold{X}_j).$$ The plug-in selectors are `hpi.kfe`

(1-d), `Hpi.kfe`

(2- to 6-d).
Kernel functional estimates are usually not required to computed
directly by the user, but only within other functions in the package.

--For kernel-based 2-sample testing, the main function is
`kde.test`

which computes the integrated
\(L_2\) distance between the two density estimates as the test
statistic, comprising a linear combination of 0-th order kernel
functional estimates:
$$\hat{T} = \hat{\psi}_{0,1} + \hat{\psi}_{0,2} - (\hat{\psi}_{0,12} +
\hat{\psi}_{0,21}),$$ and the corresponding p-value. The \(\psi\) are
zero order kernel functional estimates with the subscripts indicating
that 1 = sample 1 only, 2 = sample 2 only, and 12, 21 =
samples 1 and 2. The bandwidth selectors are `hpi.kfe`

,
`Hpi.kfe`

with `deriv.order=0`

.

--For kernel-based local 2-sample testing, the main function is
`kde.local.test`

which computes the squared distance
between the two density estimates as the test
statistic $$\hat{U}(\bold{x}) = [\hat{f}_1(\bold{x}) - \hat{f}_2(\bold{x})]^2$$ and the corresponding local
p-values. The bandwidth selectors are those used with `kde`

,
e.g. `hpi, Hpi`

.

--For kernel cumulative distribution function estimation, the main
function is `kcde`

$$\hat{F}(\bold{x}) = n^{-1} \sum_{i=1}^n
\mathcal{K}_{\bold{{\rm H}}} (\bold{x} - \bold{X}_i)$$
where \(\mathcal{K}\) is the integrated kernel.
The bandwidth selectors are `hpi.kcde`

,
`Hpi.kcde`

. Its `plot`

method is
`plot.kcde`

.
There exist analogous functions for the survival function \(\hat{\bar{F}}\).

--For kernel estimation of a ROC (receiver operating characteristic)
curve to compare two samples from \(\hat{F}_1,
\hat{F}_2\), the main function is `kroc`

$$(\hat{F}_{\hat{Y}_1}(z),
\hat{F}_{\hat{Y}_2}(z))$$ based on the cumulative distribution functions of
\(\hat{Y}_j = \hat{\bar{F}}_1(\bold{X}_j), j=1,2\).

The bandwidth selectors are those used with `kcde`

,
e.g. `hpi.kcde, Hpi.kcde`

for
\(\hat{F}_{\hat{Y}_j}, \hat{\bar{F}}_1\). Its `plot`

method
is `plot.kroc`

.

--For kernel estimation of a copula, the
main function is `kcopula`

$$\hat{C}(\bold{z}) = \hat{F}(\hat{F}_1^{-1}(z_1), \dots,
\hat{F}_d^{-1}(z_d))$$
where \(\hat{F}_j^{-1}(z_j)\) is
the \(z_j\)-th quantile of of the \(j\)-th marginal
distribution \(\hat{F}_j\).
The bandwidth selectors are those used with `kcde`

for
\(\hat{F}, \hat{F}_j\).
Its `plot`

method is `plot.kcde`

.

--For kernel mean shift clustering, the main function is `kms`

. The mean shift recurrence relation of the
candidate point \({\bold x}\)
$${\bold x}_{j+1} = {\bold x}_j + \bold{{\rm H}} {\sf D} \hat{f}({\bold
x}_j)/\hat{f}({\bold x}_j),$$
where \(j\geq 0\) and \({\bold x}_0 = {\bold x}\),
is iterated until \({\bold x}\) converges to its
local mode in the density estimate \(\hat{f}\) by following
the density gradient ascent paths. This mode determines the cluster
label for \(\bold{x}\).
The bandwidth selectors are those used with `kdde(,deriv.order=1)`

.

-- For kernel feature significance, the main function
`kfs`

. The hypothesis test at a point \(\bold{x}\) is
\(H_0(\bold{x}): \mathsf{H} f(\bold{x}) < 0\),
i.e. the density Hessian matrix \(\mathsf{H} f(\bold{x})\) is negative definite.
The test statistic is
$$W(\bold{x}) = \Vert
\mathbf{S}(\bold{x})^{-1/2} \mathrm{vech} \ \mathsf{H} \hat{f} (\bold{x})\Vert ^2$$
where \({\sf H}\hat{f}\) is the
Hessian estimate, vech is the vector-half operator, and
\(\mathbf{S}\) is an estimate of the null variance.
\(W(\bold{x})\) is
approximately \(\chi^2\) distributed with
\(d(d+1)/2\) degrees of freedom.
If \(H_0(\bold{x})\) is rejected, then \(\bold{x}\)
belongs to a significant modal region.
The bandwidth selectors are those used with
`kdde(,deriv.order=2)`

. Its `plot`

method is
`plot.kfs`

.

--Binned kernel estimation is an approximation to the exact kernel estimation and is available for d=1, 2, 3, 4. This makes kernel estimators feasible for large samples.

--For an overview of this package with 2-d density estimation, see
`vignette("kde")`

.

Bowman, A. & Azzalini, A. (1997) *Applied Smoothing Techniques
for Data Analysis*. Oxford University Press, Oxford.

Duong, T. (2004) *Bandwidth Matrices for Multivariate Kernel Density
Estimation.* Ph.D. Thesis, University of Western Australia.

Scott, D.W. (1992) *Multivariate Density Estimation: Theory,
Practice, and Visualization*. John Wiley & Sons, New York.

Silverman, B. (1986) *Density Estimation for Statistics and
Data Analysis*. Chapman & Hall/CRC, London.

Simonoff, J. S. (1996) *Smoothing Methods in Statistics*.
Springer-Verlag, New York.

Wand, M.P. & Jones, M.C. (1995) *Kernel Smoothing*. Chapman &
Hall/CRC, London.

feature, sm, KernSmooth