npudist: Kernel Distribution Estimation with Mixed Data Types

Description

npudist computes kernel unconditional cumulative distribution estimates on evaluation data, given a set of training data and a bandwidth specification (a dbandwidth object or a bandwidth vector, bandwidth type, and kernel type) using the method of Li, Li and Racine (2017).

Usage

npudist(bws, ...)
# S3 method for formula
npudist(bws, data = NULL, newdata = NULL, ...)
# S3 method for dbandwidth
npudist(bws,
        tdat = stop("invoked without training data 'tdat'"),
        edat,
        ...)
# S3 method for call
npudist(bws, ...)
# S3 method for default
npudist(bws, tdat, ...)

Value

npudist returns a npdistribution object. The generic accessor functions fitted and se

extract estimated values and asymptotic standard errors on estimates, respectively, from the returned object. Furthermore, the functions

predict, summary and plot

support objects of both classes. The returned objects have the following components:

eval: the evaluation points.
dist: estimate of the cumulative distribution at the evaluation points
derr: standard errors of the cumulative distribution estimates

Arguments

bws: a dbandwidth specification. This can be set as a dbandwidth object returned from an invocation of npudistbw, or as a \(p\)-vector of bandwidths, with an element for each variable in the training data. If specified as a vector, then additional arguments will need to be supplied as necessary to change them from the defaults to specify the bandwidth type, kernel types, training data, and so on.
...: additional arguments supplied to specify the training data, the bandwidth type, kernel types, and so on. This is necessary if you specify bws as a \(p\)-vector and not a dbandwidth object, and you do not desire the default behaviours. To do this, you may specify any of bwscaling, bwtype, ckertype, ckerorder, okertype, as described in npudistbw.
tdat: a \(p\)-variate data frame of sample realizations (training data) used to estimate the cumulative distribution. Defaults to the training data used to compute the bandwidth object.
edat: a \(p\)-variate data frame of cumulative distribution evaluation points. By default, evaluation takes place on the data provided by tdat.
data: an optional data frame, list or environment (or object coercible to a data frame by as.data.frame) containing the variables in the model. If not found in data, the variables are taken from environment(bws), typically the environment from which npudistbw was called.
newdata: An optional data frame in which to look for evaluation data. If omitted, the training data are used.

Author

Tristen Hayfield tristen.hayfield@gmail.com, Jeffrey S. Racine racinej@mcmaster.ca

Usage Issues

If you are using data of mixed types, then it is advisable to use the data.frame function to construct your input data and not cbind, since cbind will typically not work as intended on mixed data types and will coerce the data to the same type.

Details

Typical usages are (see below for a complete list of options and also the examples at the end of this help file)


    Usage 1: first compute the bandwidth object via npudistbw and then
    compute the cumulative distribution:
    
    ## Start npRmpi for interactive execution. If slaves are already running and
    ## `options(npRmpi.reuse.slaves=TRUE)` (default on some systems), this will
    ## reuse the existing pool instead of respawning. To change the number of
    ## slaves, call `npRmpi.stop(force=TRUE)` then restart.
    npRmpi.start(nslaves=1)
    mpi.bcast.Robj2slave(y)
    mpi.bcast.cmd(bw <- npudistbw(~y),caller.execute=TRUE)
    mpi.bcast.cmd(Fhat <- npudist(bw),caller.execute=TRUE)
    npRmpi.stop()
    
    Usage 2: alternatively, compute the bandwidth object indirectly:
    
    ## Start npRmpi for interactive execution. If slaves are already running and
    ## `options(npRmpi.reuse.slaves=TRUE)` (default on some systems), this will
    ## reuse the existing pool instead of respawning. To change the number of
    ## slaves, call `npRmpi.stop(force=TRUE)` then restart.
    npRmpi.start(nslaves=1)
    mpi.bcast.Robj2slave(y)
    mpi.bcast.cmd(Fhat <- npudist(~y),caller.execute=TRUE)
    npRmpi.stop()
    
    Usage 3: modify the default kernel and order:
    
    ## Start npRmpi for interactive execution. If slaves are already running and
    ## `options(npRmpi.reuse.slaves=TRUE)` (default on some systems), this will
    ## reuse the existing pool instead of respawning. To change the number of
    ## slaves, call `npRmpi.stop(force=TRUE)` then restart.
    npRmpi.start(nslaves=1)
    mpi.bcast.Robj2slave(y)
    mpi.bcast.cmd(Fhat <- npudist(~y, ckertype="epanechnikov", ckerorder=4),
                  caller.execute=TRUE)
    npRmpi.stop()
    Usage 4: use the data frame interface rather than the formula
    interface:
    ## Start npRmpi for interactive execution. If slaves are already running and
    ## `options(npRmpi.reuse.slaves=TRUE)` (default on some systems), this will
    ## reuse the existing pool instead of respawning. To change the number of
    ## slaves, call `npRmpi.stop(force=TRUE)` then restart.
    npRmpi.start(nslaves=1)
    mpi.bcast.Robj2slave(y)
    mpi.bcast.cmd(Fhat <- npudist(tdat = y, ckertype="epanechnikov", ckerorder=4),
                  caller.execute=TRUE)
    npRmpi.stop()

npudist implements a variety of methods for estimating multivariate cumulative distributions (\(p\)-variate) defined over a set of possibly continuous and/or discrete (ordered) data. The approach is based on Li and Racine (2003) who employ ‘generalized product kernels’ that admit a mix of continuous and discrete data types.

Three classes of kernel estimators for the continuous data types are available: fixed, adaptive nearest-neighbor, and generalized nearest-neighbor. Adaptive nearest-neighbor bandwidths change with each sample realization in the set, \(x_i\), when estimating the cumulative distribution at the point \(x\). Generalized nearest-neighbor bandwidths change with the point at which the cumulative distribution is estimated, \(x\). Fixed bandwidths are constant over the support of \(x\).

Data contained in the data frame tdat (and also edat) may be a mix of continuous (default) and ordered discrete (to be specified in the data frame tdat using the ordered command). Data can be entered in an arbitrary order and data types will be detected automatically by the routine (see npRmpi for details).

A variety of kernels may be specified by the user. Kernels implemented for continuous data types include the second, fourth, sixth, and eighth-order Gaussian and Epanechnikov kernels, and the uniform kernel. Ordered data types use a variation of the Wang and van Ryzin (1981) kernel.

References

Aitchison, J. and C.G.G. Aitken (1976), “ Multivariate binary discrimination by the kernel method,” Biometrika, 63, 413-420.

Li, Q. and J.S. Racine (2007), Nonparametric Econometrics: Theory and Practice, Princeton University Press.

Li, Q. and J.S. Racine (2003), “Nonparametric estimation of distributions with categorical and continuous data,” Journal of Multivariate Analysis, 86, 266-292.

Li, C. and H. Li and J.S. Racine (2017), “Cross-Validated Mixed Datatype Bandwidth Selection for Nonparametric Cumulative Distribution/Survivor Functions,” Econometric Reviews, 36, 970-987.

Ouyang, D. and Q. Li and J.S. Racine (2006), “Cross-validation and the estimation of probability distributions with categorical data,” Journal of Nonparametric Statistics, 18, 69-100.

Pagan, A. and A. Ullah (1999), Nonparametric Econometrics, Cambridge University Press.

Scott, D.W. (1992), Multivariate Density Estimation. Theory, Practice and Visualization, New York: Wiley.

Silverman, B.W. (1986), Density Estimation, London: Chapman and Hall.

Wang, M.C. and J. van Ryzin (1981), “A class of smooth estimators for discrete distributions,” Biometrika, 68, 301-309.

Examples

Run this code

if (FALSE) {
## Not run in checks: excluded to keep MPI examples stable and check times short.
## The following example is adapted for interactive parallel execution
## in R. Here we spawn 1 slave so that there will be two compute nodes
## (master and slave).  Kindly see the batch examples in the demos
## directory (npRmpi/demos) and study them carefully. Also kindly see
## the more extensive examples in the np package itself. See the npRmpi
## vignette for further details on running parallel np programs via
## vignette("npRmpi",package="npRmpi").

## Start npRmpi for interactive execution. If slaves are already running and
## `options(npRmpi.reuse.slaves=TRUE)` (default on some systems), this will
## reuse the existing pool instead of respawning. To change the number of
## slaves, call `npRmpi.stop(force=TRUE)` then restart.
npRmpi.start(nslaves=1)

data("Italy")

mpi.bcast.Robj2slave(Italy)

mpi.bcast.cmd(bw <- npudistbw(formula=~ordered(year)+gdp,
                              data=Italy),
              caller.execute=TRUE)

mpi.bcast.cmd(F <- npudist(bws=bw),
              caller.execute=TRUE)

summary(F)

## For the interactive run only we close the slaves perhaps to proceed
## with other examples and so forth. This is redundant in batch mode.

## Note: on some systems (notably macOS+MPICH), repeatedly spawning and
## tearing down slaves in the same R session can lead to hangs/crashes.
## npRmpi may therefore keep slave daemons alive by default and
## `npRmpi.stop()` performs a "soft close". Use `force=TRUE` to
## actually shut down the slaves.
##
## You can disable reuse via `options(npRmpi.reuse.slaves=FALSE)` or by
## setting the environment variable `NP_RMPI_NO_REUSE_SLAVES=1` before
## loading the package.

npRmpi.stop()               ## soft close (may keep slaves alive)
## npRmpi.stop(force=TRUE)  ## hard close

## Note that in order to exit npRmpi properly avoid quit(), and instead
## use mpi.quit() as follows.

## mpi.bcast.cmd(mpi.quit(),
##               caller.execute=TRUE)
}

Run the code above in your browser using DataLab