KernelICA (version 0.1.0)

kernel_ica: Kernel Independent Component Analysis

Description

The kernel ICA method by Bach and Jordan (see references). The contrast function was written in C++ using the Eigen3 library for computational speed. The package ManifoldOptim is utilized for minimization of the contrast function on the Stiefel manifold.

Usage

kernel_ica(
  x,
  variant = c("kgv", "kcca"),
  kernel = c("gauss", "hermite"),
  nstarts = 1,
  eps = 1e-04,
  sigma = ifelse(ncol(x) < 1000, 1, 0.5),
  kappa = ifelse(ncol(x) < 1000, 0.02, 0.002),
  hermite_rank = 3,
  init = MD_distant_matrices(p = ncol(x), n = nstarts),
  solver_params = ManifoldOptim::get.solver.params(),
  optim_method = "RSD"
)

Arguments

x

A numeric matrix, where each column contains the measurements of a mixed data source.

variant

Either "kcca" or "kgv".

kernel

Either "gauss" or "hermite".

nstarts

The number of restarts of the kernel ICA method with a default value of one. Ignored, if the starting values in parameter init are set manually.

eps

Numeric precision parameter for the approximation of the kernel matrices.

sigma

Numeric value of the kernel variance. Default value is 1 for a given x with less than 1000 rows, otherwise 0.5.

kappa

Numeric dimming parameter. Default value is 2e^-2 for a given x with less than 1000 rows, otherwise 2e^-3.

hermite_rank

Integer. Rank of the hermite polynomial with a default value of 3. Ignored, when kernel was set to "gauss".

init

A list of \(p \times p\) orthogonal matrices, which are the starting points for the optimization in the Stiefel manifold. By default a number of orthogonal matrices specified in parameter nstarts is generated.

solver_params

An object returned from the method ManifoldOptim::get.solver.params which can be given several parameters for the optimization.

optim_method

The optimization method used in the Stiefel manifold. Default value is "RSG". This value is directly passed to ManifoldOptim::manifold.optim.

Value

A class of type bss containing the following values:

Xmu

The mean values

S

The unmixed data

W

The unmixing matrix

cmin

The smallest resulting contrast function value of all kernel ICA runs

Details

Several points need to be considered when using kernel_ica:

• To comply with the notions of the JADE package, model \(\boldsymbol{X} = \boldsymbol{S} \boldsymbol{A}'\) with a \(n \times p\) source matrix \(\boldsymbol{S}\) and a \(p \times p\) mixing matrix \(\boldsymbol{A}\) is assumed.

• The returned unmixing matrix \(\boldsymbol{W}\) is found so that \(\boldsymbol{X} \boldsymbol{W}' = \boldsymbol{S} \boldsymbol{A}' \boldsymbol{W}'\) results in the desired independent data.

• It is not possible to reconstruct the original order of the sources nor their sign.

• The contrast function which is to be minimized can have several local optimal. Therefore setting the nstart parameter to a larger value than one or instead providing more matrices in init for several starts should be considered.

• Kernel ICA is started for each element given in the list init separately and returns the best result by the lowest resulting value of the contrast function.

References

Kernel ICA implementation in Matlab and C by F. Bach: https://www.di.ens.fr/~fbach/kernel-ica/index.htm

Francis R. Bach, Michael I. Jordan Kernel independent component analysis Journal of Machine Learning Research 2002 10.1162/153244303768966085

Sean Martin, Andrew M. Raim, Wen Huang, Kofi P. Adragni ManifoldOptim: An R Interface to the ROPTLIB Library for Riemannian Manifold Optimization Journal of Statistical Software 2020 10.18637/jss.v093.i01

See Also

manifold.optim get.solver.params

Examples

# NOT RUN {
require(JADE)
require(ICtest)

n <- 2000
p <- 3
S <- matrix(0, n, p)

# the three data sources used in this example
S[, 1] <- rexp(n, rate = 0.4)
S[, 2] <- runif(n, 2, 4)
S[, 3] <- rt(n, 5)

W <- ICtest::rorth(p) # creates an orthogonal matrix
y <- S %*% t(W) # mixes the data

# applying kernel ICA method
res <- KernelICA::kernel_ica(y, variant = "kgv", kernel = "hermite")

res$W # unmixing matrix
apply(S, 2, mean) # original means
res$Xmu # restored means (unordered and possibly with different sign each)

# restored data
z <- scale(res$S, center = -res$Xmu, scale = FALSE)
# MD distance of the returned matrix to the original mixing matrix.
JADE::MD(res$W, W)

# }
# NOT RUN {
# Runs kernel ICA with the slower Gaussian kernel method and
# a the starting matrix returned from the first method call.
# The maximal iteration number in the optimization is reduced to a tenth.
res2 <- KernelICA::kernel_ica(
  y,
  variant = "kgv",
  kernel = "gauss",
  init = list(res$W),
  solver_params = ManifoldOptim::get.solver.params(Max_Iteration = 100)
)
JADE::MD(res2$W, W)
# }