VIM (version 7.3.0)

imputeCellIRMI: Cellwise-robust iterative regression imputation for mixed data

Description

Extends IRMI (Templ, Kowarik, and Filzmoser, 2011) with cellwise contamination handling. Each conditional regression uses a cell-weighted IRWLS engine where per-cell weights in the design matrix downweight contaminated cells without discarding entire observations.

Usage

imputeCellIRMI(
  data,
  method = "tukey",
  alpha = NULL,
  maxit = 100,
  maxit_irwls = 50,
  eps = 0.005,
  eps_irwls = 1e-06,
  uncert = "pmm",
  weight_update = "multivariate",
  init_weights = "ddc",
  hard_threshold = 0.5,
  trace = FALSE
)

Value

A list with components:

data_imputed

the imputed data.frame.

cellweights

\(n \times p\) matrix of final cell weights (1 = clean, 0 = fully downweighted). Categorical columns always have weight 1.

converged

logical indicating whether the outer loop converged.

iterations

number of outer iterations used.

Arguments

data

a data.frame with missing values (mixed continuous and categorical variables are supported).

method

weight function: "tukey" (default, Tukey bisquare) or "huber" (Huber).

alpha

tuning constant. NULL (default) uses 1.345 for Huber and 4.685 for Tukey, giving 95% efficiency at the normal model.

maxit

maximum number of outer IRMI iterations (default: 100).

maxit_irwls

maximum number of inner IRWLS iterations per regression (default: 50).

eps

convergence tolerance for the outer loop (default: 5e-3). Convergence is declared when the relative change in imputed values falls below this threshold.

eps_irwls

convergence tolerance for the inner IRWLS (default: 1e-6).

uncert

imputation uncertainty method: "pmm" (predictive mean matching, default), "normalerror" (add normal noise), or "resid" (bootstrap residual).

weight_update

strategy for updating cell weights between outer iterations: "multivariate" (default) uses an MCD-based multivariate update for weight coherence across variables, or "univariate" updates each variable independently from its residuals.

init_weights

method for initialising cell weights, one of "ddc" (default; DetectDeviatingCells, requires the cellWise package and falls back to univariate weights when it is unavailable), "univariate" (per-column median/MAD standardisation), or "mcd" (minimum covariance determinant on the continuous block). The default is "ddc" because "mcd" downweights high-leverage points that carry the regression signal, which can make imputation worse than unconditional median imputation.

hard_threshold

numeric in \([0, 1]\). After convergence, cells with weight below this value are flagged as contaminated (default: 0.5).

trace

logical; if TRUE, print progress information.

Author

Matthias Templ

Details

The algorithm works iteratively: in each outer iteration, every variable with missing values is used as response in a conditional regression on all remaining variables. For continuous responses, the custom cellIRWLS() engine fits a weighted regression where each cell in the design matrix receives its own weight reflecting potential cellwise contamination. For categorical responses, a weighted multinomial model is used. After each regression, cell weights for the response variable are updated from the residuals.

The algorithm proceeds as follows:

1. Missing values are initialised using initialise.

2. Initial cell weights are computed with cellWeights() on all continuous variables in the initialised data.

3. Outer loop (up to maxit iterations):

   • For each variable \(j\) with missing values:

     • Form predictor matrix \(X\) (all other variables) and response \(y\) (variable \(j\)).

     • If \(j\) is continuous: fit cellIRWLS(X, y, w_cell, w_response) and impute missing values in \(j\) using the fitted model plus uncertainty.

     • If \(j\) is categorical: fit nnet::multinom() with row weights derived from the cell weight matrix and impute by sampling from predicted probabilities.

     • Update cell weights for \(j\) from residuals via cellWeightsFromResiduals().

   • Check convergence: relative change in imputed values falls below eps.

References

Templ, M., Kowarik, A. and Filzmoser, P. (2011). Iterative stepwise regression imputation using standard and robust methods. Computational Statistics & Data Analysis, 55(10), 2793--2806.

See Also

imputeCellM, imputeCellEM, initialise, irmi

Other imputation methods: hotdeck(), impPCA(), imputeCellEM(), imputeCellM(), imputeCellMCD(), imputeCellwise(), imputeRobust(), imputeRobustChain(), irmi(), kNN(), matchImpute(), medianSamp(), rangerImpute(), regressionImp(), sampleCat(), vimmi, vimpute(), xgboostImpute()

Examples

# \donttest{
data(sleep, package = "VIM")
result <- imputeCellIRMI(sleep)
head(result$data_imputed)
image(result$cellweights, main = "Cell weights")

# With Huber weights (less aggressive downweighting)
result2 <- imputeCellIRMI(sleep, method = "huber", trace = TRUE)

# Mixed data example
data(testdata)
result3 <- imputeCellIRMI(testdata$wna)
# }