Give the ensemble projection matrix and weights of the kernels in the library using stacking.
ensemble_stack(beta_exp, error_mat, A_hat)(numeric/character) A numeric value specifying the parameter
when strategy = "exp" ensemble_exp.
(matrix, n*K) A n\*K matrix indicating errors.
(list of length K) A list of projection matrices to kernel space for each kernel in the kernel library.
(matrix, n*n) The ensemble projection matrix.
(vector of length K) A vector of weights of the kernels in the library.
Empirical Risk Minimization (Stacking)
After obtaining the estimated errors \(\{\hat{\epsilon}_d\}_{d=1}^D\), we estimate the ensemble weights \(u=\{u_d\}_{d=1}^D\) such that it minimizes the overall error $$\hat{u}={argmin}_{u \in \Delta}\parallel \sum_{d=1}^Du_d\hat{\epsilon}_d\parallel^2 \quad where\; \Delta=\{u | u \geq 0, \parallel u \parallel_1=1\}$$ Then produce the final ensemble prediction: $$\hat{h}=\sum_{d=1}^D \hat{u}_d h_d=\sum_{d=1}^D \hat{u}_d A_{d,\hat{\lambda}_d}y=\hat{A}y$$ where \(\hat{A}=\sum_{d=1}^D \hat{u}_d A_{d,\hat{\lambda}_d}\) is the ensemble matrix.
Jeremiah Zhe Liu and Brent Coull. Robust Hypothesis Test for Nonlinear Effect with Gaussian Processes. October 2017.
Xiang Zhan, Anna Plantinga, Ni Zhao, and Michael C. Wu. A fast small-sample kernel inde- pendence test for microbiome community-level association analysis. December 2017.
Arnak S. Dalalyan and Alexandre B. Tsybakov. Aggregation by Exponential Weighting and Sharp Oracle Inequalities. In Learning Theory, Lecture Notes in Computer Science, pages 97<U+2013> 111. Springer, Berlin, Heidelberg, June 2007.
mode: tuning