biglasso (version 1.3-3)

biglasso-package: Extending Lasso Model Fitting to Big Data

Description

Extend lasso and elastic-net linear and logistic regression models for ultrahigh-dimensional, multi-gigabyte data sets that cannot be loaded into available RAM. This package utilizes memory-mapped files to store the massive data on the disk and only read those into memory whenever necessary during model fitting. Moreover, some advanced feature screening rules are proposed and implemented to accelerate the model fitting. As a result, this package is much more memory- and computation-efficient and highly scalable as compared to existing lasso-fitting packages such as glmnet and ncvreg, thus allowing for powerful big data analysis even with only an ordinary laptop.

Details

Package:

biglasso

Type:

Package

Version:

1.3-3

Date:

2017-01-24

License:

GPL-3

Penalized regression models, in particular the lasso, have been extensively applied to analyzing high-dimensional data sets. However, due to the memory limit, existing R packages are not capable of fitting lasso models for ultrahigh-dimensional, multi-gigabyte data sets which have been increasingly seen in many areas such as genetics, biomedical imaging, genome sequencing and high-frequency finance.

This package aims to fill the gap by extending lasso model fitting to Big Data in R. Version >= 1.2-3 represents a major redesign where the source code is converted into C++ (previously in C), and new feature screening rules, as well as OpenMP parallel computing, are implemented. Some key features of biglasso are summarized as below:

1. it utilizes memory-mapped files to store the massive data on the disk, only loading data into memory when necessary during model fitting. Consequently, it's able to seamlessly data-larger-than-RAM cases.
2. it is built upon pathwise coordinate descent algorithm with warm start, active set cycling, and feature screening strategies, which has been proven to be one of fastest lasso solvers.
3. in incorporates our newly developed hybrid hybrid safe-strong rules that outperform state-of-the-art screening rules such as the sequential strong rule (SSR) and the sequential EDPP rule (SEDPP) with additional 1.5x to 4x speedup.
4. the implementation is designed to be as memory-efficient as possible by eliminating extra copies of the data created by other R packages, making it at least 2x more memory-efficient than glmnet.
5. the underlying computation is implemented in C++, and parallel computing with OpenMP is also supported.

For more information:

• Benchmarking results: https://github.com/YaohuiZeng/biglasso.
• Tutorial: https://github.com/YaohuiZeng/biglasso/blob/master/vignettes/biglasso.pdf
• Technical paper: https://arxiv.org/abs/1701.05936

References

• Zeng, Y., and Breheny, P. (2016). The biglasso Package: A Memory- and Computation-Efficient Solver for Lasso Model Fitting with Big Data in R. https://arxiv.org/abs/1701.05936.
• Tibshirani, R., Bien, J., Friedman, J., Hastie, T., Simon, N., Taylor, J., and Tibshirani, R. J. (2012). Strong rules for discarding predictors in lasso-type problems. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 74(2), 245-266.
• Wang, J., Zhou, J., Wonka, P., and Ye, J. (2013). Lasso screening rules via dual polytope projection. In Advances in Neural Information Processing Systems, pp. 1070-1078.
• Xiang, Z. J., and Ramadge, P. J. (2012). Fast lasso screening tests based on correlations. In Acoustics, Speech and Signal Processing (ICASSP), 2012 IEEE International Conference on (pp. 2137-2140). IEEE.
• Wang, J., Zhou, J., Liu, J., Wonka, P., and Ye, J. (2014). A safe screening rule for sparse logistic regression. In Advances in Neural Information Processing Systems, pp. 1053-1061.

Examples

## Not run: 
# ## Example of reading data from external big data file, fit lasso model, 
# ## and run cross validation in parallel
# 
# # simulated design matrix, 1000 observations, 500,000 variables, ~ 5GB
# # there are 10 true variables with non-zero coefficient 2.
# xfname <- 'x_e3_5e5.txt' 
# yfname <- 'y_e3_5e5.txt' # response vector
# time <- system.time(
#   X <- setupX(xfname, sep = '\t') # create backing files (.bin, .desc)
# )
# print(time) # ~ 7 minutes; this is just one-time operation
# dim(X)
# 
# # the big.matrix then can be retrieved by its descriptor file (.desc) in any new R session. 
# rm(X)
# xdesc <- 'x_e3_5e5.desc' 
# X <- attach.big.matrix(xdesc)
# dim(X)
# 
# y <- as.matrix(read.table(yfname, header = F))
# time.fit <- system.time(
#   fit <- biglasso(X, y, family = 'gaussian', screen = 'SSR-BEDPP')
# )
# print(time.fit) # ~ 44 seconds for fitting a lasso model along the entire solution path
# 
# # cross validation in parallel
# seed <- 1234
# time.cvfit <- system.time(
#   cvfit <- cv.biglasso(X, y, family = 'gaussian', screen = 'SSR-BEDPP', 
#                        seed = seed, ncores = 4, nfolds = 10)
# )
# print(time.cvfit) # ~ 3 minutes for 10-fold cross validation
# plot(cvfit)
# summary(cvfit)
# 
# ## End(Not run)