BigDataStatMeth (version 1.0.3)

bdSolve_hdf5: Solve Linear System AX = B (HDF5-Stored)

Description

Solves the linear system AX = B where matrices A and B are stored in HDF5 format. The solution X is written back to the HDF5 file.

Usage

bdSolve_hdf5(
  filename,
  groupA,
  datasetA,
  groupB,
  datasetB,
  outgroup = NULL,
  outdataset = NULL,
  overwrite = NULL
)

Value

List with components. If an error occurs, all string values are returned as empty strings (""):

fn

Character string with the HDF5 filename

ds

Character string with the full dataset path to the solution of the linear system (group/dataset)

Arguments

filename

String. Path to the HDF5 file.

groupA

String. Group containing matrix A.

datasetA

String. Dataset name for matrix A.

groupB

String. Group containing matrix B.

datasetB

String. Dataset name for matrix B.

outgroup

Optional string. Output group name (defaults to "Solved").

outdataset

Optional string. Output dataset name (defaults to "A_B").

overwrite

Logical. Whether to overwrite existing results.

Details

This function provides an HDF5-based implementation for solving large linear systems. Key features:

• HDF5 Integration:

  • Efficient reading of input matrices

  • Memory-efficient processing

  • Direct output to HDF5 format

• Implementation Features:

  • Automatic solver selection

  • Support for large matrices

  • Flexible output options

  • Memory-efficient processing

The function handles:

• Data validation

• Memory management

• Error handling

• HDF5 file operations

References

• Anderson, E. et al. (1999). LAPACK Users' Guide, 3rd Edition. SIAM, Philadelphia.

• The HDF Group. (2000-2010). HDF5 User's Guide.

See Also

• bdSolve for in-memory matrix solving

• bdCreate_hdf5_matrix for creating HDF5 matrices

Examples

library(BigDataStatMeth)

# Create test matrices
N <- 1000
M <- 1000
fn <- "test_temp.hdf5"

set.seed(555)
Y <- matrix(rnorm(N*M), N, M)
X <- matrix(rnorm(N), N, 1)
Ycp <- crossprod(Y)

# Compare with in-memory solution
resm <- bdSolve(Ycp, X)
resr <- solve(Ycp, X)
all.equal(resm, resr)

# Save matrices to HDF5
bdCreate_hdf5_matrix(filename = fn,
                     object = Ycp,
                     group = "data",
                     dataset = "A",
                     transp = FALSE,
                     overwriteFile = TRUE,
                     overwriteDataset = TRUE,
                     unlimited = FALSE)

bdCreate_hdf5_matrix(filename = fn,
                     object = X,
                     group = "data",
                     dataset = "B",
                     transp = FALSE,
                     overwriteFile = FALSE,
                     overwriteDataset = TRUE,
                     unlimited = FALSE)

# Solve using HDF5-stored matrices
bdSolve_hdf5(filename = fn,
             groupA = "data",
             datasetA = "A",
             groupB = "data",
             datasetB = "B",
             outgroup = "Solved",
             outdataset = "A_B",
             overwrite = TRUE)

# Cleanup
if (file.exists(fn)) {
    file.remove(fn)
}