DynTxRegime (version 2.1)

iqLearnFSM: IQ-Learning: First-Stage Regression of Estimated Second-Stage Main Effects

Description

Estimate an optimal dynamic treatment regime using the Interactive Q-learning (IQ-learning) algorithm when the data has been collected from a two-stage randomized trial with binary treatments coded as {-1,1}. iqLearnFSM implements the first-stage regression of the estimated second-stage main effects (IQ2).

Usage

iqLearnFSM(..., moMain, moCont, data, response, txName,  iter = 0L, suppress = FALSE)

Arguments

...

ignored

moMain

an object of class modelObj defining the regression model and procedures to be used for the main effects component of this regression step. The object must be created by a call to modelObj::buildModelObj(). The method chosen to obtain predictions must return the prediction on the scale of the response variable. See ?modelObj for details.

moCont

an object of class modelObj defining the regression model and procedures to be used for the contrast component of this regression step. The object must be created by a call to modelObj::buildModelObj(). The method chosen to obtain predictions must return the prediction on the scale of the response variable. See ?modelObj for details.

data

an object of class data.frame containing the covariates and treatment history. Treatment must be an integer (or able to be cast to an integer) and take values {-1,1}.

response

an object is of class iqLearnSS: the value object returned by a previous call to iqLearnSS().

txName

an object of class character: the column header of the first-stage treatment variable in data.

iter

an object of class integer.

>=1 if moMain and moCont are to be fitted separately, iter is the maximum number of iterations. Assume Y = Ymain + Ycont; the iterative algorithm is as follows: (1) hat(Ycont) = 0; (2) Ymain = Y - hat(Ycont); (3) fit Ymain ~ moMain; (4) set Ycont = Y - hat(Ymain); (5) fit Ycont ~ A*moCont; (6) Repeat steps (2) - (5) until convergence or a maximum of iter iterations.

<=0 if="" the="" components="" of="" conditional="" expectation="" moMain and moCont will be combined and fit as a single object. Note that if iter <= 0,="" all="" non-model="" components="" of="" the="" moMain and moCont must be identical. By default, the choices in moMain are used.

suppress

an object of class logical. If TRUE, final screen prints are suppressed.

Value

Returns an object that inherits directly from class DynTxRegime.

Details

There are standard regression analysis tools available for the object returned by this function. In general, these tools simply extend the methods defined by the regression function. If defined, coef() returns the model parameter estimates; plot() generates the standard x-y plots; residuals returns model residuals for the combined model; and summary returns summary information.

Other tools, such as fitted() for the lm regression function, can be accessed using fitObject(). fitObject() retrieves the standard value object returned by the regression method, which can be passed as input to other functions. See ?fitObject for details.

References

Laber, E. B., Linn, K. A., and Stefanski, L.A. (2014). Interactive Q-learning. Biometrika, in press.

See Also

iqLearnSS, iqLearnFSC iqLearnFSV

Examples

  ##########################################################
  # Load and process data set
  ##########################################################
    data(bmiData)

    #----------------------------------------------------#
    # Recast treatment variables to (-1,1) --- required.
    #----------------------------------------------------#
    bmiData$A1[which (bmiData$A1=="MR")] <- 1L
    bmiData$A1[which (bmiData$A1=="CD")] <- -1L
    bmiData$A2[which (bmiData$A2=="MR")] <- 1L
    bmiData$A2[which (bmiData$A2=="CD")] <- -1L
    bmiData$A1 <- as.integer(bmiData$A1)
    bmiData$A2 <- as.integer(bmiData$A2)

    #----------------------------------------------------#
    # define response y to be the negative 12 month
    # change in BMI from baseline
    #----------------------------------------------------#
    bmiData$y <- -100*(bmiData[,6] - bmiData[,4])/bmiData[,4]

  ##########################################################
  # Second-stage regression
  ##########################################################
    #----------------------------------------------------#
    # Create modelObj object for main effect component
    #----------------------------------------------------#
    moMain <- buildModelObj(model = ~ gender + parentBMI + month4BMI,
                            solver.method = 'lm')

    #----------------------------------------------------#
    # Create modelObj object for contrast component
    #----------------------------------------------------#
    moCont <- buildModelObj(model = ~ parentBMI + month4BMI,
                            solver.method = 'lm')

    iqSS <- iqLearnSS(moMain = moMain, 
                      moCont = moCont, 
                      data = bmiData,
                      response = bmiData$y, 
                      txName = "A2", 
                      iter = 0)

  ##########################################################
  # First-stage regression - Single Regression Step
  ##########################################################
    #----------------------------------------------------#
    # Create modelObj object for main effect component
    #----------------------------------------------------#
    moMain <- buildModelObj(model = ~ gender + race + parentBMI + baselineBMI,
                            solver.method = 'lm')

    #----------------------------------------------------#
    # Create modelObj object for contrast component
    #----------------------------------------------------#
    moCont <- buildModelObj(model = ~ parentBMI + month4BMI,
                            solver.method = 'lm')

    iqFSM <- iqLearnFSM(moMain = moMain, 
                        moCont = moCont, 
                        data = bmiData,
                        response = iqSS, 
                        txName = "A1", 
                        iter = 0)

    # Estimated Q-functions
    vals <- qFuncs(iqFSM)
    head(vals)

    # Residuals
    res <- residuals(iqFSM)
    head(res)

    # Model parameter estimates
    coef(iqFSM)

    # Summary information for regression analysis
    summary(iqFSM)

    # Standard lm plots
    plot(iqFSM)

    # Value object returned by lm
    fitObj <- fitObject(iqFSM)
    fitObj

    # All standard lm  methods can be applied to the elements of this list.
    summary(fitObj[[ "Combined" ]])
    coef(fitObj[[ "Combined" ]])
    head(residuals(fitObj[[ "Combined" ]]))
    head(fitted.values(fitObj[[ "Combined" ]]))
    plot(fitObj[[ "Combined"]])


  ##########################################################
  # First-stage regression - Iterative Regression Step
  ##########################################################
    iqFSM <- iqLearnFSM(moMain = moMain, 
                        moCont = moCont, 
                        data = bmiData,
                        response = iqSS, 
                        txName = "A1", 
                        iter = 100)

    # Estimated Q-functions
    vals <- qFuncs(iqFSM)
    head(vals)

    # Residuals
    res <- residuals(iqFSM)
    head(res)

    # Model parameter estimates
    coef(iqFSM)

    # Summary information for regression analysis
    summary(iqFSM)

    # Standard lm plots
    plot(iqFSM)

    # Value object returned by lm
    fitObj <- fitObject(iqFSM)
    fitObj

    # All standard lm  methods can be applied to the elements of this list.
    summary(fitObj[[ "MainEffect" ]])
    coef(fitObj[[ "MainEffect" ]])
    head(residuals(fitObj[[ "MainEffect" ]]))
    head(fitted.values(fitObj[[ "MainEffect" ]]))
    plot(fitObj[[ "MainEffect"]])

    summary(fitObj[[ "Contrast" ]])
    coef(fitObj[[ "Contrast" ]])
    head(residuals(fitObj[[ "Contrast" ]]))
    head(fitted.values(fitObj[[ "Contrast" ]]))
    plot(fitObj[[ "Contrast"]])