calcBetaBin: Calculate reliability using a Beta-Binomial model

Description

This function estimates reliability using a Beta-Binomial model. NOTE: currently, Beta-Binomial reliability estimates do not take risk-adjustment into account.

Usage

calcBetaBin(
  df = NULL,
  model = NULL,
  entity = "entity",
  y = "y",
  df.aggregate = FALSE,
  n = "n",
  x = "x",
  show.all = FALSE,
  ctrPerf = controlPerf()
)

Value

A list containing:

alpha: estimated alpha from Beta-Binomial model
beta: estimated beta from Beta-Binomial model
entity: list of entities
n: sample sizes for each entity
var.b.BB: between-entity variance
var.w.BB: within-entity variance
est.BB: entity-level reliability estimates

If show.all is set to TRUE, then the outputted list will also contain:

var.w.FE: within-entity variance using fixed effect estimates of entity-specific outcome probabilities
var.w.RE: within-entity variance using random effect estimates of entity-specific outcome probabilities
var.w.J: within-entity variance using Bayesian estimates of entity-specific outcome probabilities, with Jeffrey's prior
est.BB.FE: entity-level reliability estimates using fixed effect estimates of entity-specific outcome probabilities
est.BB.RE: entity-level reliability estimates using random effect estimates of entity-specific outcome probabilities
est.BB.J: entity-level reliability estimates using Bayesian estimates of entity-specific outcome probabilities, with Jeffrey's prior

Arguments

df: dataframe (assumed to be observation-level unless df.aggregate is changed below); if null, will use the dataframe from the model object
model: model; if null, will use an unadjusted model (NOTE: currently, Beta-Binomial reliability estimates do not take risk-adjustment into account.)
entity: data column containing the accountable entity identifier
y: data column containing the outcome variable
df.aggregate: set this to TRUE if the data have already been aggregated to include only summary data (sample sizes and numerators) for each entity; default is FALSE.
n: if using aggregated data, data column containing the sample size by entity
x: if using aggregated data, data column containing the number of observations that met measure criteria by entity
show.all: logical parameter indicating whether all variations of reliability estimates should be calculated; default is FALSE.
ctrPerf: parameters to control performance measure calculation

Author

Kenneth Nieser (nieser@stanford.edu)

Details

To fit the Beta-Binomial model, the function first calculates method-of-moments estimates for the alpha and beta parameters which are used as starting values. Then, we use the optim() function to calculate maximum likelihood estimates with method = 'L-BFGS-B'. Reliability estimates are calculated used the maximum likelihood estimates of alpha and beta.

References

Adams JL. The Reliability of Provider Profiling: A Tutorial. 2009.

Nieser KJ, Harris AH. Comparing methods for assessing the reliability of health care quality measures. Statistics in Medicine. 2024 Oct 15;43(23):4575-94.

Zhou G, Lin Z. Improved beta-binomial estimation for reliability of healthcare quality measures. medRxiv. 2023 Jan 9:2023-01.

Examples

Run this code

# Simulate data
df <- simulateData(n.entity = 50, n.obs = 100, mu = .2, r = .7)

# Calculate reliability
out <- calcBetaBin(df = df, entity = 'entity', y = 'y')
summary(out$est.BB)

# Plot entity-level reliability by sample size
plot(out$n, out$est.BB)


## Reliability can also be calculated with data aggregated by entity
df.agg <- data.frame(
          entity = aggregate(y ~ entity, data = df, length)$entity,
          n = aggregate(y ~ entity, data = df, length)$y,
          x = aggregate(y ~ entity, data = df, sum)$y
          )

out2 <- calcBetaBin(df = df.agg, df.aggregate = TRUE, n = 'n', x = 'x')
summary(out2$est.BB)

Run the code above in your browser using DataLab