Fits the latent class stochastic frontier model of Greene (2005) and Orea and Kumbhakar (2004), in which several unobserved technologies coexist in one sample and each firm contributes to every one of them, weighted by its class probability.
lcsfm(formula, model_name = c("LCM", "LCM_Z", "LCM_CN"),
data, n_class = 2, maxit.bobyqa = 10000, maxit.psoptim = 1000,
maxit.optim = 1000, REPORT = 1, trace = 0, pgtol = 0,
start_val = FALSE, PSopt = FALSE, optHessian = TRUE,
inefdec = TRUE, penalty_c = 0, upper = NA, Method = "L-BFGS-B",
verbose = FALSE, rand.psoptim = NULL)An object of class "sfareg" containing the following components:
A matrix with parameter estimates, standard errors, and t-values. One row per parameter, named <par>_class<j> for the class blocks and logit_<var>_class<j> for the class-probability coefficients.
A list containing the optimization results from the final optimization procedure.
The total computation time for model estimation.
The starting values used in the optimization.
The name of the model estimated ("LCM" or "LCM_Z").
The formula used in the model specification.
Posterior-weighted Jondrow et al. (1982) inefficiency predictions.
An \(n \times J\) matrix of posterior class probabilities, whose rows sum to one.
An \(n \times J\) matrix of class-conditional JLMS predictions.
Integer vector giving each observation's modal class.
Average prior class probabilities, one per class.
The number of classes fitted.
A vector of estimated parameters.
A vector of standard errors for the estimated parameters (NA if optHessian = FALSE).
A vector of t-values for the estimated parameters (NA if optHessian = FALSE).
The matched call.
a symbolic description for the model to be estimated. For "LCM_Z" the second part gives the covariates driving class membership (y ~ x | z).
Which specification to fit. "LCM" holds the class probabilities constant across the sample and lets every parameter vary by class; "LCM_Z" is "LCM" with the class probabilities depending on covariates supplied in the second part of the formula. "LCM_CN" is the contaminated normal frontier, in which only the noise scale varies between components and the frontier, the inefficiency scale and the class probabilities are all common -- see ‘Details’. Matching ignores case, so "lcm", "LCM" and "Lcm" are the same choice.
A data frame containing the variables named in formula.
Integer. The number of latent classes \(J\), at least 2. Defaults to 2. The parameter count grows as \(J(k + 2)\) plus \((J-1)q\), so large \(J\) needs a correspondingly large sample; see ‘Details’.
Maximum number of iterations for the bobyqa optimization routine
Maximum number of iterations for the psoptim optimization routine
Maximum number of iterations for the optim optimization routine
reporting parameter
Integer. Tracing level passed through to the optimizer; larger values print more. 0 (the default) is silent.
Numeric. Projected-gradient tolerance passed to optim()'s "L-BFGS-B" method. 0 uses the optimizer's own default.
starting value (optional)
use psoptim optimization routine (T or F)
Logical. Should a numerically differentiated Hessian matrix be returned while using the optim routine? (for optim routine)
Production or cost function
Tuning constant for Chen et al.'s (2001) penalty \(c\log(J^J\prod_j p_j)\), which is added to the log-likelihood so that no class probability can be driven to 0 or 1. The default 0 leaves the ordinary likelihood untouched. A positive value is what lcsfm_homogeneity needs, and it changes what is maximised: $opt$value then holds the penalised objective, while $logLik_unpenalised and $penalty carry the two pieces separately. Only available for model_name = "LCM", since the penalty is defined on a scalar mixing proportion.
Vector of upper values for the optim package.
The method to be used for optim. See 'Details' within optim.
Logical. Print optimization progress messages? Default is FALSE.
Integer. seed for replication of psoptim. Default to NULL.
Christopher F. Parmeter and David H. Bernstein
"LCM_CN", the contaminated normal frontier. A different use of the same machinery: instead of separating technologies, it gives the noise heavier tails. The noise density is a scale mixture of normals, \(f_v(v) = \sum_j p_j \phi(v/\sigma_{vj})/\sigma_{vj}\), while the frontier, \(\sigma_u\) and the mixing proportions stay common across components. At \(J = 2\) this is the scale-contaminated normal, an alternative to a Student-\(t\) noise term.
It has a closed form, which is why it is cheap. Since \(f_\epsilon(e) = \int f_v(e + Su) f_u(u)\,du\) is linear in \(f_v\), a mixture noise density passes straight through:
$$f_\epsilon(e) = \sum_j p_j\, f_{NHN}(e; \sigma_{vj}, \sigma_u)$$
a mixture of ordinary normal/half-normal densities sharing one \(\sigma_u\). No extra integration is required, and the identity is verified against direct numerical integration in the tests.
The parameter vector is \((\sigma_{v1},\dots,\sigma_{vJ}, \sigma_u, \beta, \mathrm{logit})\) rather than one block per class.
It is also the specification lcsfm_homogeneity'''s asymptotic null is stated for: one scalar parameter differing between components. On "LCM", where everything varies, the \(\chi^2_{0:1}\) null rejected 63.5% of the time at a nominal 5% under a true null; on "LCM_CN" the same null gives 8.0%. Still mildly liberal, and the test says so, but usable.
The latent class stochastic frontier supposes that \(J\) distinct technologies are present in one sample and that which technology a firm operates is unobserved. Each class \(j\) has its own frontier and its own two scales,
$$y_i = x_i'\beta_j + v_{ij} - u_{ij}, \quad v_{ij} \sim N(0, \sigma_{vj}^2), \quad u_{ij} \sim N^+(0, \sigma_{uj}^2),$$
and class membership follows a multinomial logit,
$$P(j \mid q_i) = \exp(q_i'\delta_j) / \sum_m \exp(q_i'\delta_m), \quad \delta_J = 0,$$
with class \(J\) the reference. Every firm contributes to every class, weighted by its class probability, so the log-likelihood mixes the \(J\) composed densities rather than assigning firms to groups:
$$\log L_i = \log \sum_j P(j \mid q_i) f_j(\varepsilon_{ij}).$$
Under "LCM" the \(q_i\) are a constant, so the class probabilities are the same for every firm; under "LCM_Z" they are the covariates in the second part of the formula.
This is a different object from fitting separate frontiers to known subgroups: the groups are not known, and the uncertainty about them is carried through into the efficiency predictions. It is also more general than zsfm's zero-inefficiency model, which is the restricted two-class case in which one class has no inefficiency at all.
post.prob is an \(n \times J\) matrix of posterior class probabilities, jlms_class holds the class-conditional Jondrow et al. (1982) predictions, and jlms is their posterior-weighted average -- a firm's inefficiency is only defined relative to a frontier, and which frontier it faces is itself uncertain, so reporting the modal class alone would discard that. class gives the modal class and class_prob the average prior class shares.
Label switching. Class labels in a finite mixture are identified only up to permutation: relabelling the classes and permuting \(\delta\) gives the identical likelihood. Starting values are built by splitting the OLS residuals at their \(J\)-quantiles and refitting within each group, which makes a run reproducible on the same data -- and avoids the saddle point that identical starting components would sit at, since equal posterior class probabilities give a zero score with respect to the class split -- but it does not pin the labels to any external ordering. Do not compare class 1 across two fits, or against a simulation's true labels, without matching the classes first.
Choosing \(J\). The likelihood cannot fall when a class is added, and the usual regularity conditions for a likelihood-ratio test of \(J\) against \(J+1\) do not hold, since the null puts a class probability on the boundary. Use AIC()/BIC(), the separation of the fitted frontiers, and whether the extra class is economically interpretable, rather than a p-value.
Greene, W. (2005) 'Reconsidering heterogeneity in panel data estimators of the stochastic frontier model', Journal of Econometrics, 126(2), pp. 269-303. doi:10.1016/j.jeconom.2004.05.003.
Orea, L. and Kumbhakar, S.C. (2004) 'Efficiency measurement using a latent class stochastic frontier model', Empirical Economics, 29(1), pp. 169-183. doi:10.1007/s00181-003-0184-2.
Caudill, S.B. (2003) 'Estimating a mixture of stochastic frontier regression models via the em algorithm: a multiproduct cost function application', Empirical Economics, 28(3), pp. 581-598. doi:10.1007/s001810200142.
Jondrow, J., Lovell, C.A.K., Materov, I.S. and Schmidt, P. (1982) 'On the estimation of technical inefficiency in the stochastic frontier production function model', Journal of Econometrics, 19(2-3), pp. 233-238. doi:10.1016/0304-4076(82)90004-5.
lcsfm_homogeneity, zsfm, sfm, psfm, ttsfm, sfareg-methods
# \donttest{
library(sfa)
## Two coexisting technologies, class probabilities constant.
set.seed(42)
n <- 400
x1 <- rnorm(n); x2 <- rnorm(n)
cls <- rbinom(n, 1, 0.4) + 1L
b <- rbind(c(1, 0.5, 0.5), c(3, 1.0, 0.2))
sv <- c(0.2, 0.2); su <- c(0.5, 1.0)
y <- b[cls, 1] + b[cls, 2] * x1 + b[cls, 3] * x2 +
rnorm(n, 0, sv[cls]) - abs(rnorm(n, 0, su[cls]))
fit <- lcsfm(y ~ x1 + x2, model_name = "LCM",
data = data.frame(y, x1, x2), n_class = 2)
fit$class_prob # average class shares
head(fit$post.prob) # posterior class membership, one row per firm
# }
Run the code above in your browser using DataLab