gsl_nls_loss: Robust loss functions with tunable parameters

Description

Allow the user to tune the coefficient(s) of the robust loss functions supported by gsl_nls. For all choices other than rho = "default", the MM-estimation problem is optimized by means of iterative reweighted least squares (IRLS).

Usage

gsl_nls_loss(
  rho = c("default", "huber", "barron", "bisquare", "welsh", "optimal", "hampel", "ggw",
    "lqq"),
  cc = NULL
)

Value

A list with two components:

with meanings as explained under ‘Arguments’.

Arguments

rho: character loss function, one of "default", "huber", "barron", "bisquare", "welsh", "optimal", "hampel", "ggw", "lqq".
cc: named or unnamed numeric vector of tuning parameters. The length of this argument depends on the selected rho function (see ‘Details’). If NULL, the default tuning parameters are returned.

Loss function details

default: Default squared loss, no iterative reweighted least squares (IRLS) is required in this case. $$\rho(x) = x^2$$

huber

Huber loss function with scaling constant $k$, defaulting to $k = 1.345$ for 95% efficiency of the regression estimator. $$\rho(x, k) = \left\{ \begin{array}{ll} \frac{1}{2} x^2 &\quad \text{ if } |x| \leq k \\[2pt] k(|x| - \frac{k}{2}) &\quad \text{ if } |x| > k \end{array} \right. $$

barron

Barron's smooth family of loss functions with robustness parameter $\alpha \leq 2$ (default $\alpha = 1$) and scaling constant $k$ (default $k = 1.345$). Special cases include: (scaled) squared loss for $\alpha = 2$; L1-L2 loss for $\alpha = 1$; Cauchy loss for $\alpha = 0$; Geman-McClure loss for $\alpha = -2$; and Welsch/Leclerc loss for $\alpha = -\infty$. See Barron (2019) for additional details. $$\rho(x, \alpha, k) = \left\{ \begin{array}{ll} \frac{1}{2} (x / k)^2 &\quad \text{ if } \alpha = 2 \\[2pt] \log\left(\frac{1}{2} (x / k)^2 + 1 \right) &\quad \text{ if } \alpha = 0 \\[2pt] 1 - \exp\left( -\frac{1}{2} (x / k)^2 \right) &\quad \text{ if } \alpha = -\infty \\[2pt] \frac{|\alpha - 2|}{\alpha} \left( \left( \frac{(x / k)^2}{|\alpha-2|} + 1 \right)^{\alpha / 2} - 1 \right) &\quad \text{ otherwise } \end{array} \right. $$

bisquare

Tukey's biweight/bisquare loss function with scaling constant $k$, defaulting to $k = 4.685$ for 95% efficiency of the regression estimator, ($k = 1.548$ gives a breakdown point of 0.5 of the S-estimator). $$\rho(x, k) = \left\{ \begin{array}{ll} 1 - (1 - (x / k)^2)^3 &\quad \text{ if } |x| \leq k \\[2pt] 1 &\quad \text{ if } |x| > k \end{array} \right. $$

welsh

Welsh/Leclerc loss function with scaling constant $k$, defaulting to $k = 2.11$ for 95% efficiency of the regression estimator, ($k = 0.577$ gives a breakdown point of 0.5 of the S-estimator). This is equivalent to the Barron loss function with robustness parameter $\alpha = -\infty$. $$\rho(x, k) = 1 - \exp\left( -\frac{1}{2} (x / k)^2 \right) $$

optimal

Optimal loss function as in Section 5.9.1 of Maronna et al. (2006) with scaling constant $k$, defaulting to $k = 1.060$ for 95% efficiency of the regression estimator, ($k = 0.405$ gives a breakdown point of 0.5 of the S-estimator). $$\rho(x, k) = \int_0^x \text{sign}(u) \left( - \dfrac{\phi'(|u|) + k}{\phi(|u|)} \right)^+ du$$ with $\phi$ the standard normal density.

hampel

Hampel loss function with a single scaling constant $k$, setting $a = 1.5k$, $b = 3.5k$ and $r = 8k$. $k = 0.902$ by default, resulting in 95% efficiency of the regression estimator, ($k = 0.212$ gives a breakdown point of 0.5 of the S-estimator). $$\rho(x, a, b, r) = \left\{ \begin{array}{ll} \frac{1}{2} x^2 / C &\quad \text{ if } |x| \leq a \\[2pt] \left( \frac{1}{2}a^2 + a(|x|-a)\right) / C &\quad \text{ if } a < |x| \leq b \\[2pt] \frac{a}{2}\left( 2b - a + (|x| - b) \left(1 + \frac{r - |x|}{r-b}\right) \right) / C &\quad \text{ if } b < |x| \leq r \\[2pt] 1 &\quad \text{ if } r < |x| \end{array} \right. $$ with $C = \rho(\infty) = \frac{a}{2} (b - a + r)$.

ggw

Generalized Gauss-Weight loss function, a generalization of the Welsh/Leclerc loss with tuning constants $a, b, c$, defaulting to $a = 1.387$, $b = 1.5$, $c = 1.063$ for 95% efficiency of the regression estimator, ($a = 0.204, b = 1.5, c = 0.296$ gives a breakdown point of 0.5 of the S-estimator). $$\rho(x, a, b, c) = \int_0^x \psi(u, a, b, c) du$$ with, $$\psi(x, a, b, c) = \left\{ \begin{array}{ll} x &\quad \text{ if } |x| \leq c \\[2pt] \exp\left( -\frac{1}{2} \frac{(|x| - c)^b}{a} \right) x &\quad \text{ if } |x| > c \end{array} \right. $$

lqq

Linear Quadratic Quadratic loss function with tuning constants $b, c, s$, defaulting to $b = 1.473$, $c = 0.982$, $s = 1.5$ for 95% efficiency of the regression estimator, ($b = 0.402, c = 0.268, s = 1.5$ gives a breakdown point of 0.5 of the S-estimator). $$\rho(x, b, c, s) = \int_0^x \psi(u, b, c, s) du$$ with, $$\psi(x, b, c, s) = \left\{ \begin{array}{ll} x &\quad \text{ if } |x| \leq c \\[2pt] \text{sign}(x) \left( |x| - \frac{s}{2b} (|x| - c)^2 \right) &\quad \text{ if } c < |x| \leq b + c \\[2pt] \text{sign}(x) \left( c + b - \frac{bs}{2} + \frac{s-1}{a} \left( \frac{1}{2} \tilde{x}^2 - a\tilde{x}\right) \right) &\quad \text{ if } b + c < |x| \leq a + b + c \\[2pt] 0 &\quad \text{ otherwise } \end{array} \right. $$ where $\tilde{x} = |x| - b - c$ and $a = (2c + 2b - bs) / (s - 1)$.

References

J.T. Barron (2019). A general and adaptive robust loss function. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 4331-4339).

M. Galassi et al., GNU Scientific Library Reference Manual (3rd Ed.), ISBN 0954612078.

R.A. Maronna et al., Robust Statistics: Theory and Methods, ISBN 0470010924.

Examples

Run this code

## Huber loss with default scale k
gsl_nls_loss(rho = "huber")

## L1-L2 loss (alpha = 1)
gsl_nls_loss(rho = "barron", cc = c(1, 1.345))

## Cauchy loss (alpha = 0)
gsl_nls_loss(rho = "barron", cc = c(k = 1.345, alpha = 0))

Run the code above in your browser using DataLab