redistribute_excess_loss: Redistribute large losses for severity or risk-premium modelling

Description

Large claims can have a disproportionate influence on observed severity and on estimated risk-factor effects. redistribute_excess_loss() decomposes each selected claim amount into a retained component up to a specified threshold and an excess component above that threshold. The excess component is subsequently allocated using portfolio-wide, risk-factor-level or partially pooled experience. The allocation preserves the total excess loss, subject to numerical tolerance.

The allocated excess can be incorporated in the pricing analysis in two ways:

output = "redistributed_claim" adds the allocated excess loss to the retained claim amount. The resulting variable contains both components and can be used as the response in a single severity model.
output = "excess_loading" keeps retained claim severity and excess loss as separate quantities. The function returns an excess loading per unit of redistribution_weight. This loading can be added to the risk premium based on predicted frequency and retained severity.

Both output forms use the same threshold, credibility and allocation calculations. They therefore differ only in how the allocated excess loss is represented in subsequent modelling; the total amount allocated is the same.

The default is output = "redistributed_claim".

Usage

redistribute_excess_loss(
  data,
  claim_amount,
  threshold,
  claim_count = NULL,
  redistribution_weight = NULL,
  receives_redistribution = NULL,
  redistribute_excess = NULL,
  risk_factor = NULL,
  redistribution_method = c("portfolio", "risk_factor", "partial"),
  credibility = NULL,
  credibility_basis = c("claims", "excess_records"),
  credibility_threshold = 50,
  credibility_scale = 1,
  calculation_details = TRUE,
  output = c("redistributed_claim", "excess_loading")
)

Value

The input data.frame with additional columns and class "excess_redistribution". The object uses standard data.frame printing. summary.excess_redistribution() aggregates contributed, allocated and shifted loss. Both output forms add:

<claim_amount>_capped: Observed claim amount capped at threshold when its excess is allocated. Rows excluded through redistribute_excess retain their full observed amount.
<claim_amount>_excess: Observed amount above threshold, whether or not that amount is selected for allocation.
<claim_amount>_is_excess: Logical indicator that the observed claim amount exceeds threshold.

With output = "redistributed_claim", the result additionally contains:

<claim_amount>_redistributed_excess: Row-level allocated excess loss. Rows without claims receive zero.
<claim_amount>_adjusted: Retained claim amount plus row-level allocated excess loss.
<claim_amount>_adjusted_average: Redistributed claim amount divided by claim count. Rows without claims contain zero.

With output = "excess_loading", the result additionally contains:

allocated_excess_loss: Absolute excess-loss amount allocated to the row.
excess_loading: Allocated excess loss per unit of redistribution_weight.

With calculation_details = TRUE, the result also contains <risk_factor>_excess_loading, <risk_factor>_credibility, portfolio_excess_loading, blended_excess_loading, redistribution_scaling_factor and final_redistribution_loading. For receiving rows, final_redistribution_loading equals blended_excess_loading multiplied by redistribution_scaling_factor. The selected output and effective redistribution-weight label are stored in the "output" and "redistribution_weight_label" attributes.

Arguments

data: A data.frame containing portfolio-level or claim-level observations.
claim_amount: Character string naming a finite, non-negative numeric column with observed claim amounts or aggregate claim loss per row.
threshold: Positive numeric scalar defining the boundary between retained and excess loss. For selected rows, the amount above this value is allocated.
claim_count: Optional character string naming a non-negative, whole-number claim-count column. Claim count identifies claim-bearing rows and is the denominator of the adjusted average claim amount. It is also the default redistribution weight. If NULL, each row with claim_amount > 0 is treated as one claim.
redistribution_weight: Optional character string naming a finite, non-negative numeric column. The column determines the relative allocation shares and the unit of the resulting loading. If NULL, claim count is used. Claim count or expected claim count expresses the allocation per claim; earned exposure expresses it per exposure unit. Rows with zero weight receive no allocation. At least one eligible row must have positive weight.
receives_redistribution: Optional character string. Logical column indicating which rows may receive allocated excess loss. Rows with FALSE receive zero, while their observed excess remains in the total allocation unless excluded by redistribute_excess. If NULL, all otherwise eligible rows are included. Eligibility additionally requires positive claim count for redistributed claims and positive redistribution weight for excess loadings.
redistribute_excess: Optional character string. Logical column indicating which rows contribute their excess component to the allocation. Rows with FALSE retain their full observed amount and contribute no excess to the allocation pool. If NULL, every row above threshold contributes.
risk_factor: Optional character string naming the risk-factor column for redistribution_method = "risk_factor" or redistribution_method = "partial".
redistribution_method: Character string specifying the experience level used in the allocation: "portfolio", "risk_factor" or "partial".
credibility: Optional numeric scalar in [0, 1]. For partial redistribution, the supplied value is applied to every risk-factor level. If NULL, credibility is calculated from credibility_basis.
credibility_basis: Character string specifying the experience measure used in automatic credibility: "claims" or "excess_records".
credibility_threshold: Positive numeric scalar representing the amount of credibility-basis experience at which automatic credibility equals 0.5, before applying credibility_scale.
credibility_scale: Non-negative numeric scalar multiplying automatic credibility before truncation to [0, 1].
calculation_details: Logical. If TRUE, append the risk-factor loading, credibility, portfolio loading, blended loading, scaling factor and final loading used in the row-level calculation. If FALSE, these columns are omitted from data but remain available through summary.excess_redistribution().
output: Character string specifying the representation of allocated excess. "redistributed_claim" adds it to retained claim amounts; "excess_loading" returns it separately per unit of redistribution_weight.

Author

Martin Haringa

Details

For each row selected through redistribute_excess, the observed claim amount is decomposed as:

$$ claim\_amount = capped\_claim\_amount + excess\_claim\_amount $$

If a row is not selected through redistribute_excess, its full observed amount is retained. Any amount above the threshold remains available as a diagnostic quantity but is excluded from the amount to be allocated.

The excess amount is allocated over eligible rows in proportion to redistribution_weight. If no weight column is supplied, claim count is used. For redistributed-claim output, only rows with a positive claim count are eligible. For excess-loading output, eligibility is determined by a positive redistribution weight, so policy rows without observed claims may receive a loading when exposure is used.

For output = "redistributed_claim", the redistributed claim amount is:

$$ adjusted\_claim\_amount = capped\_claim\_amount + redistributed\_excess $$

This redistributed claim amount is returned as <claim_amount>_adjusted. The corresponding average per claim is suitable as the response in a claim-count-weighted severity GLM. Because this response includes allocated excess loss, the same excess component should not subsequently be added to the estimated risk premium.

For output = "excess_loading", capped claim severity remains separate and:

$$ excess\_loading_i = \frac{allocated\_excess\_loss_i}{redistribution\_weight_i} $$

The loading can be added to the risk premium derived from predicted frequency and retained severity. When earned exposure is used as redistribution_weight, the loading is expressed per unit of earned exposure. The total allocated excess loss is preserved, subject to numerical tolerance.

Interpretation of the output forms

Redistributed-claim output combines retained and allocated loss in one model response. It requires one severity model and assigns the complete historical loss burden to that response. Its interpretation is most direct when the modelled risk-factor levels contain sufficient claim experience and the estimated effects are stable across observation periods.

The allocated component is not an observed loss for the receiving row. For example, an observed claim of 10,000 may receive an allocation of 20,000, resulting in a redistributed amount of 30,000. If the row belongs to a risk-factor level with few claims, the fitted model may attribute a material part of this allocated portfolio experience to that individual level. This can increase the sampling variability of its estimated effect.

Excess-loading output estimates retained severity from observed loss up to the threshold and represents allocated excess as a separate risk-premium component:

$$ retained\ risk\ premium = predicted\ frequency \cdot predicted\ retained\ severity $$

$$ total\ risk\ premium = retained\ risk\ premium + excess\ loading $$

The two output forms therefore imply different model interpretations rather than different total loss amounts. The selection should reflect claim volume, the stability of risk-factor effects and the intended construction of the technical risk premium.

Sparse risk-factor levels

Before fitting a redistributed-claim severity model, claim volume should be assessed by risk-factor level. Levels with limited information may be combined using an economically or actuarially meaningful hierarchy. Coefficient stability across periods and agreement between observed and predicted severity provide additional diagnostics. Excess-loading output is an alternative when separate level estimates remain weakly supported.

For example, a model-preparation rule may map sectors with fewer than 20 claims to "Other". The value 20 is illustrative and should not be treated as a general minimum. An appropriate threshold depends on portfolio size, heterogeneity and validation results. Grouping is therefore outside the scope of this function.

Reproducing the redistribution

The optional calculation columns allow the allocation to be reproduced. For partial redistribution, the loading before total-preservation scaling is:

$$ blended\_loading = Z_g \cdot risk\_factor\_loading_g + (1 - Z_g) \cdot portfolio\_loading $$

where Z_g denotes the credibility assigned to risk-factor level g. Blending may change the total amount implied by the unscaled loadings. A common scaling factor is therefore applied:

$$ preservation\_factor = \frac{total\ excess\ to\ redistribute} {\sum_i blended\_loading_i \cdot redistribution\_weight_i} $$

The final amount received by row i is:

$$ redistributed\_excess_i = blended\_loading_i \cdot preservation\_factor \cdot redistribution\_weight_i $$

For example, let the sector loading be 30 per unit of weight, the portfolio loading 20 and sector credibility 0.40. The blended loading equals 0.40 * 30 + 0.60 * 20 = 24. With a scaling factor of 1.10, the final loading equals 26.4. A receiving row with redistribution weight 2 is then allocated 26.4 * 2 = 52.8. In redistributed-claim output, 52.8 is added to the retained claim amount; in excess-loading output, 26.4 is retained as the loading per unit of weight.

With portfolio redistribution, credibility is zero and the blend equals the portfolio loading. With risk-factor redistribution, credibility is one and the blend equals the risk-factor loading.

Eligibility and redistribution weights

receives_redistribution identifies the rows to which excess loss may be allocated. Rows with value FALSE receive zero. Their observed excess loss is nevertheless included in the total amount to be allocated unless excluded through redistribute_excess. For redistributed-claim output, a receiving row must also have a positive claim count. For excess-loading output, a receiving row must have a positive redistribution weight.

redistribute_excess controls which large losses contribute their excess part to the redistribution. If it is NULL, every row with claim_amount > threshold contributes. For a row marked FALSE, the claim is not capped: its full observed amount is retained and its excess does not enter the allocation pool. This permits specific loss types, such as events treated outside the regular large-loss procedure, to remain unchanged.

redistribution_weight controls both the relative shares among receiving rows and the unit of the resulting loading. Claim count produces an amount per claim, expected claim count an amount per expected claim, earned exposure an amount per exposure unit, and insured amount an amount per unit insured. If it is NULL, claim count is used. For an excess loading that is added to an annual risk premium, earned exposure is usually the corresponding unit.

Rows with zero redistribution weight remain in the output but receive zero. For example, claim_count * insured_amount assigns a larger share to claim observations with a higher insured amount. The excess threshold and an insured-amount criterion for receiving allocations are separate model specifications.

Redistribution methods

The redistribution_method argument determines the level at which excess experience is estimated:

"portfolio" derives one loading from the complete allocation portfolio. Risk-factor-level excess experience is not used.
"risk_factor" derives a separate loading for each risk-factor level. Each level is based only on its own excess experience and allocation weight.
"partial" blends the risk-factor loading with the portfolio loading using credibility.

For partial redistribution, credibility is either supplied directly through credibility or calculated as:

$$ Z_g = \frac{n_g}{n_g + credibility\_threshold} $$

With credibility_basis = "claims", n_g is the number of claims in the risk-factor level. With credibility_basis = "excess_records", it is the number of records containing a positive excess amount. The resulting value is multiplied by credibility_scale and bounded between zero and one.

Credibility and redistribution weight have distinct roles. Credibility determines the contribution of risk-factor-level experience to a partial loading. Redistribution weight determines the row-level allocation shares and the unit of the final loading. Thus, with credibility_basis = "claims" and earned exposure as redistribution_weight, claim volume determines credibility while the resulting loading is expressed per unit of exposure.

Aggregated portfolio rows

When a row contains multiple claims, claim_amount is treated as the total loss for that row and threshold is applied to that row total. The function cannot identify which individual claims exceeded the threshold from an aggregated row. Claim-level input is required when the threshold is intended to apply separately to each claim.

Examples

Run this code

portfolio <- data.frame(
  policy_id = 1:10,
  sector = c(rep("Industry", 5), rep("Retail", 4), "Office"),
  claim_count = c(0, 1, 1, 1, 1, 0, 1, 1, 1, 1),
  claim_amount = c(
    0, 25000, 120000, 50000, 175000,
    0, 40000, 90000, 150000, 300000
  ),
  policy_years = rep(1, 10)
)

# Output form 1: include allocated excess in the severity response.
adjusted <- redistribute_excess_loss(
  portfolio,
  claim_amount = "claim_amount",
  threshold = 100000,
  claim_count = "claim_count",
  risk_factor = "sector",
  redistribution_method = "partial",
  output = "redistributed_claim"
)
summary(adjusted)

# Inspect the row-level calculation. The allocated amount in the final column
# equals final_redistribution_loading times redistribution weight.
adjusted[c(
  "sector_excess_loading", "sector_credibility",
  "portfolio_excess_loading", "blended_excess_loading",
  "redistribution_scaling_factor", "final_redistribution_loading",
  "claim_amount_redistributed_excess"
)]

# Omit row-level calculation columns while retaining them in summary().
compact_adjusted <- redistribute_excess_loss(
  portfolio,
  claim_amount = "claim_amount",
  threshold = 100000,
  claim_count = "claim_count",
  risk_factor = "sector",
  redistribution_method = "partial",
  calculation_details = FALSE
)
summary(compact_adjusted)

# Combine levels with limited claim experience before model estimation.
# Three claims is used for this small example; it is not a general minimum.
adjusted$sector_claim_count <- ave(
  adjusted$claim_count, adjusted$sector, FUN = sum
)
adjusted$sector_model <- ifelse(
  adjusted$sector_claim_count >= 3,
  adjusted$sector,
  "Other"
)

# Fit a severity model to redistributed average claim amount. For aggregated
# rows, claim count represents the number of observations underlying each
# average. With one row per claim, this additional weight is unnecessary.
severity_data <- adjusted[adjusted$claim_count > 0, ]
stats::glm(
  claim_amount_adjusted_average ~ sector_model,
  weights = claim_count,
  family = stats::Gamma(link = "log"),
  data = severity_data
)

# Output form 2: estimate retained severity and excess loading separately.
# Using policy years expresses excess_loading per policy year.
loading_result <- redistribute_excess_loss(
  portfolio,
  claim_amount = "claim_amount",
  threshold = 100000,
  claim_count = "claim_count",
  redistribution_weight = "policy_years",
  risk_factor = "sector",
  redistribution_method = "partial",
  output = "excess_loading"
)

frequency_model <- stats::glm(
  claim_count ~ sector + offset(log(policy_years)),
  family = stats::poisson(link = "log"),
  data = loading_result
)
retained_severity_model <- stats::glm(
  claim_amount_capped ~ sector,
  weights = claim_count,
  family = stats::Gamma(link = "log"),
  data = loading_result[loading_result$claim_count > 0, ]
)

loading_result$predicted_claim_frequency <- stats::predict(
  frequency_model,
  newdata = loading_result,
  type = "response"
) / loading_result$policy_years
loading_result$predicted_retained_severity <- stats::predict(
  retained_severity_model,
  newdata = loading_result,
  type = "response"
)
loading_result$predicted_retained_risk_premium <-
  loading_result$predicted_claim_frequency *
  loading_result$predicted_retained_severity
loading_result$predicted_total_risk_premium <-
  loading_result$predicted_retained_risk_premium +
  loading_result$excess_loading

# Portfolio redistribution estimates one loading across all sectors. Sector-
# specific excess experience does not enter the allocation loading.
portfolio_adjusted <- redistribute_excess_loss(
  portfolio,
  claim_amount = "claim_amount",
  threshold = 100000,
  claim_count = "claim_count",
  redistribution_method = "portfolio"
)
summary(portfolio_adjusted, by = "sector")

# Risk-factor redistribution estimates a separate loading for each sector.
# The estimate for a sector uses only that sector's excess loss and weight.
sector_adjusted <- redistribute_excess_loss(
  portfolio,
  claim_amount = "claim_amount",
  threshold = 100000,
  claim_count = "claim_count",
  risk_factor = "sector",
  redistribution_method = "risk_factor"
)

# Allocate in proportion to claim count times insured amount, restricted to
# policies with an insured amount of at least 100,000.
weighted_portfolio <- transform(
  portfolio,
  insured_amount = rep(c(50000, 250000), each = 5)
)
weighted_portfolio$receives_redistribution <-
  weighted_portfolio$insured_amount >= 100000
weighted_portfolio$redistribution_weight <-
  weighted_portfolio$claim_count * weighted_portfolio$insured_amount

weighted_adjusted <- redistribute_excess_loss(
  weighted_portfolio,
  claim_amount = "claim_amount",
  threshold = 100000,
  claim_count = "claim_count",
  redistribution_weight = "redistribution_weight",
  receives_redistribution = "receives_redistribution"
)

# Exclude catastrophe events and unsettled claims from the allocation pool.
# Their full observed claim amounts remain retained in the model data.
portfolio$is_catastrophe <- c(
  FALSE, FALSE, FALSE, FALSE, TRUE,
  FALSE, FALSE, FALSE, FALSE, FALSE
)
portfolio$claim_status <- c(
  "settled", "settled", "settled", "settled", "settled",
  "settled", "settled", "open", "settled", "settled"
)
portfolio$redistribute_excess <-
  !portfolio$is_catastrophe &
  portfolio$claim_status == "settled"

selected_adjusted <- redistribute_excess_loss(
  portfolio,
  claim_amount = "claim_amount",
  threshold = 100000,
  claim_count = "claim_count",
  redistribute_excess = "redistribute_excess"
)

Run the code above in your browser using DataLab