KFAS (version 1.5.1)

KFS: Kalman Filter and Smoother with Exact Diffuse Initialization for Exponential Family State Space Models

Description

Performs Kalman filtering and smoothing with exact diffuse initialization using univariate approach for exponential family state space models.

Usage

KFS(
  model,
  filtering,
  smoothing,
  simplify = TRUE,
  transform = c("ldl", "augment"),
  nsim = 0,
  theta,
  maxiter = 50,
  convtol = 1e-08,
  return_model = TRUE,
  expected = FALSE,
  H_tol = 1e+15,
  transform_tol
)

Value

What KFS returns depends on the arguments filtering,

smoothing and simplify, and whether the model is Gaussian or not:

model

Original state space model.

KFS_transform

How the non-diagonal H was handled.

logLik

Value of the log-likelihood function. Only returned for fully Gaussian models.

a

One-step-ahead predictions of states, $a_t=E({\alpha }_t|y_{t-1},\dots ,y_1)$.

P

Non-diffuse parts of the error covariance matrix of predicted states, $P_t=Var({\alpha }_t|y_{t-1},\dots ,y_1)$.

Pinf

Diffuse part of the error covariance matrix of predicted states. Only returned for Gaussian models.

att

Filtered estimates of states, $a_{t}t=E({\alpha }_t|y_t,\dots ,y_1)$.

Ptt

Non-diffuse parts of the error covariance matrix of filtered states, $P_{t}t=Var({\alpha }_t|y_t,\dots ,y_1)$.

t

One-step-ahead predictions of signals, $E(Z_t{\alpha }_t|y_{t-1},\dots ,y_1)$.

P_theta

Non-diffuse part of $Var(Z_t{\alpha }_t|y_{t-1},\dots ,y_1)$.

m

One-step-ahead predictions $f({\theta }_t)|y_{t-1},\dots ,y_1)$, where $f$ is the inverse link function. In case of Poisson distribution these predictions are multiplied with exposure $u_t$.

P_mu

Non-diffuse part of $Var(f({\theta }_t)|y_{t-1},\dots ,y_1)$. In case of Poisson distribution this is $Var(u_{t}f({\theta }_t)|y_{t-1},\dots ,y_1)$. If nsim = 0, only diagonal elements (variances) are computed, using the Delta method.

alphahat

Smoothed estimates of states, $E({\alpha }_t|y_1,\dots ,y_n)$.

V

Error covariance matrices of smoothed states, $Var({\alpha }_t|y_1,\dots ,y_n)$.

thetahat

Smoothed estimates of signals, $E(Z_t{\alpha }_t|y_1,\dots ,y_n)$.

V_theta

Error covariance matrices of smoothed signals $Var(Z[t]{\alpha }_t|y_1,\dots ,y_n).$.

muhat

Smoothed estimates of $f({\theta }_t)|y_1,\dots ,y_n)$, where $f$ is the inverse link function, or in Poisson case $u_{t}f({\theta }_t)|y_1,\dots ,y_n)$, where $u$ is the exposure term.

V_mu

Error covariances $Cov(f({\theta }_t)|y_1,\dots ,y_n)$ (or the covariances of $u_{t}f({\theta }_t)$ given the data in case of Poisson distribution). If nsim = 0, only diagonal elements (variances) are computed, using the Delta method.

etahat

Smoothed disturbance terms $E({\eta }_t|y_1,\dots ,y_n)$. Only for Gaussian models.

V_eta

Error covariances $Var({\eta }_t|y_1,\dots ,y_n)$. Note that for computing auxiliary residuals you shoud use method rstandard.KFS.

epshat

Smoothed disturbance terms $E({ϵ}_{t,i}|y_1,\dots ,y_n)$. Note that due to the possible diagonalization these are on transformed scale. Only for Gaussian models.

V_eps

Diagonal elements of $Var({ϵ}_t|y_1,\dots ,y_n)$. Note that due to the diagonalization the off-diagonal elements are zero. Only for Gaussian models. Note that for computing auxiliary residuals you shoud use method rstandard.KFS.

iterations

The number of iterations used in linearization of non-Gaussian model.

v

Prediction errors $v_{t,i}=y_{t,i}-Z_{i,t}{a}_{t,i},i=1,\dots ,p$, where $$a_{t,i}=E({\alpha }_t|y_{t,i-1},\dots ,y_{t,1},\dots ,y_{1,1})$$. Only returned for Gaussian models.

F

Prediction error variances $Var(v_{t,i})$. Only returned for Gaussian models.

Finf

Diffuse part of prediction error variances. Only returned for Gaussian models.

d

The last time index of diffuse phase, i.e. the non-diffuse phase began at time $d+1$. Only returned for Gaussian models.

j

The last observation index $i$ of $y_{i,t}$ of the diffuse phase. Only returned for Gaussian models.

In addition, if argument simplify = FALSE, list contains following components:

K

Covariances $Cov({\alpha }_{t,i},y_{t,i}|y_{t,i-1},\dots ,y_{t,1},y_{t-1},\dots ,y_1),i=1,\dots ,p$.

Kinf

Diffuse part of $K_t$.

r

Weighted sums of innovations $v_{t+1},\dots ,v_n$. Notice that in literature $t$ in $r_t$ goes from $0,\dots ,n$. Here $t=1,\dots ,n+1$. Same applies to all $r$ and $N$ variables.

r0, r1

Diffuse phase decomposition of $r_t$.

N

Covariances $Var(r_t)$.

N0, N1, N2

Diffuse phase decomposition of $N_t$.

Arguments

model

Object of class SSModel.

filtering

Types of filtering. Possible choices are "state", "signal", "mean", and "none". Default is "state" for Gaussian and "none" for non-Gaussian models. Multiple values are allowed. For Gaussian models, the signal is the mean. Note that filtering for non-Gaussian models with importance sampling can be very slow with large models.

smoothing

Types of smoothing. Possible choices are "state", "signal", "mean", "disturbance", and "none". Default is "state" and "mean". For non-Gaussian models, option "disturbance" is not supported, and for Gaussian models option "mean" is identical to "signal". Multiple values are allowed.

simplify

If FALSE and the model is completely Gaussian, KFS returns some generally not so interesting variables from filtering and smoothing. Default is TRUE.

transform

How to transform the model in case of non-diagonal covariance matrix H. Defaults to "ldl". See function transformSSM for details.

nsim

The number of independent samples used in importance sampling. Only used for non-Gaussian models. Default is 0, which computes the approximating Gaussian model by approxSSM and performs the usual Gaussian filtering/smoothing so that the smoothed state estimates equals to the conditional mode of $p({\alpha }_t|y)$. In case of nsim = 0, the mean estimates and their variances are computed using the Delta method (ignoring the covariance terms).

theta

Initial values for conditional mode theta. Only used for non-Gaussian models.

maxiter

The maximum number of iterations used in Gaussian approximation. Default is 50. Only used for non-Gaussian models.

convtol

Tolerance parameter for convergence checks for Gaussian approximation. Only used for non-Gaussian models.

return_model

Logical, indicating whether the original input model should be returned as part of the output. Defaults to TRUE, but for large models can be set to FALSE in order to save memory. However, many of the methods operating on the output of KFS use this model so this will not work if return_model=FALSE.

expected

Logical value defining the approximation of H_t in case of Gamma and negative binomial distribution. Default is FALSE which matches the algorithm of Durbin & Koopman (1997), whereas TRUE uses the expected value of observations in the equations, leading to results which match with glm (where applicable). The latter case was the default behaviour of KFAS before version 1.3.8. Essentially this is the difference between observed and expected information in the GLM context. Only used for non-Gaussian model.

H_tol

Tolerance parameter for check max(H) > tol_H, which suggests that the approximation converged to degenerate case with near zero signal-to-noise ratio. Default is very generous 1e15. Only used for non-Gaussian model.

transform_tol

Tolerance parameter for LDL decomposition in case of a non-diagonal H and transform = "ldl". See transformSSM and ldl for details.

Details

Notice that in case of multivariate Gaussian observations, v, F, Finf, K and Kinf are usually not the same as those calculated in usual multivariate Kalman filter. As filtering is done one observation element at the time, the elements of the prediction error $v_t$ are uncorrelated, and F, Finf, K and Kinf contain only the diagonal elemens of the corresponding covariance matrices. The usual multivariate versions of F and v can be obtained from the output of KFS using the function mvInnovations.

In rare cases (typically with regression components with high multicollinearity or long cyclic patterns), the cumulative rounding errors in Kalman filtering and smoothing can cause the diffuse phase end too early, or the backward smoothing gives negative variances (in diffuse and nondiffuse cases). Since version 1.4.0, filtering and smoothing algorithms truncate these values to zero during the recursions, but this can still leads some numerical problems. In these cases, redefining the prior state variances more informative is often helpful. Changing the tolerance parameter tol of the model (see SSModel) to smaller (or larger), or scaling the model input can sometimes help as well. These numerical issues are well known in Kalman filtering/smoothing in general (there are other numerically more stable versions available, but these are in general slower).

Fon non-Gaussian models the components corresponding to diffuse filtering (Finf, Pinf, d, Kinf) are not returned even when filtering is used. Results based on approximating Gaussian model can be obtained by running KFS using the output of approxSSM.

In case of non-Gaussian models with nsim = 0, the smoothed estimates relate to the conditional mode of $p(\alpha |y)$. When using importance sampling (nsim>0), results correspond to the conditional mean of $p(\alpha |y)$.

References

Koopman, S.J. and Durbin J. (2000). Fast filtering and smoothing for non-stationary time series models, Journal of American Statistical Assosiation, 92, 1630-38.

Koopman, S.J. and Durbin J. (2001). Time Series Analysis by State Space Methods. Oxford: Oxford University Press.

Koopman, S.J. and Durbin J. (2003). Filtering and smoothing of state vector for diffuse state space models, Journal of Time Series Analysis, Vol. 24, No. 1.

See Also

KFAS for examples

logLik, KFAS, fitSSM, boat, sexratio, GlobalTemp, SSModel, importanceSSM, approxSSM for examples.

Examples

set.seed(1)
x <- cumsum(rnorm(100, 0, 0.1))
y <- rnorm(100, x, 0.1)
model <- SSModel(y ~ SSMtrend(1, Q = 0.01), H = 0.01)
out <- KFS(model)
ts.plot(ts(x), out$a, out$att, out$alpha, col = 1:4)