luz_callback_gradient_clip

By adding the GradientClip callback, the gradient <code>norm_type</code> (default:2) norm
is clipped to at most <code>max_norm</code> (default:1) using <code>torch::nn_utils_clip_grad_norm_()</code>,
which can avoid loss divergence.

A high level interface for 'torch' providing utilities to reduce the
the amount of code needed for common tasks, abstract away torch details and
make the same code work on both the 'CPU' and 'GPU'. It's flexible enough to
support expressing a large range of models. It's heavily inspired by 'fastai' by
Howard et al. (2020) <doi:10.48550/arXiv.2002.04688>, 'Keras' by Chollet et al. (2015) and
'PyTorch Lightning' by Falcon et al. (2019) <doi:10.5281/zenodo.3828935>.

Daniel Falbel

Higher Level 'API' for 'torch'

Christophe Regouby

 RStudio

luz_callback_gradient_clip function

<dl><dt>max_norm</dt>
<dd>(float or int): max norm of the gradients</dd>
<dt>norm_type</dt>
<dd>(float or int): type of the used p-norm. Can be <code>Inf</code> for
infinity norm.</dd></dl>

Arguments

Gradient clipping callback — luz_callback_gradient_clip

<dl>

<dt>max_norm</dt>
<dd>(float or int): max norm of the gradients</dd>


<dt>norm_type</dt>
<dd>(float or int): type of the used p-norm. Can be <code>Inf</code> for
infinity norm.</dd>

</dl>

luz_callback_gradient_clip: Gradient clipping callback

Description

Usage

Arguments

References