nn.functional

Functional implementations

Table of Contents

Loss Functions

linear_operator_learning.nn.functional.l2_contrastive_loss(x, y)[source]

See linear_operator_learning.nn.L2ContrastiveLoss for details.

Parameters:

  • x (Tensor)

  • y (Tensor)

Return type:

Tensor

linear_operator_learning.nn.functional.kl_contrastive_loss(X, Y)[source]

See linear_operator_learning.nn.KLContrastiveLoss for details.

Parameters:

  • X (Tensor)

  • Y (Tensor)

Return type:

Tensor

linear_operator_learning.nn.functional.vamp_loss(x, y, schatten_norm=2, center_covariances=True)[source]

See linear_operator_learning.nn.VampLoss for details.

Parameters:

  • x (Tensor)

  • y (Tensor)

  • schatten_norm (int)

  • center_covariances (bool)

Return type:

Tensor

linear_operator_learning.nn.functional.dp_loss(x, y, relaxed=True, center_covariances=True)[source]

See linear_operator_learning.nn.DPLoss for details.

Parameters:

  • x (Tensor)

  • y (Tensor)

  • relaxed (bool)

  • center_covariances (bool)

Return type:

Tensor

Regularization Functions

linear_operator_learning.nn.functional.orthonormal_fro_reg(x)[source]

Orthonormality regularization with Frobenious norm of covariance of x.

Given a batch of realizations of x, the orthonormality regularization term penalizes:

  1. Orthogonality: Linear dependencies among dimensions,

  2. Normality: Deviations of each dimension’s variance from 1,

  3. Centering: Deviations of each dimension’s mean from 0.

\[\frac{1}{D} \| \mathbf{C}_{X} - I \|_F^2 + 2 \| \mathbb{E}_{X} x \|^2 = \frac{1}{D} (\text{tr}(\mathbf{C}^2_{X}) - 2 \text{tr}(\mathbf{C}_{X}) + D + 2 \| \mathbb{E}_{X} x \|^2)\]
Parameters:

x (Tensor) – Input features.

Return type:

Tensor

Shape:

x: \((N, D)\), where \(N\) is the batch size and \(D\) is the number of features.

linear_operator_learning.nn.functional.orthonormal_logfro_reg(x)[source]

Orthonormality regularization with log-Frobenious norm of covariance of x by Kostic et al.[1].

\[\frac{1}{D}\text{Tr}(C_X^{2} - C_X -\ln(C_X)).\]
Parameters:

x (Tensor) – Input features.

Return type:

Tensor

Shape:

x: \((N, D)\), where \(N\) is the batch size and \(D\) is the number of features.