nn.stats

Statistics Utilities

linear_operator_learning.nn.stats.covariance(X, Y=None, center=True, norm=None)

Computes the covariance of X or cross-covariance between X and Y if Y is given.

Parameters:

• X (Tensor) – Input features.

• Y (Tensor | None, optional) – Output features. Defaults to None.

• center (bool, optional) – Whether to compute centered covariances. Defaults to True.

• norm (float | None, optional) – Normalization factor. Defaults to None.

Return type:

Tensor

Shape:

X: \((N, D)\), where \(N\) is the batch size and \(D\) is the number of features.

Y: \((N, D)\), where \(N\) is the batch size and \(D\) is the number of features.

Output: \((D, D)\), where \(D\) is the number of features.

linear_operator_learning.nn.stats.cov_norm_squared_unbiased(x, permutation=None)

Compute the unbiased estimation of \(\|\mathbf{C}_x\|_F^2\) from a batch of samples.

Given the Covariance matrix \(\mathbf{C}_x = \mathbb{E}_p(x) [x^{\top} x]\), this function computes an unbiased estimation of the Frobenius norm of the covariance matrix from a single sampling set.

\[\begin{split}\begin{align} \|\mathbf{C}_x\|_F^2 &= \text{tr}(\mathbf{C}_x^{\top} \mathbf{C}_x) = \sum_i \sum_j (\mathbb{E}_{x} [x_i x_j]) (\mathbb{E}_{x'} [x'_j x'_i]) \\ &= \mathbb{E}_{x,x' \sim p(x)} [(x^{\top} x')^2] \\ &\approx \frac{1}{N^2} \sum_n \sum_m [(x_n^{\top} x'_m)^2] \end{align}\end{split}\]

Note

The random variable is assumed to be centered.

Parameters:

• x (Tensor) – (n_samples, r) Centered realizations of a random variable x = [x_1, …, x_r].

• permutation (Tensor, optional) – List of integer indices of shape (n_samples,) used to permute the samples.

Returns:

Tensor – Unbiased estimation of \(\|\mathbf{C}_x\|_F^2\) using U-statistics.

linear_operator_learning.nn.stats.cross_cov_norm_squared_unbiased(x, y, permutation=None)

Compute the unbiased estimation of \(\|\mathbf{C}_{xy}\|_F^2\) from a batch of samples, using U-statistics.

Given the Covariance matrix \(\mathbf{C}_{xy} = \mathbb{E}_p(x,y) [x^{\top} y]\), this function computes an unbiased estimation of the Frobenius norm of the covariance matrix from two independent sampling sets (an effective samples size of \(N^2\)).

\[\begin{split}\begin{align} \|\mathbf{C}_{xy}\|_F^2 &= \text{tr}(\mathbf{C}_{xy}^{\top} \mathbf{C}_{xy}) = \sum_i \sum_j (\mathbb{E}_{x,y \sim p(x,y)} [x_i y_j]) (\mathbb{E}_{x',y' \sim p(x,y)} [x_j y_i']) \\ &= \mathbb{E}_{(x,y),(x',y') \sim p(x,y)} [(x^{\top} y') (x'^{T} y)] \\ &\approx \frac{1}{N^2} \sum_n \sum_m [(x_{n}^{\top} y^{\prime}_m) (x^{\prime \top}_m y_n)] \end{align}\end{split}\]

The random variable is assumed to be centered.

Parameters:

• x (Tensor) – Centered realizations of a random variable x of shape (N, D_x).

• y (Tensor) – Centered realizations of a random variable y of shape (N, D_y).

• permutation (Tensor, optional) – List of integer indices of shape (n_samples,) used to permute the samples.

Returns:

Tensor – Unbiased estimation of \(\|\mathbf{C}_{xy}\|_F^2\) using U-statistics.

linear_operator_learning.nn.stats.whitening(u, v)

Computes whitening matrices for u and v.

Parameters:

• u (Tensor) – Input features.

• v (Tensor) – Output features.

Return type:

tuple

Shape:

u: \((N, D)\), where \(N\) is the batch size and \(D\) is the number of features.

v: \((N, D)\), where \(N\) is the batch size and \(D\) is the number of features.

sqrt_cov_u_inv: \((D, D)\)

sqrt_cov_v_inv: \((D, D)\)

sing_val: \((D,)\)

sing_vec_l: \((D, D)\)

sing_vec_r: \((D, D)\)