distance between regression models Consider two multivariate linear regression models (vector inputs and outputs) with the same domain observations.  Namely, let:

*

*$X \in \mathcal{R}^{a \times N}$ be a matrix of domain observations (design matrix)

*$Y_i \in \mathcal{R}^{b \times N_i}$ be one (of two) co-domain observations matrices

where corresponding columns of $X$ and $Y_i$ refer to paired vector observations.  One can find the matrix, $\beta_i$, which maps each observation in $X$ to $Y_i$ as $y = \beta x$ while minimizing the mean squared error as $\beta_i = Y_i X^+$ where $X^+$ is the right pseudo-inverse of $X$.
Is there a natural metric between these two regression models?
I'm looking for something which captures the covariances in $Y_i$ (so metrics on $\beta$ directly, the space of mappings from $X$ to $Y$, don't fit the bill since $\beta_0 = \beta_1$ does not imply $Y_0 = Y_1$.)

A model is really just a distribution;$^\dagger$ you can measure the distances between $Y_0$ and $Y_1$ (or rather, their distributions) via metrics like statistical distances. For example, f-divergences (which encompass the total variation distance, squared Hellinger distance, and Kullback-Leibler divergence).
$^\dagger$By that I mean $Y_{0, i} | X_i \sim \text{Normal}_i$ for $i \in 1, 2, \dots, n$ where $n$ is the number of observations you have. Same goes for $Y_1, i | X_i$.
References:

*

*Example on how to compute the Hellinger distance for Gaussian distributions

*Quick tutorial on Monte Carlo integration (use it to approximate expectations, e.g., in this case, f-divergences.)

