Linear regression using two different classes of basis functions Let's say we have 1D data $Y = \{ y_i \mid y_i \in \mathbb{R} \}$, and regressors $X = \{ x_i \mid x_i \in \mathbb{R} \}$, and we try doing basis regression.
Suppose we find that we can perfectly describe $Y$ as being linear with respect to two different classes of basis functions: $Y = \Phi_1(X)\beta_1$ and $Y = \Phi_2(X)\beta_2$. (For a concrete example, suppose $\Phi_1(X) \in \mathbb{R}^{N \times K}$ comes from the responses of $K$ radial basis functions, while $\Phi_2(X) \in \mathbb{R}^{N \times K}$ is from $K$ cosine basis functions.)
Can we conclude anything about $\Phi_1(X)$ and $\Phi_2(X)$? I wouldn't have thought so, but empirically, it seems like you can write $\Phi_1(X) \approx \Phi_2(X) A + \mu$, with $A \in \mathbb{R}^{K \times K}$. In other words, the bases are linear transformations of one another.
Is this always true? This seems to suggest that if you can write some signal $y$ as being linear in some feature space, there is only one such feature space (up to linear transformations). Counterexamples would be very helpful!

I do not have enough reputation for a comment, so I make an answer.
I think you have an easy counter-example with $ X= y$ and $\Phi_i$ as the identity in the first coordinate and whatever else on other coordinates, taking $\beta_1 = \beta_2 = (1, 0, ..., 0)$
