We give the first efficient algorithm for the problem of list-decodable subspace recovery . Our algorithm takes input n samples \alpha n (  \alpha \ll 1/2) are generated i.i.d. from Gaussian distribution \mathcal{N(0,\Sigma_*) on R^d with covariance \Sigma_* of rank r }  with a isotropic rank r covariance \Pi_* (the } and the rest are arbitrary, potentially adversarial outliers. It outputs a list of O(1/\alpha) projection matrices guaranteed to contain a projection matrix%DIFDELCMD < \Pisuch %%% that\hat{\Pi}  satisfies certifiable anti-concentration. The running time of both these algorithms, however, is d^{\Omega(1/\alpha^4)} and the error bounds on } \|\Pi-\Pi_*\|_F ^2 = \kappa^4 \log (r )%DIFDELCMD < \tilde{O}%%% (\emph{  1/ \alpha^2), where%DIFDELCMD < \tilde{O} %%% hides polylogarithmic factors in 1/ \alpha . Here, \Pi_* is the projection matrix to the range space of \Sigma_*. The algorithm needs n=d ^{ \log ( r \kappa ) %DIFDELCMD < \tilde{O}%%% (\hat{\Pi} \hat{\Pi}  1/ \alpha^2) samples and runs in time n^{\log (r \kappa) \tilde{O(1/\alpha^4)} time where \kappa is the ratio of the largest to smallest non-zero eigenvalues of \Sigma_*. Our algorithm builds on the recently developed framework for list-decodable learning via the sum-of-squares (SoS)method }%DIFDELCMD < [%%% KKK'19, RY'20 with some key technical and conceptual advancements. Our key conceptual contribution involves showing a (SoS "certified") lower bound on the eigenvalues of covariances of arbitrary small subsamples of an i.i.d. sample of a certifiably anti-concentrated distribution. One of our key technical contributions gives a new method that allows error reduction "within SoS" with only a logarithmic cost in the exponent in the running time (in contrast to polynomial cost in%DIFDELCMD < [%%% KKK'19, RY'20 . In a concurrent and independent work , to have }\emph{  Raghavendra and Yau proved related results for list-decodable subspace recovery RY'20 satisfies certifiable anti-concentration, we obtain a stronger error guarantee of \|}\hat{\Pi} .
In list-decodable subspace recovery , the input is a collection of n points \alpha n ( for some \alpha \ll 1/2) of which are drawn i.i.d. from (0,\Sigma_*) on R^d with covariance \Sigma_* of rank r } a distribution \mathcal{D with a isotropic rank r covariance \Pi_* (the }inliers) and the rest are arbitrary, potential adversarial outliers. The goal is to recover a O(1/\alpha) %DIFDELCMD < \Pisuch %%% size list of candidate covariances that contains a\hat{\Pi} close to \Pi_*. Two recent independent works (Raghavendra-Yau, Bakshi-Kothari (2020)) gave algorithms for this problem that work whenever \mathcal{D satisfies certifiable anti-concentration. The running time of both these algorithms, however, is d^{\Omega(1/\alpha^4)} and the error bounds on } \|\Pi-\Pi_*\|_F %DIFDELCMD < \tilde{O}%%% grow with r (which can be \Omega(d)). In this work, we improve on these results on all three fronts:\emph{dimension-independent error via a faster fixed-polynomial running time under less restrictive distributional assumptions. Specifically, we give a poly( 1/ %DIFDELCMD < \tilde{O} %%% \alpha ) d ^{ O ( 1 ) %DIFDELCMD < \tilde{O}%%% time algorithm that outputs a list containing a\hat{\Pi} satisfying \|\hat{\Pi} -\Pi_*\|_F \leq O( 1/ (1/\alpha^4)} time where \kappa is the ratio of the largest to smallest non-zero eigenvalues of \Sigma_*. Our algorithm builds on the recently developed framework for list-decodable learning via the sum-of-squares (SoS)method }%DIFDELCMD < [%%% %DIFDELCMD < [%%% \alpha). Our result only needs \mathcal{D to have }\emph{certifiably hypercontractive degree 2 polynomials - a condition satisfied by a much broader family of distributions in contrast to certifiable anticoncentration. As a result, in addition to Gaussians, our algorithm applies to uniform distribution on the hypercube and q-ary cubes and arbitrary product distributions with subgaussian marginals. Prior work ( Raghavendra and Yau , 2020) had identified such distributions as potential hard examples as such distributions do not exhibit strong enough anti-concentration. When \mathcal{D satisfies certifiable anti-concentration, we obtain a stronger error guarantee of \|}\hat{\Pi}-\Pi_*\|_F \leq \eta for any arbitrary \eta > 0 in d^{O(poly(1/\alpha) + \log (1/\eta)) .
R
We give the first efficient algorithm for the problem of
In
R
. Our algorithm takes input n samples
, the input is a collection of n points
A
for some
R
are generated
of which are drawn
D
Gaussian distribution \mathcal{N
A
a distribution \mathcal{D
A
inliers
A
)
R
potentially
potential
R
It outputs a list of
The goal is to recover a
D
projection matrices guaranteed to contain a projection matrix
R
that
size list of candidate covariances that contains a
A
close to \Pi_*. Two recent independent works (Raghavendra-Yau, Bakshi-Kothari (2020)) gave algorithms for this problem that work whenever \mathcal{D
D
^2 = \kappa^4 \log (r )
R
(
grow with r (which can be \Omega(d)). In this work, we improve on these results on all three fronts:
A
dimension-independent
A
error via a faster fixed-polynomial running time under less restrictive distributional assumptions. Specifically, we give a poly(
D
\alpha^2), where
D
hides polylogarithmic factors in 1/
R
. Here, \Pi_* is the projection matrix to the range space of \Sigma_*. The algorithm needs n=d
) d
R
\log
O
R
r \kappa
1
R
(
time algorithm that outputs a list containing a
A
satisfying \|
A
-\Pi_*\|_F \leq O(
D
\alpha^2)
D
samples and runs in time n^{\log (r \kappa) \tilde{O
D
KKK'19, RY'20
D
with some key technical and conceptual advancements. Our key conceptual contribution involves showing a (SoS "certified") lower bound on the eigenvalues of covariances of arbitrary small subsamples of an i.i.d. sample of a certifiably anti-concentrated distribution. One of our key technical contributions gives a new method that allows error reduction "within SoS" with only a logarithmic cost in the exponent in the running time (in contrast to polynomial cost in
D
KKK'19, RY'20
R
. In a concurrent and independent work ,
\alpha). Our result only needs \mathcal{D
A
certifiably hypercontractive
A
degree 2 polynomials - a condition satisfied by a much broader family of distributions in contrast to certifiable anticoncentration. As a result, in addition to Gaussians, our algorithm applies to uniform distribution on the hypercube and q-ary cubes and arbitrary product distributions with subgaussian marginals. Prior work (
D
proved related results for list-decodable subspace recovery
R
RY'20
, 2020) had identified such distributions as potential hard examples as such distributions do not exhibit strong enough anti-concentration. When \mathcal{D
A
-\Pi_*\|_F \leq \eta for any arbitrary \eta > 0 in d^{O(poly(1/\alpha) + \log (1/\eta))