  Although equivariant machine learning has proven effective at many tasks,
success depends heavily on the assumption that the ground truth function is
symmetric over the entire domain matching the symmetry in an equivariant neural
network. A missing piece in the equivariant learning literature is the analysis
of equivariant networks when symmetry exists only partially in the domain. In
this work, we present a general theory for such a situation. We propose
pointwise definitions of correct, incorrect, and extrinsic equivariance, which
allow us to quantify continuously the degree of each type of equivariance a
function displays. We then study the impact of various degrees of incorrect or
extrinsic symmetry on model error. We prove error lower bounds for invariant or
equivariant networks in classification or regression settings with partially
incorrect symmetry. We also analyze the potentially harmful effects of
extrinsic equivariance. Experiments validate these results in three different
environments.

Machine Learning;Machine Learning