Question: Does a fully convolutional network share the same translation invariance properties we get from networks that use max-pooling?

If not, why do they perform as well as networks which use max-pooling?

Answer: 
Neural networks are not **invariant** to translations, but **equivariant**,

### Invariance vs Equivariance

Suppose we have input $x$ and the output $y=f(x)$ of some map between spaces $X$ and $Y$. We apply transformation $T$ in the input domain. For general map,output will change in some complicated and unpredictable way. However, for certain class of maps, change of the output becomes very tractable.

**Invariance** means that output doesn't change after application of the map $T$. Namely:
$$
f(T(x)) = f(x)
$$

For CNN example of the map, invariant to translations, is the **GlobalPooling** operation.

**Equivariance** means that symmetry transformation $T$ on the input domain leads to the symmetry transformation $T^{'}$ on the output. Here $T^{'}$ can be the same map $T$, identity map - which reduces to invariance, or some other kind of transformation.

This picture is illustration of translational equivariance.

[![from https://towardsdatascience.com/translational-invariance-vs-translational-equivariance-f9fbc8fca63a](https://i.stack.imgur.com/hxRvr.png)](https://i.stack.imgur.com/hxRvr.png)

### Equivariance of operations in CNN

* Convolutions with `stride=1`:
$$ f(T(x)) = T f(x)
$$
Output feature map is shifted in same direction and number of steps.
* Downsampling operations. Convolutions with `stride=1`, `Pooling` (non-global):
$$ f(T\_{1/s}(x)) = T\_{1/s} f(x)
$$
They are equivariant to the subgroup of translations, which involves translations with integer number of strides.
* `GlobalPooling` :
$$ f(T(x)) = f(x)
$$
These are invariant to arbitrary shifts, this property is useful in classification tasks.

### Combination of layers

Stacking multiple equivariant layers you obtain equivariant architecture a whole.

For classification layer it makes sense to put `GlobalPooling` in the end in order to for NN to output the same probabilities for the shifted image.

For **segmentation** or **detection** problem architecture should be equivariant with the same map $T$, in order to translate bounding boxes or segmentation masks by the same amount as the transform on the input.

Non-global downsampling operations *reduce equivariance* to the subgroup with shifts integer multiples of stride.
All convolutional networks (with or without max-pooling) are translation-invariant (AKA spatially invariant) because their filters slide over every position in the image. This means that if a pattern that "matches" a filter is present anywhere in the image, then at least one neuron should activate.

Max-pooling, on the other hand, has nothing to do with spatial invariance. It's simply a regularization technique to help reduce the number of parameters later in the network by downsizing activation layers within the network. This can help combat overfitting, although it's not strictly necessary. Alternatively, neural networks can achieve the same effect by using a convolutional layer with a stride of 2 instead of 1.