Bilinear Interpolation
I am trying to implement bilinear interpolation as described in the paper Spatial Tranformer Networks by <PERSON> et. al (see link to paper). They describe bilinear interpolation in Equation 5 as:
$$ V_i^c = \sum_{n}^{H}\sum_{m}^{W} U_{nm}^c \max(0,1-|x_i^s - m|)\cdot\max(0,1-|y_i^s - n|), $$ where:
* $V_i^c$ is the resulting pixel value in the new image
* $H$ and $W$ are the height and width of the original image (or feature map) in pixels
* $c$ refers to the channel (e.g. RGB)
* $(x_i^s, y_i^s)$ are the coordinates where the original image is sampled (where the image is normalized such that $-1 \le x_i^s, y_i^s\le 1$)
* $U_{nm}^c$ is defined as the pixel value at location $(n,m)$ in channel $c$.
I am having trouble interpreting the variables $n$ and $m$. Are these
* coordinates in the normalized image (i.e. $-1 \le n, m\le 1$, where you would sum $n$ from $n=-1$ to $H=1$ in steps of the normalized resolution, e.g. steps of $1/100$ for an image that is 100 px in height)
* or are these row and column values (e.g. 1021
you sum $n$ from $n=0$ to $n=100$ for an image that is 100px in height)?
I have tried out both to do downsampling of an image, but don't get consistent results.
If someone can help me out interpreting this, I would appreciate it very much.
Below I have included what I understand of bilinear interpolation. Maybe that someone can help me out based on this.
In the below figure, a single channel feature map (or image) with one channel is displayed that consists of four pixels with values $ U_{nm} $, where $ n $ and $ m $ are the coordinates of the center of the pixels, i.e. $ m,n \in {-0.5, 0.5} $. If we index $ m $ and $ n $ as $ m_k, n_k $, with $ k \in [1,4] $, we can also index the pixel values as $ U_{n_km_k} $. The values of all four pixels can be reduced to a single value $ V $ at position $ (x_i^s, y_i^s) $ by applying bilinear interpolation.
The procedure can be divided into three linear interpolations. First the value $ U_1' $ at position $ (x_{U_1'}, y_{U_1'}) $ can be computed by interpolating the values $ U_{n_1m_1} $ and $ U_{n_2m_2} $: \begin{equation} U_1' = \Delta x_2\ U_{n_1m_1} + \Delta x_1\ U_{n_2m_2}. \end{equation} As the sum of $ \Delta x_1 $ and $ \Delta x_2 $ is equal to one, due to normalization of the axes, the above equation can be rewritten as: \begin{equation} U_1' = (1-\Delta x_1) U_{n_1m_1} + (1-\Delta x_2) U_{n_2m_2}. \end{equation} The terms $ \Delta x_1 $ and $ \Delta x_2 $ can be expressed as: \begin{align} \Delta x_1 = |x_i^s - {m_1}|\ \Delta x_2 = |x_i^s - {m_2}|, \end{align} which, substituted into the equation for $U_1'$ yields: \begin{equation} U_1' = U_{n_1m_1}(1-|x_i^s - {m_1}|) + U_{n_2m_2}(1-|x_i^s - {m_2}|). 1021