I need to implement the following function:
out(N_i, C_{out_j}) = bias(C_{out_j}) +
\sum_{k = 0}^{C_{in} - 1} f(weight(C_{out_j}, k) \star input(N_i, k))
Where f is a simple elementwise non-linearity such relu function, applied before summation.
I can easily do this via setting group=C_in and doing it one-by-one for each output channel. However, this gives memory error for large C_in. Is there a more efficient way to do this?