Thanks, but I think this would not be an ideal work around, because if I have ~100 output units, I would have to do 100 separate (matrix * sharedvector + bias_i) operations. So, I thought that instead of doing that I want to have a weight matrix with sharedvector as columns so that the GPU can do the regular matrix-matrix multiplication.