Hi,
Is there any reason why you do not use builtin functions?
I guess you could do something along the lines of:
import torch.nn.functional as F

def batch_matmul_bias(seq, weight, bias, nonlinearity=''):
  s = F.linear(seq, weight, bias)
  if nonlinearity=='tanh':
    s = F.tanh(s)
  return s