Hi Yuhang
I think you can add a if statement in the forward() function. something like:
x1 = path1(...)
x2 = path2(...)
if voice:
    x = torch.cat(x1, x2)
else:
    x = torch.cat(x1, Variable(torch.zeros(...,...)))