Broadcasting? Or alternative solutions
Hi, I really like the library so far. However, I was wondering if broadcasting is on the roadmap, and what would be your current suggestion for work-arounds? <SCODE>x = Variable(torch.Tensor([[1.0, 1.0], 
                           [2.0, 2.0], 
                           [3.0, 3.0], 
                           [4.0, 4.0], 
                           [5.0, 5.0], 
                           [6.0, 6.0]]))

weights = Variable(torch.zeros(2, 1))
bias = Variable(torch.zeros(1))

net_input = x.mm(weights) + bias
<ECODE> A workaround would be to add 1s to the input tensor, I guess: <SCODE>x = Variable(torch.Tensor([[1.0, 1.0, 1.0], 
                           [1.0, 2.0, 2.0], 
                           [1.0, 3.0, 3.0], 
                           [1.0, 4.0, 4.0], 
                           [1.0, 5.0, 5.0], 
                           [1.0, 6.0, 6.0]]))

weights = Variable(torch.zeros(3, 1))

net_input = x.mm(weights)
<ECODE> What would be your thoughts on that?
rasbt
Adding broadcasting to most operations is definitely on our roadmap, and will be hopefully ready quite soon. Since it’s so often asked for we’ll probably reprioritize that and have it implemented soonish. For now there are two solutions - one that works already, another one that will be implemented very soon (presented in the same order): You can do broadcasting by manually adding singleton dimensions and expanding along them. This doesn’t do any memory copy, and only does some stride tricks: <SCODE>net_input = x.mm(weights)
net_input += bias.unsqueeze(0).expand_as(net_input)
<ECODE> <SCODE>import torch.nn.functional as F
output = F.linear(input, weights, bias) # bias is optional
<ECODE>
apaszke
rasbt
ecolss
<SCODE>x = torch.randn(1)
y = torch.randn(2, 3, 4, 5, 6)
print(y + x.expand_as(y))
<ECODE>
apaszke
jphoward
apaszke
Let me know if you need more info - I’m not sure I’ve done a great job of explaining!
jphoward
apaszke