No its a combination of convolutions, pools and fully connected layers so fairly straightforward, but its likely to get weirder once we get the base working and we try new things. The issue is this:
My benchmarks:
slowpen
1000 loops, best of 3: 1.88 ms per loop
100 loops, best of 3: 1.82 ms per loop
fastpen
1000 loops, best of 3: 258 µs per loop
1000 loops, best of 3: 443 µs per loop
create
10000 loops, best of 3: 132 µs per loop
So currently I’m currently creating a list of views and then using the stack command to combine them and take a snapshot each time i want to calculate the penalty. The speed for the forward/backward steps is listed above. Its already about 7 times faster than the for loop method and this only gets worse as the network gets deeper.
More importantly, that cat step takes like half the calculation time (‘create’) for the forward pass.
Further, this is only for a <10 layer network, the time only gets worse when I added more layers (making them wider doesn’t make a huge difference though).
All I need is a way to combine 2 view objects into a new view object without breaking their respective links. If that command exists thats all I need but thus far haven’t been able to find it (if it exists). I don’t need to copy the whole buffer, just do a series of straightforward calculations on it like various norms or means.