maaft:

So you would also do something like my SampleConv approach from my first post i.e. selecting certain kernels?


In your code you are slicing the input channels, i.e. you are making the kernels “smaller”:
self.weight[:, sampled_channels < self.channel_sample_ratio]

The weight parameter is defined as [out_channels, in_channels, height, width].



 maaft:

Would it make sense to also select only output channels?


I don’t understand the posted section as removing kernels, but the input channels and bypassing the removed channels (i.e. not applying the convolution on them). But I’m still unsure. 



 maaft:

Unfortunately I still can’t wrap my head around their claimed 75% decrease in memory consumption.


That’s also, why I’m thinking, that I misunderstand the section, so let’s better wait for smarter users who could help us out.