I’ve been looking into this technique: Scalable and Sustainable Deep Learning via Randomized Hashing 18 that allegedly “uses only 5% of the total multiplications, while
keeping on average within 1% of the accuracy of the original model.” It looks pretty straightforward and appears to be suitable for PyTorch, so I’d like some guidance on how we might go about implementing this functionality unless it would have undesirable consequences that are not immediately apparent to me.