Autopsy of a deep learning paper
Cmon, wtf? Some of the criticisms here just aren't even close to valid. He spends a half of the blog post criticizing them spending 100 GPUs on the Imagenet classification experiment,> So they trained it using 100 GPUs (100 GPUs dear lord!), and got no difference until fourth decimal digit! 100 GPU's to get a difference on fourth decimal digit! I think somebody at Google of Facebook should reproduce this result using 10000 GPU's, perhaps they will get a difference at a third decimal digit. Or maybe not, but whatever, those GPU's need to do something right?Wow. This is just a blatant mischaracterization of what's going on. First of all, this result is in the appendix. It's not meant to be an important result of the paper. In the appendix, they explicitly write:>Of all vision tasks, we might expect image classification to show the least performance change when
using CoordConv instead of convolution, as classification is more about what is in the image than
where it is. This tiny amount of improvement validates that.In contrast, they compare against object detection (in which the spatial location matters), and get substantially better results.This is just a standard "negative" result, to validate the fact that what they think is happening is actually happening empirically.The fact that this blog post mocks them for that, and much of HN is laughing along with the blog is seriously disappointing.