Edge intelligence especially binary neural network (BNN) has attracted considerable attention of the artificial intelligence community recently. BNNs significantly reduce the computational cost, model size, and memory footprint. However, there is still a performance gap between the successful full-precision neural network with ReLU activation and BNNs. We argue that the accuracy drop of BNNs is due to their geometry. We analyze the behaviour of the full-precision neural network with ReLU activation and compare it with its binarized counterpart. This comparison suggests random bias initialization as a remedy to activation saturation in full-precision networks and leads us towards an improved BNN training. Our numerical experiments confirm our geometric intuition.
Summary: This paper tries to improve the training for the binary neural network.
Weaknesses:
[-] A lack of related works. There have been many related works about BNN in these years (after 2017), but the authors do not have a quick summary of them.
[-] More reference. e.g, when authors mention 'many related works require to store the full-precision activation map during the inference stage',  some reference is necessary.
[-] Weak Motivation: The authors argue 'We analyze the behaviour of the full-precision neural network with ReLU activation' in the abstract. However, in Section 3, I cannot find any analysis. Only writing down the backward and forward cannot be called analysis. Initialization is different from the training dynamics. Assumptions and theorems should be highlighted. 
[-] Poor writing: A lot of typos. Only in the last paragraph in Section 2, I find many typos,  e.g. 'replaced replacing ReLU activation', 'any relaated works'.
Questions:
[.] In experiments, what structure is used for ResNet? ResNet-18-like or ResNet-110-like? (The results for these two kinds of structure are totally different for binary neural network, as the difference in the number of channels) 
[.] In experiments, the performance of the baselines seems lower than related papers? Do the authors increase the number of channels in each layer as the other people do? It can improve the result a lot, and I wonder whether the improvement still exists in this setting.
[.] In experiments, only CIFAR10 results have been reported, but I wonder what is the error bar looks like? (Do the authors run the experiments several times and calculate the variance?)