Machine Learning, Artificial Intelligence, Neural Networks, Data Science, Towards Data Science.
initialization strategies, and hence heuristics are often used to select a reasonable set of starting weights from which the network is trained. For a fully connected layer with m inputs and n outputs, we can sample from the following uniform distribution to obtain our initial weights: