Machine Learning, Artificial Intelligence, Deep Learning, Sap, Research.
attention masking identifies the network segments that are essential for memorising currently learned information and prevents their updates during the future learning. The network is further incentivised to reuse previously learned knowledge, which was stored in such “reserved” network segments