Have a look at this link 110.
It’s an iterative orthogonalization procedure which you have to call iteratively until an acted upon linear layer converges to orthogonality. If you are wondering about their implementation, self.params.map_beta is just a scalar hyperparameter which defaults to 0.001.
A bit simplified, their procedure looks as follows:
beta = 0.001

def orthogonalize(self):
        """
        Orthogonalize the mapping.
        """
        W = self.linear_layer.weight.data
        W.copy_((1 + beta) * W - beta * W.mm(W.transpose(0, 1).mm(W)))