Thanks Piotr, you reminded me that dropout should have been causing this weird behavior.  But it is still quite intriguing to see the link prediction task going so well without dropout while the embeddings are updated so badly.  Such an interesting behavior…