TreeNets are also pretty popular. Word embeddings are also a form of weight tying, and can&#x27;t always be described as part of an RNN. If you want, you can think of most attentional mechanisms as involving a form of weight tying, in addition to the RNN tying.<p>There are also more obscure things, like deep symmetry networks (where are really a close relative of conv nets, with convolution replaced by group convolution).