The invention discloses an attention-enhanced fully-relevant variational auto-encoder for segmenting grammar and semantics, and relates to artificial intelligence, in particular to the field of NLP. Described herein are implementations of a framework called a full-correlation variational auto-encoder (TC_VAE) that unwraps the syntax and semantics by exploiting the full-correlation penalty of KL differences. Decomposing one or more Kulbeck-Rynolder (KL) divergent terms in the loss of the variational autoencoder so that the resulting hidden variables can be separated. Implementations of the TC_VAE framework are used for inspection of semantic similarity tasks and syntactic similarity tasks. Experimental results show that a better unwrapping between the syntactic representation and the semantic representation is achieved on the same dataset under similar settings compared to the state-of-the-art (SOTA).
Attention-enhanced fully-relevant variational auto-encoder for segmenting grammar and semantics