Hi @lewtun,
Wanted to report back, did a lot of reading starting with the Universal Sentence Encoder (which I’d foolishly neglected in my previous passes over the literature). It looked like a great starting point but I was really looking for something like SciBERT that had the vocab needed to capture some of the more detailed parts of the data.
Landed upon DeCLUTR (git 14 and paper 4) and it looks like we are onto a winner!


Many thanks for the input,
Chris