Hey everyone!
amateur data scientist here, have worked with nltk/spacy/gensim insofar as to do soft cosine similarity of text documents in the past. I’m working on a new project, where I’m trying to identify questions and answers in a 6 month chat history where there are multiple participants and multiple threads of conversation (some linked some not).
I’ve gotten question identification somewhat down, but am struggling to figure out a good approach to identifying answers. One approach I’ve thought about is through entity recognition clustering, though that seems a little too narrow/rigid.
I saw this adversarial_qa · Datasets at Hugging Face 3 and was wondering if it might be applicable to my use case? If so, any suggestions on how to transfer learning train it on the domain of my corpus (which is mostly code and niche topic related text)?
Thanks in advance all