Answers.Body: <p>According to a nice article by Sebastian Ruder <a href="https://ruder.io/4-biggest-open-problems-in-nlp/" rel="nofollow noreferrer">https://ruder.io/4-biggest-open-problems-in-nlp/</a> based on answers from top NLP researchers <a href="https://docs.google.com/document/d/18NoNdArdzDLJFQGBMVMsQ-iLOowP1XXDaSVRmYN0IyM/edit" rel="nofollow noreferrer">https://docs.google.com/document/d/18NoNdArdzDLJFQGBMVMsQ-iLOowP1XXDaSVRmYN0IyM/edit</a></p>
  
  <ol>
  <li>Natural language understanding</li>
  <li>NLP for low-resource scenarios</li>
  <li>Reasoning about large or multiple documents</li>
  <li>Datasets, problems, and evaluation</li>
  </ol>
  
  <p>I recommend having a look at the article. More details in the slides <a href="https://drive.google.com/file/d/15ehMIJ7wY9A7RSmyJPNmrBMuC7se0PMP/view" rel="nofollow noreferrer">https://drive.google.com/file/d/15ehMIJ7wY9A7RSmyJPNmrBMuC7se0PMP/view</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-12-04T13:05:48.363Z
Answers.LastActivityDate: 2019-12-04T13:05:48.363Z
Answers.UserDisplayName: cookiemonster
Answers.Body: <p>According to <a href="https://web.stanford.edu/~jurafsky/" rel="nofollow noreferrer">Dan Jurafsky</a>, a researcher on NLP and NLU, the current hard problems in NLP are (see <a href="http://spark-public.s3.amazonaws.com/nlp/slides/intro.pdf" rel="nofollow noreferrer">slide 6</a>)</p>
  
  <ul>
  <li>Questioning answering</li>
  <li>Paraphrase</li>
  <li>Summarisation</li>
  <li>Dialogue</li>
  </ul>
  
  <p>Other hard problems for which there are already some good solutions are</p>
  
  <ul>
  <li>Sentiment analysis</li>
  <li>Coreference resolution</li>
  <li>Word sense disambiguation</li>
  <li>Parsing</li>
  <li>Machine translation</li>
  <li>Information extraction</li>
  </ul>
  
  <p>Ambiguity in natural language is one of the biggest challenges for NLP and NLU systems. Other challenges are (see <a href="http://spark-public.s3.amazonaws.com/nlp/slides/intro.pdf" rel="nofollow noreferrer">slide 10</a>)</p>
  
  <ul>
  <li>non-standard words</li>
  <li>idioms</li>
  <li>tricky entity names</li>
  <li>neologisms </li>
  <li>world knowledge </li>
  <li>segmentation issues</li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-14T18:12:56.040Z
Answers.LastActivityDate: 2020-03-14T18:12:56.040Z
Answers.UserDisplayName: nbro
Body: <p>I'm doing a paper for a class on the topic of big problems that are still prevalent in AI, specifically in the area of natural language processing and understanding. From what I understand, the areas:</p>
  
  <ul>
  <li>Text classification</li>
  <li>Entity recognition </li>
  <li>Translation</li>
  <li>POS tagging</li>
  </ul>
  
  <p>are for the most part solved or perform at a high level currently, but areas such as:</p>
  
  <ul>
  <li>Text summarization</li>
  <li>Conversational systems</li>
  <li>Contextual systems (relying on the previous context that will impact current prediction)</li>
  </ul>
  
  <p>are still relatively unsolved or are a big area of research (although this could very well change soon with the releases of big transformer models from what I've read).</p>
  
  <p>For people who have experience in the field, what are areas that are still big challenges in NLP and NLU? Why are these areas (doesn't have to be ones I've listed) so tough to figure out?</p>
  
Tags: <natural-language-processing><natural-language-understanding>
Title: What are the current big challenges in natural language processing and understanding?
UserDisplayName: Landon G
ThreadText: # What are the current big challenges in natural language processing and understanding?
  
  (Asked by: username_0 on 2019-12-02)
  
  I'm doing a paper for a class on the topic of big problems that are still prevalent in AI, specifically in the area of natural language processing and understanding. From what I understand, the areas:
  
  * Text classification
  * Entity recognition
  * Translation
  * POS tagging
  
  are for the most part solved or perform at a high level currently, but areas such as:
  
  * Text summarization
  * Conversational systems
  * Contextual systems (relying on the previous context that will impact current prediction)
  
  are still relatively unsolved or are a big area of research (although this could very well change soon with the releases of big transformer models from what I've read).
  
  For people who have experience in the field, what are areas that are still big challenges in NLP and NLU? Why are these areas (doesn't have to be ones I've listed) so tough to figure out?
  
  **Tags:** natural-language-processing, natural-language-understanding
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2019-12-04. Score: 3
  
  According to a nice article by Sebastian Ruder <https://ruder.io/4-biggest-open-problems-in-nlp/> based on answers from top NLP researchers <https://docs.google.com/document/d/18NoNdArdzDLJFQGBMVMsQ-iLOowP1XXDaSVRmYN0IyM/edit>
  
  1. Natural language understanding
  2. NLP for low-resource scenarios
  3. Reasoning about large or multiple documents
  4. Datasets, problems, and evaluation
  
  I recommend having a look at the article. More details in the slides <https://drive.google.com/file/d/15ehMIJ7wY9A7RSmyJPNmrBMuC7se0PMP/view>
  
  
  ---
  
  ### Answer by username_2 on 2020-03-14. Score: 1
  
  According to [Dan Jurafsky](https://web.stanford.edu/~jurafsky/), a researcher on NLP and NLU, the current hard problems in NLP are (see [slide 6](http://spark-public.s3.amazonaws.com/nlp/slides/intro.pdf))
  
  * Questioning answering
  * Paraphrase
  * Summarisation
  * Dialogue
  
  Other hard problems for which there are already some good solutions are
  
  * Sentiment analysis
  * Coreference resolution
  * Word sense disambiguation
  * Parsing
  * Machine translation
  * Information extraction
  
  Ambiguity in natural language is one of the biggest challenges for NLP and NLU systems. Other challenges are (see [slide 10](http://spark-public.s3.amazonaws.com/nlp/slides/intro.pdf))
  
  * non-standard words
  * idioms
  * tricky entity names
  * neologisms
  * world knowledge
  * segmentation issues
  
  
  ---
  
  