{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Deep learning for Sentiment Analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Motivation\n",
    "\n",
    "*Sentiment analysis* is a field of study that covers that analyzes people's opinions, appraisals, attitudes, and emotions towards entities such as products, services, organizations, individuals, issues, events, topics, and their attributes. \n",
    "\n",
    "Although linguistics and natural language processing (NLP) have a long history, little research had been done about people's *opinions* and *sentiments*. Most current techniques (example, search engines work with facts (example, knowledge graph) rather than working with opinions. The advent of social media and availability of huge volumes of opinionated data in these media have caused a resurgence in research in this field. \n",
    "\n",
    "*Opinions* are <u>key influencers</u> of human behaviors. Businesses and organizations always want to find consumer or public opinions about their products and services. Individual consumers also want to know the opinions of existing users of a product before purchasing it, and peer opinions about political candidates before making a voting decision in a political election. An example of an application would be tracking user opinions about companies to predict their stock prices or box office revenues for a movie.\n",
    "\n",
    "#### Problem Description\n",
    "\n",
    "This capstone project <u>intends to explore recent advances in Natural Language Processing to improve accuracy of sentiment classification</u>. The dataset used for the project would be the Rotten Tomatoes movie reviews dataset. The Rotten Tomatoes movie review dataset is a corpus of movie reviews used for sentiment analysis, originally collected by Pang and Lee [1]. In their work on sentiment treebanks, Socher et al. [2] used the Stanford parser [3] to create fine-grained labels for all parsed phrases in the corpus annotated by three human judges. The Sentiment Treebank along with the sentiment classification can be viewed at [5].\n",
    "\n",
    "This is a **classification problem** where review phrases are labelled on a scale of five values: *negative, somewhat negative, neutral, somewhat positive, positive*. The goal of the project would be to <u>accurately classify the sentiment of a any movie review sentence</u>.\n",
    "\n",
    "In general, sentiment analysis is investigated at different granularities, namely, *document, sentence* or *entity* levels. The review text usually comprises of a single sentence in this dataset, and the data does not contain which movie the review talked about. So, <u>the scope of the project is only restricted to sentence level analysis without entity recognition as such</u>. However, the text might be specific to some aspect of the movie such as its screenplay. For example, \n",
    "\n",
    "> *“The acting was mediocre, but the screenplay was top quality”.* \n",
    "\n",
    "In this case, the screenplay had very positive review (emphasized) while the acting had a very negative review (not emphasized). <u>Aspects make sentiment analysis very domain specific</u>. In general, every domain tends to have a specific vocabulary that cannot be used for other domains. <u>The project scope is only restricted to movie review sentences</u>.\n",
    "\n",
    "Sentiment words themselves share a *common lexicon*. Example, would be positive words such as great, excellent or amazing and negative words such as bad, mediocre or terrible. Numerous efforts have been made to build such lexicons, which are not sufficient as the problem is more complex. \n",
    "\n",
    "Some of the obstacles are:\n",
    "* **Language ambiguity**: Example, \n",
    "\n",
    "> “The movie blows” vs “The movie blows away all expectations”. \n",
    "\n",
    "    The word “blows” has negative orientation in one and positive in other.\n",
    "\n",
    "* **Contrapositive conjunction**: The review \n",
    "\n",
    "> “The acting sucked but the movie was entertaining”. \n",
    "\n",
    "    Here the review is of the form “X but Y”. The goal of the project would be to classify each phrase, X and Y accurately and then determine the overall sentiment for the movie, positive in this case.\n",
    "\n",
    "* **Sentence negation**: There are two kinds of examples here,\n",
    "** Negative positives: \n",
    "\n",
    "> “The movie was not very great”. \n",
    "\n",
    "    Here the phrase “very great” is positive but “not” changes the sentiment of the review.\n",
    "\n",
    "** Negative negatives: \n",
    "\n",
    "> “The movie was not that terrible”. \n",
    "\n",
    "    Here the phrase “terrible” is negative, but “not” does not make it positive.\n",
    "\n",
    "* **Sarcasm**: Sarcastic comments are hardest to detect and deal with. Example, \n",
    "\n",
    "> “I was never this excited to leave the theater”. \n",
    "\n",
    "    This is a very negative comment, but very hard to classify.\n",
    "\n",
    "* **Sentences with no opinions**: These are usually questions such as \n",
    "\n",
    "> “Has any seen this movie?” \n",
    "\n",
    "    or conditionals \n",
    "\n",
    "> “If the acting is good, I will go and see the movie.” \n",
    "\n",
    "    Both are neutral sentences. However not all questions or conditionals are neutral, example \n",
    "\n",
    "> “Has anyone else seen this terrible movie?” or \n",
    "\n",
    "> “If you looking for good acting, go and see this movie”.\n",
    "\n",
    "* **Sentences with no sentiment words but with opinions**: Example, \n",
    "\n",
    "> “The movie solved my insomnia”. \n",
    "\n",
    "    This is a very negative review without a sentiment word such as good or bad.\n",
    "\n",
    "<u>This project will evaluate the models only on first three of the issues mentioned above.</u> Language ambiguity is mitigated considerably by restricting scope only to movie reviews. The other two issues, contrapositive conjunction and sentence negation are mitigated by constructing parse trees of the text and using compositionality functions trained over known examples, which is what mostly the bulk of this project is about. The sarcastic comments and the sentences with no opinions will be not be specially handled."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Methodology\n",
    "\n",
    "The goal of the project is to build a sentiment classification system using deep learning, namely Recursive Tensor Neural Network (RNTN). This method uses tensors to remove dependency on the vocabulary and captures different types of associations in the RNN. \n",
    "\n",
    "The main components of the project would be:\n",
    "* **Training Engine**: Train the RNTN model with the training data set.\n",
    "* **Parser**: Parse the trees and extract sentiment labels.\n",
    "* **Prediction App**: A Web Application to view the predictions of any movie review.\n",
    "* **Stanford CoreNLP**: The project will reuse Stanford CoreNLP to do constituency parsing of the sentence for which prediction needs to be made.\n",
    "\n",
    "#### Data Set Description\n",
    "\n",
    "The project uses the data set from the original paper as contains fully parsed trees and sentiment labels. The train, test and dev data already split and parsed using the standard parser is exposed at\n",
    "https://nlp.stanford.edu/sentiment/trainDevTestTrees_PTB.zip\n",
    "\n",
    "In addition, the original data set that the paper [2] uses the following data:\n",
    "http://nlp.stanford.edu/sentiment/stanfordSentimentTreebank.zip\n",
    "http://nlp.stanford.edu/sentiment/stanfordSentimentTreebankRaw.zip\n",
    "\n",
    "The data contains raw scores in range (1 to 25) which are mapped to (1 to 5) range for both complete sentences and parsed sub phrases. \n",
    "\n",
    "#### References\n",
    "\n",
    "[1] Pang and L. Lee. 2005. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In ACL, pages 115–124.\n",
    "\n",
    "[2] Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank, Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Chris Manning, Andrew Ng and Chris Potts. Conference on Empirical Methods in Natural Language Processing (EMNLP 2013).\n",
    "\n",
    "[3] D. Klein and C. D. Manning. 2003. Accurate unlexicalized parsing. In ACL\n",
    "\n",
    "[4] https://nlp.stanford.edu/sentiment/\n",
    "\n",
    "[5] https://nlp.stanford.edu/sentiment/treebank.html\n",
    "\n",
    "[6] Sentiment Analysis and Opinion Mining, Bing Liu.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
