title: word frequency scatterplot in R (words as labels)
body: <p>I'm currently working on a paper comparing British MPs' roles in Parliament and their roles on twitter. I have collected twitter data (most importantly, the raw text) and speeches in Parliament from one MP and wish to do a scatterplot showing which words are common in both twitter and Parliament (top right hand corner) and which ones are not (bottom left hand corner). So, x-axis is word frequency in parliament, y-axis is word frequency on twitter.</p>
  
  <p>So far, I have done all the work on this paper with R. I have ZERO experience with R, up until now I've only worked with STATA. </p>
  
  <p>I tried adapting this code (<a href="http://is-r.tumblr.com/post/37975717466/text-analysis-made-too-easy-with-the-tm-package" rel="nofollow">http://is-r.tumblr.com/post/37975717466/text-analysis-made-too-easy-with-the-tm-package</a>), but I just can't work it out. The main problem is that the person who wrote this code uses one text document and regular expressions to demarcate which text belongs on which axis. I however have two separate documents (I have saved them as .txt, corpi, or term-document-matrices) which should correspond to the separate axis.</p>
  
  <p>I'm sorry that a novice such as myself is bothering you with this, and I will devote more time this year to learning the basics of R so that I could solve this problem by myself. However, this paper is due next Monday and I simply can't do so much backtracking right now to solve the problem. </p>
  
  <p>I would be really grateful if you could help me,</p>
  
  <p>thanks very much,</p>
  
  <p>Nik</p>
  
  <p>EDIT: I'll put in the code that I've made, even though it's not quite in the right direction, but that way I can offer a proper example of what I'm dealing with. </p>
  
  <p>I have tried implementing is.R()s approach by using the text in question in a csv file, with a dummy variable to classify whether it is twitter text or speech text. i follow the approach, and at the end i even get a scatterplot, however, it plots the number ( i think it is the number at which the word is located in the dataset??) rather than the word. i think the problem might be that R is handling every line in the csv file as a seperate text document. </p>
  
  <pre><code># in excel i built a csv dataset that contains all the text, each instance (single tweet / speech) in one line, with an added dummy variable that clarifies whether the text is a tweet or a speech ("istweet", 1=twitter). 
  
  comparison_watson.df &lt;- read.csv(file="data/watson_combo.csv", stringsAsFactors = FALSE)
  
  # now to make a text corpus out of the data frame
  
  comparison_watson_corpus &lt;- Corpus(DataframeSource(comparison_watson.df))
  inspect(comparison_watson_corpus)
  
  # now to make a term-document-matrix
  
  comparison_watson_tdm &lt;-TermDocumentMatrix(comparison_watson_corpus)
  inspect(comparison_watson_tdm)
  
  comparison_watson_tdm &lt;- inspect(comparison_watson_tdm)
  sort(colSums(comparison_watson_tdm))
  table(colSums(comparison_watson_tdm))
  
  termCountFrame_watson &lt;- data.frame(Term = rownames(comparison_watson_tdm))
  termCountFrame_watson$twitter &lt;- colSums(comparison_watson_tdm[comparison_watson.df$istwitter == 1, ])
  termCountFrame_watson$speech &lt;- colSums(comparison_watson_tdm[comparison_watson.df$istwitter == 0, ])
  
  head(termCountFrame_watson)
  
  zp1 &lt;- ggplot(termCountFrame_watson)
  zp1 &lt;- zp1 + geom_text(aes(x = twitter, y = speech, label = Term))
  print(zp1)
  </code></pre>