title: Sending requests to webpages using Python's urllib2
body: <p>I am interested in using Python to automate certain tasks.  Specifically, I would like to use Python to interact with website to perform tasks such as getting specific information from a page, make request (POST data and read the response), and downloading and uploading files.
  So far, I have only been able to use Python to get the HTML from a page using urllib2.  The next thing I tried was sending a request to a page; I made several attempts, but they all failed.</p>
  
  <pre><code>    &gt;&gt;&gt; import urllib2
      &gt;&gt;&gt; import urllib
      &gt;&gt;&gt; url = "http://www.stackoverflow.com/"
      &gt;&gt;&gt; values = {}
      &gt;&gt;&gt; values["input"] = "foo"
      &gt;&gt;&gt; data = urllib.urlencode(values)
      &gt;&gt;&gt; request = urllib2.Request(url + "search/", data)
      &gt;&gt;&gt; response = urllib2.urlopen(request)
      &gt;&gt;&gt; html = response.read()
      &gt;&gt;&gt; print html
  </code></pre>
  
  <p>The way I understand things so far is that I need to create a dictionary with the names of the fields and the input and encode it with urllib.urllencode(values).  Then I need to make a request with urllib2.Request(theUrlReceivingTheRequest, data, headers) which, if only given a url will only GET, but, if given data, will POST, and can be given headers that can disguise the program as a common browser such as Firefox or IE.  I then get a response with urllib2.urlopen(request) which returns a file like object which, consequently, I can read().  As I understand it, I can also use urllib2.build_opener() which can receive handlers (that can process cookies, redirrections, authentication etc) and add headers using .addheaders("User-Agent", "").  I'd like to be able to eventually do (and understand) all of these things, but, first, I'd just like to get a form submitted.
  In the above code from my interactive session with Python, did I follow the correct procedure?  (I was attempting to input a search for "foo" in the search field on the front page of stackoverflow.)</p>