title: Encoding of requirement file using pip
body: <p>On OSX (El Capitan, 10.11.6), using virtualenv (15.1.0), I am getting an error when installing requirements from a text file with pip (9.0.1):</p>
  
  <pre><code>virtualenv env
  source env/bin/activate
  pip install -r requirements.txt
  </code></pre>
  
  <p>but not when looping over each requirement manually:</p>
  
  <pre><code>for r in $(cat requirements.txt); do pip install "$r"; done
  </code></pre>
  
  <p>This makes me think there might be an issue with the default encoding assumed by pip when reading the requirements file. <strong>Is there a way (environment variable, I presume) to set the default encoding of requirement files?</strong></p>
  
  <hr>
  
  <p>The error I get is:</p>
  
  <pre><code>Exception:
  Traceback (most recent call last):
    File "/path/to/env/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
      status = self.run(options, args)
    File "/path/to/env/lib/python2.7/site-packages/pip/commands/install.py", line 312, in run
      wheel_cache
    File "/path/to/env/lib/python2.7/site-packages/pip/basecommand.py", line 295, in populate_requirement_set
      wheel_cache=wheel_cache):
    File "/path/to/env/lib/python2.7/site-packages/pip/req/req_file.py", line 84, in parse_requirements
      filename, comes_from=comes_from, session=session
    File "/path/to/env/lib/python2.7/site-packages/pip/download.py", line 422, in get_file_content
      content = auto_decode(f.read())
    File "/path/to/env/lib/python2.7/site-packages/pip/utils/encoding.py", line 31, in auto_decode
      return data.decode(locale.getpreferredencoding(False))
  LookupError: unknown encoding:
  </code></pre>
  
  <hr>
  
  <p>The following test code:</p>
  
  <pre><code>#!/usr/bin/env python
  
  import sys
  import locale
  
  print sys.stdin.encoding
  print locale.getpreferredencoding()
  print locale.getpreferredencoding(False)
  print sys.getdefaultencoding()
  print sys.getfilesystemencoding()
  </code></pre>
  
  <p>returns:</p>
  
  <pre><code>None
  
  US-ASCII
  ascii
  utf-8
  </code></pre>
  
  <hr>
  
  <p>From the command-line:</p>
  
  <pre><code>$ locale
  LANG="en_GB.UTF-8"
  LC_COLLATE="en_GB.UTF-8"
  LC_CTYPE="utf-8"
  LC_MESSAGES="en_GB.UTF-8"
  LC_MONETARY="en_GB.UTF-8"
  LC_NUMERIC="en_GB.UTF-8"
  LC_TIME="en_GB.UTF-8"
  LC_ALL=
  </code></pre>