title: What is the pythonic way of downloading hdf5 files from http server?
body: <p>I am trying to download a hdf5 file from http server. I can do this with python subprocess module and wget, but I feel like I am cheating</p>
  
  <pre><code>    # wget solution
      import subprocess
      url = 'http://url/to/file.h5' 
      subprocess(['wget', '--proxy=off', url])
  </code></pre>
  
  <p>I can also use the urllib and request modules for downloading images like this:</p>
  
  <pre><code>    # requests solution
      url2 = 'http://url/to/image.png'
      r = requests.get(url2)
      with open('image.png', 'wb') as img:
      img.write(r.content)
  
      # urllib solution
      urllib.urlretrieve(url2, 'outfile.png')
  </code></pre>
  
  <p>However, when I try to download hdf5-file with this method and run shell command 'file' I get:</p>
  
  <pre><code>    &gt;file test.h5 
      &gt;test.h5: HTML document, ASCII text, with very long lines
  </code></pre>
  
  <p>Here is the header from requests.get() (not sure if it helps)</p>
  
  <pre><code>    {'accept-ranges': 'bytes',
      'content-length': '413399',
      'date': 'Tue, 19 Feb 2013 08:51:06 GMT',
      'etag': 'W/"413399-1361177055000"',
      'last-modified': 'Mon, 18 Feb 2013 08:44:15 GMT',
      'server': 'Apache-Coyote/1.1'}
  </code></pre>
  
  <p><strong>Should I just use wget throug subprocess or is there a pythonic solution?</strong></p>
  
  <p><strong>Solution:</strong>
  The problem was caused by the fact that I didn't disable the proxy before I tried to download the file and because of that, the transfer was intercepted. This piece of code did the trick.</p>
  
  <pre><code>    import urllib2
      proxy_handler = urllib2.ProxyHandler({})
      opener = urllib2.build_opener(proxy_handler)
      urllib2.install_opener(opener)
  
      url = 'http://url/to/file.h5'
  
      req = urllib2.Request(url)
      r = opener.open(req)
      result = r.read()
  
      with open('my_file.h5', 'wb') as f:
          f.write(result)
  </code></pre>