<p dir="auto">This is fastapi http side:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="       from fastapi import Request
       import aiofiles

       @app.post('/upload')
        async def upload(request: Request):
            try:
                filename = request.headers['filename']
                async with aiofiles.open(filename, 'wb') as f:
                    async for chunk in request.stream():
                        await f.write(chunk)
            except Exception:
                return {&quot;message&quot;: &quot;There was an error uploading the file&quot;}
         
            return {&quot;message&quot;: f&quot;Successfuly uploaded {filename}&quot;}"><pre class="notranslate"><code class="notranslate">       from fastapi import Request
       import aiofiles

       @app.post('/upload')
        async def upload(request: Request):
            try:
                filename = request.headers['filename']
                async with aiofiles.open(filename, 'wb') as f:
                    async for chunk in request.stream():
                        await f.write(chunk)
            except Exception:
                return {"message": "There was an error uploading the file"}
         
            return {"message": f"Successfuly uploaded {filename}"}
</code></pre></div>
<p dir="auto">this is the request side:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="    def gen_file_stream():
        with open(file_name, &quot;rb&quot;) as f:
            while True:
                data = f.read(1024*1024)
                if not data:
                    break
                yield data
          resp=requests.post(url,data=gen_file_stream())"><pre class="notranslate"><code class="notranslate">    def gen_file_stream():
        with open(file_name, "rb") as f:
            while True:
                data = f.read(1024*1024)
                if not data:
                    break
                yield data
          resp=requests.post(url,data=gen_file_stream())
</code></pre></div>
<p dir="auto">when I check the size of the chunk its 64k and not 1MB (as Im trying to send here).<br>
If I use TestClient from fastapi it is working, I mean Im able to send a chunk of 1MB, so it seems to me requests package is dividing the size of the chunk internally to 64KB.<br>
is there any way I can make request.post to send a chunk of 1MB?</p>
<p dir="auto">In a rare instance, I managed to do a GET request to a server that should retrieve an xml document. However the response.text field contains the document twice (just concatenated, i.e. the first half of the data and second half of the data are exact duplicates):</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="&gt;&gt;&gt; resp = requests.get('http://bigr-rad-xnat.erasmusmc.nl/schemas/xnat/xnat.xsd', auth=('username', 'password'))
&gt;&gt;&gt; resp.text[:len(resp.text)/2] == resp.text[len(resp.text)/2:]
True"><pre class="notranslate"><span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">resp</span> <span class="pl-c1">=</span> <span class="pl-s1">requests</span>.<span class="pl-en">get</span>(<span class="pl-s">'http://bigr-rad-xnat.erasmusmc.nl/schemas/xnat/xnat.xsd'</span>, <span class="pl-s1">auth</span><span class="pl-c1">=</span>(<span class="pl-s">'username'</span>, <span class="pl-s">'password'</span>))
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">resp</span>.<span class="pl-s1">text</span>[:<span class="pl-en">len</span>(<span class="pl-s1">resp</span>.<span class="pl-s1">text</span>)<span class="pl-c1">/</span><span class="pl-c1">2</span>] <span class="pl-c1">==</span> <span class="pl-s1">resp</span>.<span class="pl-s1">text</span>[<span class="pl-en">len</span>(<span class="pl-s1">resp</span>.<span class="pl-s1">text</span>)<span class="pl-c1">/</span><span class="pl-c1">2</span>:]
<span class="pl-c1">True</span></pre></div>
<p dir="auto">Now of course, it could happen that the document really is that, but it shouldn't and I tried the request using curl, postman (chromium extension), and chromium and it all returns just half the response of requests. As a final test I tried it using urllib2:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="&gt;&gt;&gt; import urllib2 base64
&gt;&gt;&gt; req = urllib2.Request('http://bigr-rad-xnat.erasmusmc.nl/schemas/xnat/xnat.xsd')
&gt;&gt;&gt; base64string = base64.encodestring(
        '%s:%s' % ('password', 'password'))[:-1]
        authheader =  &quot;Basic %s&quot; % base64string
        req.add_header(&quot;Authorization&quot;, authheader)
&gt;&gt;&gt; handler = urllib2.urlopen(req)
&gt;&gt;&gt; content = handler.read()"><pre class="notranslate"><span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-k">import</span> <span class="pl-s1">urllib2</span> <span class="pl-s1">base64</span>
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">req</span> <span class="pl-c1">=</span> <span class="pl-s1">urllib2</span>.<span class="pl-v">Request</span>(<span class="pl-s">'http://bigr-rad-xnat.erasmusmc.nl/schemas/xnat/xnat.xsd'</span>)
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">base64string</span> <span class="pl-c1">=</span> <span class="pl-s1">base64</span>.<span class="pl-en">encodestring</span>(
        <span class="pl-s">'%s:%s'</span> <span class="pl-c1">%</span> (<span class="pl-s">'password'</span>, <span class="pl-s">'password'</span>))[:<span class="pl-c1">-</span><span class="pl-c1">1</span>]
        <span class="pl-s1">authheader</span> <span class="pl-c1">=</span>  <span class="pl-s">"Basic %s"</span> <span class="pl-c1">%</span> <span class="pl-s1">base64string</span>
        <span class="pl-s1">req</span>.<span class="pl-en">add_header</span>(<span class="pl-s">"Authorization"</span>, <span class="pl-s1">authheader</span>)
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">handler</span> <span class="pl-c1">=</span> <span class="pl-s1">urllib2</span>.<span class="pl-en">urlopen</span>(<span class="pl-s1">req</span>)
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">content</span> <span class="pl-c1">=</span> <span class="pl-s1">handler</span>.<span class="pl-en">read</span>()</pre></div>
<p dir="auto">And that also gave me a single copy of the document. Hence I conclude something is probably going wrong in requests.</p>
<p dir="auto">I am using requests version 2.9.1 on python 2.7.11 (and I also tried on 3.5.1) on debian stretch. The server is a tomcat server running XNAT (<a href="http://www.xnat.org" rel="nofollow">www.xnat.org</a>). The problem is that the server serving this is located behind a hospital firewall and therefor I cannot give anyone from outside access to reproduce/test, but I can gather additional information.</p>
<p dir="auto">I tried to look at the open bugs and did not find a similar report, but I could have missed it, if so I am sorry.</p>