title: Flask Request passing pandas to_json versus curl or Postman
body: <p>I am calling a flask app that is hosted with AWS Elastic Beanstalk. The app takes in json, creates a pandas data frame from it, does some other processing and passes data into a model. The predictions are returned.</p>
  
  <p>If I run the following, the code executes and returns back the proper response.</p>
  
  <pre><code>import requests
  v=pandas_df.iloc[0:5].to_json(orient='records')
  headers = {'content-type': 'application/json'}
  r = requests.post('http://XXXXXXXXX.us-east-1.elasticbeanstalk.com/', json=v, headers = headers)
  r.json() #the predictions
  </code></pre>
  
  <p>However I can not get the same result using curl or postman. </p>
  
  <p>For curl I have tried to send the json object created by pandas:</p>
  
  <pre><code>pandas_df.iloc[0:5].to_json(orient='records',path_or_buf=jsonexp.json)
  
  curl -X POST "http://XXXXXXXXX.us-east-1.elasticbeanstalk.com/" -H "Content-Type: application/json" -d "jsonexp.json"
  </code></pre>
  
  <p>but the result is </p>
  
  <pre><code>&lt;!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"&gt;
  &lt;title&gt;400 Bad Request&lt;/title&gt;
  &lt;h1&gt;Bad Request&lt;/h1&gt;
  &lt;p&gt;The browser (or proxy) sent a request that this server could not understand.&lt;/p&gt;
  </code></pre>
  
  <p>I have tried passing the string version of v, for the sample of 5 records above </p>
  
  <pre><code>curl -X POST "http://XXXXXXXXX.us-east-1.elasticbeanstalk.com/" -H "Content-Type: application/json" -d '[{"field1":14239302,"field2":29....}]'
  </code></pre>
  
  <p>but it returns </p>
  
  <pre><code>&lt;!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"&gt;
  &lt;title&gt;500 Internal Server Error&lt;/title&gt;
  &lt;h1&gt;Internal Server Error&lt;/h1&gt;
  &lt;p&gt;The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.&lt;/p&gt;
  </code></pre>
  
  <p>How can I get the same result request through curl?  </p>
  
  <p><strong>ADD</strong></p>
  
  <p>Here is a reproducible example. The issue seems to be in how the json is read for creation of a pandas data frame.</p>
  
  <p>Here is the flask app:</p>
  
  <pre><code>from flask import Flask, jsonify, request, render_template, make_response
  import pandas as pd
  
  
  application = Flask(__name__)
  
  
  @application.route('/', methods=['GET','POST'])
  def apicall():
      if request.method == 'POST':
  
          f = request.get_json()
          print(f)
          print(type(f))
          f=str(f)
          print(type(f))
          df=pd.read_json(f,orient='records')
          print(df)
  
          return(jsonify(df.to_json(orient='records')))
  
  if __name__ == '__main__':
  
      application.run()
  </code></pre>
  
  <p>This works perfectly:</p>
  
  <pre><code>r=requests.post('http://127.0.0.1:5000/', json='[{"field1":14239302,"field2":29.90}]', headers = {'content-type': 'application/json'})
  r.json()
  
  127.0.0.1 - - [13/Sep/2019 15:28:16] "POST / HTTP/1.1" 500 -
  [{"field1":14239302,"field2":29.90}]
  &lt;class 'str'&gt;
  &lt;class 'str'&gt;
       field1  field2
  0  14239302    29.9
  127.0.0.1 - - [13/Sep/2019 15:34:51] "POST / HTTP/1.1" 200 -
  </code></pre>
  
  <p>This fails:</p>
  
  <pre><code>!curl -d '[{"field1":14239302,"field2":29.90}]' -H "Content-Type: application/json" -X POST 'http://127.0.0.1:5000/'
  </code></pre>
  
  <p>with the following</p>
  
  <pre><code>127.0.0.1 - - [13/Sep/2019 15:28:13] "POST / HTTP/1.1" 200 -
  {'field1': 14239302, 'field2': 29.9}
  &lt;class 'list'&gt;
  &lt;class 'str'&gt;
  [2019-09-13 15:28:16,851] ERROR in app: Exception on / [POST]
  Traceback (most recent call last):
    File "/home/ubuntu/env_india_flask_eb/lib/python3.6/site-packages/flask/app.py", line 2446, in wsgi_app
      response = self.full_dispatch_request()
    File "/home/ubuntu/env_india_flask_eb/lib/python3.6/site-packages/flask/app.py", line 1951, in full_dispatch_request
      rv = self.handle_user_exception(e)
    File "/home/ubuntu/env_india_flask_eb/lib/python3.6/site-packages/flask/app.py", line 1820, in handle_user_exception
      reraise(exc_type, exc_value, tb)
    File "/home/ubuntu/env_india_flask_eb/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise
      raise value
    File "/home/ubuntu/env_india_flask_eb/lib/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request
      rv = self.dispatch_request()
    File "/home/ubuntu/env_india_flask_eb/lib/python3.6/site-packages/flask/app.py", line 1935, in dispatch_request
      return self.view_functions[rule.endpoint](**req.view_args)
    File "application.py", line 17, in apicall
      df=pd.read_json(f,orient='records')
    File "/home/ubuntu/env_india_flask_eb/lib/python3.6/site-packages/pandas/io/json/_json.py", line 592, in read_json
      result = json_reader.read()
    File "/home/ubuntu/env_india_flask_eb/lib/python3.6/site-packages/pandas/io/json/_json.py", line 717, in read
      obj = self._get_object_parser(self.data)
    File "/home/ubuntu/env_india_flask_eb/lib/python3.6/site-packages/pandas/io/json/_json.py", line 739, in _get_object_parser
      obj = FrameParser(json, **kwargs).parse()
    File "/home/ubuntu/env_india_flask_eb/lib/python3.6/site-packages/pandas/io/json/_json.py", line 849, in parse
      self._parse_no_numpy()
    File "/home/ubuntu/env_india_flask_eb/lib/python3.6/site-packages/pandas/io/json/_json.py", line 1116, in _parse_no_numpy
      loads(json, precise_float=self.precise_float), dtype=None
  ValueError: Expected object or value
  127.0.0.1 - - [13/Sep/2019 15:28:16] "POST / HTTP/1.1" 500 -
  </code></pre>
  
  <p>It seems like the issue is with the fact that the input to the flask app is a string for requests and a list for curl?</p>
  
  <p><strong>ADD BASED ON KEVIN's ANSWER:</strong></p>
  
  <p>I tried this function:</p>
  
  <pre><code>from flask import Flask, jsonify, request, render_template, make_response
  import pandas as pd
  
  
  application = Flask(__name__)
  
  
  @application.route('/', methods=['GET','POST'])
  def apicall():
      if request.method == 'POST':
  
          f = request.get_json(force=True)
          print(f)
          print(type(f))
  
          df=pd.read_json(f,orient='records')
  
          return(jsonify(df.to_json(orient='records')))
  
  if __name__ == '__main__':
  
      application.run()
  </code></pre>
  
  <p>and this input:</p>
  
  <pre><code>s=json.dumps([{'field1':14239302,'field2':29.90}])
  
  !curl  -d s -H "Content-Type:application/json" -X POST 'http://127.0.0.1:5000/'
  </code></pre>
  
  <p>But still receive an error:</p>
  
  <pre><code>&lt;!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"&gt;
  &lt;title&gt;400 Bad Request&lt;/title&gt;
  &lt;h1&gt;Bad Request&lt;/h1&gt;
  &lt;p&gt;The browser (or proxy) sent a request that this server could not understand.&lt;/p&gt;
  </code></pre>