title: What response times can be expected from GAE/NDB?
body: <p>We are currently building a small and simple central HTTP service that maps "external identities" (like a facebook id) to an "internal (uu)id", unique across all our services to help with analytics.</p>
  
  <p>The first prototype in "our stack" (flask+postgresql) was done within a day. But since we want the service to (almost) never fail and scale automagically, we decided to use Google App Engine.</p>
  
  <p>After a week of reading&amp;trying&amp;benchmarking this question emerges:</p>
  
  <p><strong>What response times are considered "normal" on App Engine (with NDB)?</strong></p>
  
  <p>We are getting response times that are consistently <em>above 500ms on average</em> and <em>well above 1s in the 90percentile</em>.</p>
  
  <p>I've attached a stripped down version of our code below, hoping somebody can point out the obvious flaw. We really like the autoscaling and the distributed storage, but we can not imagine 500ms really is the expected performance in our case. The sql based prototype responded much faster (consistently), hosted on one single Heroku dyno using the free, cache-less postgresql (even with an ORM).</p>
  
  <p>We tried both synchronous and asynchronous variants of the code below and looked at the appstats profile. It's always RPC calls (both memcache and datastore) that take very long (50ms-100ms), made worse by the fact that there are always multiple calls (eg. mc.get() + ds.get() + ds.set() on a write). We also tried deferring as much as possible to the task queue, without noticeable gains.</p>
  
  <pre class="lang-py prettyprint-override"><code>import json
  import uuid
  
  from google.appengine.ext import ndb
  
  import webapp2
  from webapp2_extras.routes import RedirectRoute
  
  
  def _parse_request(request):
      if request.content_type == 'application/json':
          try:
              body_json = json.loads(request.body)
              provider_name = body_json.get('provider_name', None)
              provider_user_id = body_json.get('provider_user_id', None)
          except ValueError:
              return webapp2.abort(400, detail='invalid json')
      else:
          provider_name = request.params.get('provider_name', None)
          provider_user_id = request.params.get('provider_user_id', None)
  
      return provider_name, provider_user_id
  
  
  class Provider(ndb.Model):
      name = ndb.StringProperty(required=True)
  
  
  class Identity(ndb.Model):
      user = ndb.KeyProperty(kind='GlobalUser')
  
  
  class GlobalUser(ndb.Model):
      uuid = ndb.StringProperty(required=True)
  
      @property
      def identities(self):
          return Identity.query(Identity.user==self.key).fetch()
  
  
  class ResolveHandler(webapp2.RequestHandler):
      @ndb.toplevel
      def post(self):
          provider_name, provider_user_id = _parse_request(self.request)
  
          if not provider_name or not provider_user_id:
              return self.abort(400, detail='missing provider_name and/or provider_user_id')
  
          identity = ndb.Key(Provider, provider_name, Identity, provider_user_id).get()
  
          if identity:
              user_uuid = identity.user.id()
          else:
              user_uuid = uuid.uuid4().hex
  
              GlobalUser(
                  id=user_uuid,
                  uuid=user_uuid
              ).put_async()
  
              Identity(
                  parent=ndb.Key(Provider, provider_name),
                  id=provider_user_id,
                  user=ndb.Key(GlobalUser, user_uuid)
              ).put_async()
  
          return webapp2.Response(
              status='200 OK',
              content_type='application/json',
              body = json.dumps({
                  'provider_name' : provider_name,
                  'provider_user_id' : provider_user_id,
                  'uuid' : user_uuid
              })
          )
  
  app = webapp2.WSGIApplication([
        RedirectRoute('/v1/resolve', ResolveHandler, 'resolve', strict_slash=True)
  ], debug=False)
  </code></pre>
  
  <p>For completeness sake the (almost default) app.yaml</p>
  
  <pre><code>application: GAE_APP_IDENTIFIER
  version: 1
  runtime: python27
  api_version: 1
  threadsafe: yes
  
  handlers:
  - url: .*
    script: main.app
  
  libraries:
  - name: webapp2
    version: 2.5.2
  - name: webob
    version: 1.2.3
  
  inbound_services:
  - warmup
  </code></pre>