title: Why is my App Engine app hitting the Stackdriver Logging 256KB limit?
body: <p>I'm migrating a Python 2  App Engine Standard app to the new Python 3 Standard runtime, which is <a href="https://cloud.google.com/appengine/docs/standard/python3/python-differences#app_engine_apis" rel="nofollow noreferrer">very different</a>. One big difference is that you use Google Cloud APIs explicitly, including <a href="https://cloud.google.com/logging/docs/" rel="nofollow noreferrer">Stackdriver Logging</a>. I'm using <a href="https://googleapis.github.io/google-cloud-python/latest/logging/" rel="nofollow noreferrer">google-cloud-logging</a>'s <a href="https://googleapis.dev/python/logging/latest/handlers-app-engine.html" rel="nofollow noreferrer">App Engine Handler</a> and configuring it to correlate logs with HTTP requests:</p>
  
  <pre class="lang-py prettyprint-override"><code>import google.cloud.logging
  logging_client = google.cloud.logging.Client()
  
  from google.cloud.logging.handlers import AppEngineHandler, setup_logging
  setup_logging(AppEngineHandler(logging_client, name='stdout'))
  </code></pre>
  
  <p>This works great!...except it hits <a href="https://cloud.google.com/logging/quotas#log-limits" rel="nofollow noreferrer">a 256KB log entry limit</a> and crashes somewhat regularly. Stack trace below. This happens in long-running background requests that do a fair amount of work, but when I count the total amount of logged text (including string timestamps) in requests that crash, it only averages 5-15KB or so, far smaller than 256KB. (Yes, the docs warn <a href="https://cloud.google.com/logging/quotas#log-limits" rel="nofollow noreferrer"><em>This approximate limit is based on internal data sizes</em></a>, but it's hard to believe that overhead is 15-50x.)</p>
  
  <p>The App Engine Handler defaults to <a href="https://googleapis.dev/python/logging/latest/transports-thread.html" rel="nofollow noreferrer"><code>BackgroundThreadTransport</code></a>. I've tried lowering its <code>batch_size</code> from 10 (the default) to 1, and switching to <a href="https://googleapis.dev/python/logging/latest/transports-sync.html" rel="nofollow noreferrer"><code>SyncTransport</code></a>, but no luck.</p>
  
  <p>Any idea what I'm missing? Thank you in advance!</p>
  
  <pre><code>Traceback (most recent call last):
    File "/env/lib/python3.7/site-packages/google/cloud/logging/handlers/transports/background_thread.py", line 123, in _safely_commit_batch
      batch.commit()
    File "/env/lib/python3.7/site-packages/google/cloud/logging/logger.py", line 383, in commit
      client.logging_api.write_entries(entries, **kwargs)
    File "/env/lib/python3.7/site-packages/google/cloud/logging/_gapic.py", line 126, in write_entries
      partial_success=partial_success,
    File "/env/lib/python3.7/site-packages/google/cloud/logging_v2/gapic/logging_service_v2_client.py", line 477, in write_log_entries
      request, retry=retry, timeout=timeout, metadata=metadata
    File "/env/lib/python3.7/site-packages/google/api_core/gapic_v1/method.py", line 143, in __call__
      return wrapped_func(*args, **kwargs)
    File "/env/lib/python3.7/site-packages/google/api_core/retry.py", line 277, in retry_wrapped_func
      on_error=on_error,
    File "/env/lib/python3.7/site-packages/google/api_core/retry.py", line 182, in retry_target
      return target()
    File "/env/lib/python3.7/site-packages/google/api_core/timeout.py", line 214, in func_with_timeout
      return func(*args, **kwargs)
    File "/env/lib/python3.7/site-packages/google/api_core/grpc_helpers.py", line 59, in error_remapped_callable
      six.raise_from(exceptions.from_grpc_error(exc), exc)
    File "&lt;string&gt;", line 3, in raise_from
  google.api_core.exceptions.InvalidArgument: 400 Log entry with size 258.3K exceeds maximum size of 256.0K
  </code></pre>