<p dir="auto">I want to know how to handle cases where everything goes wrong, I am not too much worry for the workers, queue or db. I already know how their up time  or how they handle crashes.</p>
<p dir="auto">But, I am still not sure to understand how works airflow. I assume that you a server or a process that know when to add stuff in the queue to respect the dag. Am I right ? if yes, how can I handle a crashes ? (can I duplicate this server/process on a other container ?).</p>
<p dir="auto">--&gt;</p>
<p dir="auto"><strong>Apache Airflow version</strong>: 1.10.10<br>
<strong>Kubernetes version (if you are using kubernetes)</strong> (use <code class="notranslate">kubectl version</code>): NA</p>
<p dir="auto"><strong>Environment</strong>:</p>
<ul dir="auto">
<li><strong>Cloud provider or hardware configuration</strong>: AWS</li>
<li><strong>OS</strong> (e.g. from /etc/os-release): CentOS Linux release 7.7.1908 (Core)</li>
<li><strong>Kernel</strong> (e.g. <code class="notranslate">uname -a</code>): 3.10.0-1062.12.1.el7.x86_64 <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="69689814" data-permission-text="Title is private" data-url="https://github.com/apache/airflow/issues/1" data-hovercard-type="pull_request" data-hovercard-url="/apache/airflow/pull/1/hovercard" href="https://github.com/apache/airflow/pull/1">#1</a> SMP Tue Feb 4 23:02:59 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux</li>
<li><strong>Install tools</strong>:</li>
<li><strong>Others</strong>:</li>
</ul>
<p dir="auto"><strong>What happened</strong>:<br>
After upgrading 1.10.2 to 1.10.10 cant get into WebUI</p>
<ul dir="auto">
<li>no RBAC</li>
<li>no DAG Serialization</li>
<li>postgres source\dest 11.5</li>
<li>prepare new db from 1.10.2 snapshot</li>
<li>connect it to Airflow 1.10.10 python env</li>
<li>double check no live connections to db</li>
<li>run airflow upgradedb</li>
</ul>
<details>
<summary>Alembic log</summary>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="(airflow)$ airflow upgradedb
/opt/python-envs/airflow/lib/python3.6/site-packages/airflow/configuration.py:631: DeprecationWarning: Specifying both AIRFLOW_HOME environment variable and airflow_home in the config file is deprecated. Please use only the AIRFLOW_HOME environment variable and remove the config file entry.
  warnings.warn(msg, category=DeprecationWarning)
[2020-06-14 08:30:14,925] {rest_api_plugin.py:48} WARNING - [rest_api_plugin/REST_API_PLUGIN_EXPECTED_HTTP_TOKEN] value is empty
DB: postgresql://airflow:***@airflow-aws-test-airflowdb.*.us-east-1.rds.amazonaws.com:5432/airflow
[2020-06-14 08:30:15,214] {db.py:378} INFO - Creating tables
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade 41f5f12752f8 -&gt; c8ffec048a3b, add fields to dag
INFO  [alembic.runtime.migration] Running upgrade c8ffec048a3b -&gt; dd4ecb8fbee3, Add schedule interval to dag
INFO  [alembic.runtime.migration] Running upgrade dd4ecb8fbee3 -&gt; 939bb1e647c8, task reschedule fk on cascade delete
INFO  [alembic.runtime.migration] Running upgrade 939bb1e647c8 -&gt; 6e96a59344a4, Make TaskInstance.pool not nullable
INFO  [alembic.runtime.migration] Running upgrade 6e96a59344a4 -&gt; d38e04c12aa2, add serialized_dag table
Revision ID: d38e04c12aa2
Revises: 6e96a59344a4
Create Date: 2019-08-01 14:39:35.616417
INFO  [alembic.runtime.migration] Running upgrade d38e04c12aa2 -&gt; b3b105409875, add root_dag_id to DAG
Revision ID: b3b105409875
Revises: d38e04c12aa2
Create Date: 2019-09-28 23:20:01.744775
INFO  [alembic.runtime.migration] Running upgrade 6e96a59344a4 -&gt; 74effc47d867, change datetime to datetime2(6) on MSSQL tables
INFO  [alembic.runtime.migration] Running upgrade 939bb1e647c8 -&gt; 004c1210f153, increase queue name size limit
INFO  [alembic.runtime.migration] Running upgrade c8ffec048a3b -&gt; a56c9515abdc, Remove dag_stat table
INFO  [alembic.runtime.migration] Running upgrade a56c9515abdc, 004c1210f153, 74effc47d867, b3b105409875 -&gt; 08364691d074, Merge the four heads back together
INFO  [alembic.runtime.migration] Running upgrade 08364691d074 -&gt; fe461863935f, increase_length_for_connection_password
INFO  [alembic.runtime.migration] Running upgrade fe461863935f -&gt; 7939bcff74ba, Add DagTags table
INFO  [alembic.runtime.migration] Running upgrade 7939bcff74ba -&gt; a4c2fd67d16b, add pool_slots field to task_instance
INFO  [alembic.runtime.migration] Running upgrade a4c2fd67d16b -&gt; 852ae6c715af, Add RenderedTaskInstanceFields table
INFO  [alembic.runtime.migration] Running upgrade 852ae6c715af -&gt; 952da73b5eff, add dag_code table"><pre class="notranslate"><code class="notranslate">(airflow)$ airflow upgradedb
/opt/python-envs/airflow/lib/python3.6/site-packages/airflow/configuration.py:631: DeprecationWarning: Specifying both AIRFLOW_HOME environment variable and airflow_home in the config file is deprecated. Please use only the AIRFLOW_HOME environment variable and remove the config file entry.
  warnings.warn(msg, category=DeprecationWarning)
[2020-06-14 08:30:14,925] {rest_api_plugin.py:48} WARNING - [rest_api_plugin/REST_API_PLUGIN_EXPECTED_HTTP_TOKEN] value is empty
DB: postgresql://airflow:***@airflow-aws-test-airflowdb.*.us-east-1.rds.amazonaws.com:5432/airflow
[2020-06-14 08:30:15,214] {db.py:378} INFO - Creating tables
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade 41f5f12752f8 -&gt; c8ffec048a3b, add fields to dag
INFO  [alembic.runtime.migration] Running upgrade c8ffec048a3b -&gt; dd4ecb8fbee3, Add schedule interval to dag
INFO  [alembic.runtime.migration] Running upgrade dd4ecb8fbee3 -&gt; 939bb1e647c8, task reschedule fk on cascade delete
INFO  [alembic.runtime.migration] Running upgrade 939bb1e647c8 -&gt; 6e96a59344a4, Make TaskInstance.pool not nullable
INFO  [alembic.runtime.migration] Running upgrade 6e96a59344a4 -&gt; d38e04c12aa2, add serialized_dag table
Revision ID: d38e04c12aa2
Revises: 6e96a59344a4
Create Date: 2019-08-01 14:39:35.616417
INFO  [alembic.runtime.migration] Running upgrade d38e04c12aa2 -&gt; b3b105409875, add root_dag_id to DAG
Revision ID: b3b105409875
Revises: d38e04c12aa2
Create Date: 2019-09-28 23:20:01.744775
INFO  [alembic.runtime.migration] Running upgrade 6e96a59344a4 -&gt; 74effc47d867, change datetime to datetime2(6) on MSSQL tables
INFO  [alembic.runtime.migration] Running upgrade 939bb1e647c8 -&gt; 004c1210f153, increase queue name size limit
INFO  [alembic.runtime.migration] Running upgrade c8ffec048a3b -&gt; a56c9515abdc, Remove dag_stat table
INFO  [alembic.runtime.migration] Running upgrade a56c9515abdc, 004c1210f153, 74effc47d867, b3b105409875 -&gt; 08364691d074, Merge the four heads back together
INFO  [alembic.runtime.migration] Running upgrade 08364691d074 -&gt; fe461863935f, increase_length_for_connection_password
INFO  [alembic.runtime.migration] Running upgrade fe461863935f -&gt; 7939bcff74ba, Add DagTags table
INFO  [alembic.runtime.migration] Running upgrade 7939bcff74ba -&gt; a4c2fd67d16b, add pool_slots field to task_instance
INFO  [alembic.runtime.migration] Running upgrade a4c2fd67d16b -&gt; 852ae6c715af, Add RenderedTaskInstanceFields table
INFO  [alembic.runtime.migration] Running upgrade 852ae6c715af -&gt; 952da73b5eff, add dag_code table
</code></pre></div>
</details>
<p dir="auto">After auth getting next screen constantly</p>
<details>
<summary>Crush log from web-ui</summary>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="-------------------------------------------------------------------------------
Node: airflow-master-test-1-10-10.dev.somehost.net
-------------------------------------------------------------------------------
Traceback (most recent call last):
  File &quot;/opt/python-envs/airflow/lib/python3.6/site-packages/flask/app.py&quot;, line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File &quot;/opt/python-envs/airflow/lib/python3.6/site-packages/flask/app.py&quot;, line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File &quot;/opt/python-envs/airflow/lib/python3.6/site-packages/flask/app.py&quot;, line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File &quot;/opt/python-envs/airflow/lib/python3.6/site-packages/flask/_compat.py&quot;, line 39, in reraise
    raise value
  File &quot;/opt/python-envs/airflow/lib/python3.6/site-packages/flask/app.py&quot;, line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File &quot;/opt/python-envs/airflow/lib/python3.6/site-packages/flask/app.py&quot;, line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File &quot;/opt/python-envs/airflow/lib/python3.6/site-packages/flask_admin/base.py&quot;, line 69, in inner
    return self._run_view(f, *args, **kwargs)
  File &quot;/opt/python-envs/airflow/lib/python3.6/site-packages/flask_admin/base.py&quot;, line 368, in _run_view
    return fn(self, *args, **kwargs)
  File &quot;/opt/python-envs/airflow/lib/python3.6/site-packages/flask_login/utils.py&quot;, line 261, in decorated_view
    return func(*args, **kwargs)
  File &quot;/opt/python-envs/airflow/lib/python3.6/site-packages/airflow/utils/db.py&quot;, line 74, in wrapper
    return func(*args, **kwargs)
  File &quot;/opt/python-envs/airflow/lib/python3.6/site-packages/airflow/www/views.py&quot;, line 2330, in index
    auto_complete_data=auto_complete_data)
  File &quot;/opt/python-envs/airflow/lib/python3.6/site-packages/airflow/www/views.py&quot;, line 389, in render
    return super(AirflowViewMixin, self).render(template, **kwargs)
  File &quot;/opt/python-envs/airflow/lib/python3.6/site-packages/flask_admin/base.py&quot;, line 308, in render
    return render_template(template, **kwargs)
  File &quot;/opt/python-envs/airflow/lib/python3.6/site-packages/flask/templating.py&quot;, line 140, in render_template
    ctx.app,
  File &quot;/opt/python-envs/airflow/lib/python3.6/site-packages/flask/templating.py&quot;, line 120, in _render
    rv = template.render(context)
  File &quot;/opt/python-envs/airflow/lib/python3.6/site-packages/jinja2/asyncsupport.py&quot;, line 76, in render
    return original_render(self, *args, **kwargs)
  File &quot;/opt/python-envs/airflow/lib/python3.6/site-packages/jinja2/environment.py&quot;, line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File &quot;/opt/python-envs/airflow/lib/python3.6/site-packages/jinja2/environment.py&quot;, line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File &quot;/opt/python-envs/airflow/lib/python3.6/site-packages/jinja2/_compat.py&quot;, line 37, in reraise
    raise value.with_traceback(tb)
  File &quot;/opt/python-envs/airflow/lib/python3.6/site-packages/airflow/www/templates/airflow/dags.html&quot;, line 20, in top-level template code
    {% extends &quot;airflow/master.html&quot; %}
  File &quot;/opt/python-envs/airflow/lib/python3.6/site-packages/airflow/www/templates/airflow/master.html&quot;, line 20, in top-level template code
    {% extends &quot;admin/master.html&quot; %}
  File &quot;/opt/python-envs/airflow/lib/python3.6/site-packages/airflow/www/templates/admin/master.html&quot;, line 20, in top-level template code
    {% extends 'admin/base.html' %}
  File &quot;/opt/python-envs/airflow/lib/python3.6/site-packages/flask_admin/templates/bootstrap3/admin/base.html&quot;, line 38, in top-level template code
    {% block page_body %}
  File &quot;/opt/python-envs/airflow/lib/python3.6/site-packages/airflow/www/templates/admin/master.html&quot;, line 191, in block &quot;page_body&quot;
    {% block body %}
  File &quot;/opt/python-envs/airflow/lib/python3.6/site-packages/airflow/www/templates/airflow/dags.html&quot;, line 84, in block &quot;body&quot;
    &lt;a href=&quot;{{ url_for('airflow.'+ dag.get_default_view(), dag_id=dag.dag_id) }}&quot; title=&quot;{{ dag.description[0:80] + '...' if dag.description|length &gt; 80 else dag.description }}&quot;&gt;
TypeError: object of type 'NoneType' has no len()
"><pre class="notranslate"><code class="notranslate">-------------------------------------------------------------------------------
Node: airflow-master-test-1-10-10.dev.somehost.net
-------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/python-envs/airflow/lib/python3.6/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "/opt/python-envs/airflow/lib/python3.6/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/opt/python-envs/airflow/lib/python3.6/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/opt/python-envs/airflow/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/opt/python-envs/airflow/lib/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/opt/python-envs/airflow/lib/python3.6/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/opt/python-envs/airflow/lib/python3.6/site-packages/flask_admin/base.py", line 69, in inner
    return self._run_view(f, *args, **kwargs)
  File "/opt/python-envs/airflow/lib/python3.6/site-packages/flask_admin/base.py", line 368, in _run_view
    return fn(self, *args, **kwargs)
  File "/opt/python-envs/airflow/lib/python3.6/site-packages/flask_login/utils.py", line 261, in decorated_view
    return func(*args, **kwargs)
  File "/opt/python-envs/airflow/lib/python3.6/site-packages/airflow/utils/db.py", line 74, in wrapper
    return func(*args, **kwargs)
  File "/opt/python-envs/airflow/lib/python3.6/site-packages/airflow/www/views.py", line 2330, in index
    auto_complete_data=auto_complete_data)
  File "/opt/python-envs/airflow/lib/python3.6/site-packages/airflow/www/views.py", line 389, in render
    return super(AirflowViewMixin, self).render(template, **kwargs)
  File "/opt/python-envs/airflow/lib/python3.6/site-packages/flask_admin/base.py", line 308, in render
    return render_template(template, **kwargs)
  File "/opt/python-envs/airflow/lib/python3.6/site-packages/flask/templating.py", line 140, in render_template
    ctx.app,
  File "/opt/python-envs/airflow/lib/python3.6/site-packages/flask/templating.py", line 120, in _render
    rv = template.render(context)
  File "/opt/python-envs/airflow/lib/python3.6/site-packages/jinja2/asyncsupport.py", line 76, in render
    return original_render(self, *args, **kwargs)
  File "/opt/python-envs/airflow/lib/python3.6/site-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/opt/python-envs/airflow/lib/python3.6/site-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/opt/python-envs/airflow/lib/python3.6/site-packages/jinja2/_compat.py", line 37, in reraise
    raise value.with_traceback(tb)
  File "/opt/python-envs/airflow/lib/python3.6/site-packages/airflow/www/templates/airflow/dags.html", line 20, in top-level template code
    {% extends "airflow/master.html" %}
  File "/opt/python-envs/airflow/lib/python3.6/site-packages/airflow/www/templates/airflow/master.html", line 20, in top-level template code
    {% extends "admin/master.html" %}
  File "/opt/python-envs/airflow/lib/python3.6/site-packages/airflow/www/templates/admin/master.html", line 20, in top-level template code
    {% extends 'admin/base.html' %}
  File "/opt/python-envs/airflow/lib/python3.6/site-packages/flask_admin/templates/bootstrap3/admin/base.html", line 38, in top-level template code
    {% block page_body %}
  File "/opt/python-envs/airflow/lib/python3.6/site-packages/airflow/www/templates/admin/master.html", line 191, in block "page_body"
    {% block body %}
  File "/opt/python-envs/airflow/lib/python3.6/site-packages/airflow/www/templates/airflow/dags.html", line 84, in block "body"
    &lt;a href="{{ url_for('airflow.'+ dag.get_default_view(), dag_id=dag.dag_id) }}" title="{{ dag.description[0:80] + '...' if dag.description|length &gt; 80 else dag.description }}"&gt;
TypeError: object of type 'NoneType' has no len()

</code></pre></div>
</details>
<p dir="auto"><strong>What you expected to happen</strong>: WebUI works</p>
<p dir="auto"><strong>How to reproduce it</strong>:</p>
<ul dir="auto">
<li>Prepare some DAGs without description executed under  Airflow 1.10.2</li>
<li>Upgrade to Airflow 1.10.10</li>
</ul>
<p dir="auto"><strong>Anything else we need to know</strong>:</p>
<details><summary>DB data which wont work (sensitive data removed)</summary>
<div class="highlight highlight-source-sql notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="
airflow=&gt; select dag_id,default_view,description from dag where (description = '') IS NOT FALSE  limit 10;
                               dag_id                                | default_view | description 
---------------------------------------------------------------------+--------------+-------------
 hello_world                                                         |              | 

(10 rows)"><pre class="notranslate">airflow<span class="pl-k">=&gt;</span> <span class="pl-k">select</span> dag_id,default_view,description <span class="pl-k">from</span> dag <span class="pl-k">where</span> (description <span class="pl-k">=</span> <span class="pl-s"><span class="pl-pds">'</span><span class="pl-pds">'</span></span>) IS NOT FALSE  <span class="pl-k">limit</span> <span class="pl-c1">10</span>;
                               dag_id                                | default_view | description 
<span class="pl-c"><span class="pl-c">--</span>-------------------------------------------------------------------+--------------+-------------</span>
 hello_world                                                         |              | 

(<span class="pl-c1">10</span> rows)</pre></div>
</details>
<details>
<summary>Quick fix which helped  (sensitive data removed)</summary>
<div class="highlight highlight-source-sql notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="airflow=&gt; update dag SET description = 'Desc' where (description = '') IS NOT FALSE;
UPDATE 14"><pre class="notranslate">airflow<span class="pl-k">=&gt;</span> <span class="pl-k">update</span> dag <span class="pl-k">SET</span> description <span class="pl-k">=</span> <span class="pl-s"><span class="pl-pds">'</span>Desc<span class="pl-pds">'</span></span> <span class="pl-k">where</span> (description <span class="pl-k">=</span> <span class="pl-s"><span class="pl-pds">'</span><span class="pl-pds">'</span></span>) IS NOT FALSE;
<span class="pl-k">UPDATE</span> <span class="pl-c1">14</span></pre></div>
<div class="highlight highlight-source-sql notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="airflow=&gt; select dag_id,default_view,description from dag where (description = '') IS NOT FALSE  limit 10;
 dag_id | default_view | description 
--------+--------------+-------------
(0 rows)"><pre class="notranslate">airflow<span class="pl-k">=&gt;</span> <span class="pl-k">select</span> dag_id,default_view,description <span class="pl-k">from</span> dag <span class="pl-k">where</span> (description <span class="pl-k">=</span> <span class="pl-s"><span class="pl-pds">'</span><span class="pl-pds">'</span></span>) IS NOT FALSE  <span class="pl-k">limit</span> <span class="pl-c1">10</span>;
 dag_id | default_view | description 
<span class="pl-c"><span class="pl-c">--</span>------+--------------+-------------</span>
(<span class="pl-c1">0</span> rows)</pre></div>
<div class="highlight highlight-source-sql notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="airflow=&gt; select dag_id,default_view,description from dag limit 10;
                               dag_id                                | default_view | description 
---------------------------------------------------------------------+--------------+-------------
 hello_world                                                         |              | Desc

(10 rows)

"><pre class="notranslate">airflow<span class="pl-k">=&gt;</span> <span class="pl-k">select</span> dag_id,default_view,description <span class="pl-k">from</span> dag <span class="pl-k">limit</span> <span class="pl-c1">10</span>;
                               dag_id                                | default_view | description 
<span class="pl-c"><span class="pl-c">--</span>-------------------------------------------------------------------+--------------+-------------</span>
 hello_world                                                         |              | <span class="pl-k">Desc</span>

(<span class="pl-c1">10</span> rows)

</pre></div>
</details>