<h3 dir="auto">Apache Airflow version</h3>
<p dir="auto">2.3.2 (latest released)</p>
<h3 dir="auto">What happened</h3>
<p dir="auto">Upgrade Airflow 2.2.3 or 2.2.5 -&gt; 2.3.2 fails on migration-job.</p>
<p dir="auto"><strong>first time upgrade execution:</strong></p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Referencing column 'task_id' and referenced column 'task_id' in foreign key constraint 'task_map_task_instance_fkey' are incompatible.&quot;)
[SQL: 
CREATE TABLE task_map (
    dag_id VARCHAR(250) COLLATE utf8mb3_bin NOT NULL, 
    task_id VARCHAR(250) COLLATE utf8mb3_bin NOT NULL, 
    run_id VARCHAR(250) COLLATE utf8mb3_bin NOT NULL, 
    map_index INTEGER NOT NULL, 
    length INTEGER NOT NULL, 
    `keys` JSON, 
    PRIMARY KEY (dag_id, task_id, run_id, map_index), 
    CONSTRAINT task_map_length_not_negative CHECK (length &gt;= 0), 
    CONSTRAINT task_map_task_instance_fkey FOREIGN KEY(dag_id, task_id, run_id, map_index) REFERENCES task_instance (dag_id, task_id, run_id, map_index) ON DELETE CASCADE
)
]"><pre class="notranslate"><code class="notranslate">Referencing column 'task_id' and referenced column 'task_id' in foreign key constraint 'task_map_task_instance_fkey' are incompatible.")
[SQL: 
CREATE TABLE task_map (
    dag_id VARCHAR(250) COLLATE utf8mb3_bin NOT NULL, 
    task_id VARCHAR(250) COLLATE utf8mb3_bin NOT NULL, 
    run_id VARCHAR(250) COLLATE utf8mb3_bin NOT NULL, 
    map_index INTEGER NOT NULL, 
    length INTEGER NOT NULL, 
    `keys` JSON, 
    PRIMARY KEY (dag_id, task_id, run_id, map_index), 
    CONSTRAINT task_map_length_not_negative CHECK (length &gt;= 0), 
    CONSTRAINT task_map_task_instance_fkey FOREIGN KEY(dag_id, task_id, run_id, map_index) REFERENCES task_instance (dag_id, task_id, run_id, map_index) ON DELETE CASCADE
)
]
</code></pre></div>
<p dir="auto"><strong>after the first failed execution (should be due to the first failed execution):</strong></p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Can't DROP 'task_reschedule_ti_fkey'; check that column/key exists&quot;)
[SQL: ALTER TABLE task_reschedule DROP FOREIGN KEY task_reschedule_ti_fkey[]"><pre class="notranslate"><code class="notranslate">Can't DROP 'task_reschedule_ti_fkey'; check that column/key exists")
[SQL: ALTER TABLE task_reschedule DROP FOREIGN KEY task_reschedule_ti_fkey[]
</code></pre></div>
<h3 dir="auto">What you think should happen instead</h3>
<p dir="auto">The migration-job shouldn't fail ;)</p>
<h3 dir="auto">How to reproduce</h3>
<p dir="auto">Everytime in my environment just need to create a snapshot from last working DB-Snapshot (Airflow Version 2.2.3)<br>
and then deploy Airflow 2.3.2.<br>
I can update in between to 2.2.5 but ran into the same issue by update to 2.3.2.</p>
<h3 dir="auto">Operating System</h3>
<p dir="auto">Debian GNU/Linux 10 (buster) - apache/airflow:2.3.2-python3.8 (hub.docker.com)</p>
<h3 dir="auto">Versions of Apache Airflow Providers</h3>
<p dir="auto">apache-airflow-providers-amazon==2.4.0<br>
apache-airflow-providers-celery==2.1.0<br>
apache-airflow-providers-cncf-kubernetes==2.2.0<br>
apache-airflow-providers-docker==2.3.0<br>
apache-airflow-providers-elasticsearch==2.1.0<br>
apache-airflow-providers-ftp==2.0.1<br>
apache-airflow-providers-google==6.2.0<br>
apache-airflow-providers-grpc==2.0.1<br>
apache-airflow-providers-hashicorp==2.1.1<br>
apache-airflow-providers-http==2.0.1<br>
apache-airflow-providers-imap==2.0.1<br>
apache-airflow-providers-microsoft-azure==3.4.0<br>
apache-airflow-providers-mysql==2.1.1<br>
apache-airflow-providers-odbc==2.0.1<br>
apache-airflow-providers-postgres==2.4.0<br>
apache-airflow-providers-redis==2.0.1<br>
apache-airflow-providers-sendgrid==2.0.1<br>
apache-airflow-providers-sftp==2.3.0<br>
apache-airflow-providers-slack==4.1.0<br>
apache-airflow-providers-sqlite==2.0.1<br>
apache-airflow-providers-ssh==2.3.0<br>
apache-airflow-providers-tableau==2.1.4</p>
<h3 dir="auto">Deployment</h3>
<p dir="auto">Official Apache Airflow Helm Chart</p>
<h3 dir="auto">Deployment details</h3>
<ul dir="auto">
<li>K8s Rev: v1.21.12-eks-a64ea69</li>
<li>helm chart version: 1.6.0</li>
<li>Database: AWS RDS MySQL 8.0.28</li>
</ul>
<h3 dir="auto">Anything else</h3>
<p dir="auto">Full error Log <strong>first</strong> execution:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="/home/airflow/.local/lib/python3.8/site-packages/airflow/configuration.py:529: DeprecationWarning: The auth_backend option in [api[] has been renamed to auth_backends - the old setting has been used, but please update your config.
  option = self._get_option_from_config_file(deprecated_key, deprecated_section, key, kwargs, section)
/home/airflow/.local/lib/python3.8/site-packages/airflow/configuration.py:356: FutureWarning: The auth_backends setting in [api[] has had airflow.api.auth.backend.session added in the running config, which is needed by the UI. Please update your config before Apache Airflow 3.0.
  warnings.warn(
DB: mysql+mysqldb://airflow:***@test-airflow2-db-blue.fsgfsdcfds76.eu-central-1.rds.amazonaws.com:3306/airflow
Performing upgrade with database mysql+mysqldb://airflow:***@test-airflow2-db-blue.fsgfsdcfds76.eu-central-1.rds.amazonaws.com:3306/airflow
[2022-06-17 12:19:59,724[] {db.py:920} WARNING - Found 33 duplicates in table task_fail.  Will attempt to move them.
[2022-06-17 12:36:18,813[] {db.py:1448} INFO - Creating tables
INFO  [alembic.runtime.migration[] Context impl MySQLImpl.
INFO  [alembic.runtime.migration[] Will assume non-transactional DDL.
INFO  [alembic.runtime.migration[] Running upgrade be2bfac3da23 -&gt; c381b21cb7e4, Create a ``session`` table to store web session data
INFO  [alembic.runtime.migration[] Running upgrade c381b21cb7e4 -&gt; 587bdf053233, Add index for ``dag_id`` column in ``job`` table.
INFO  [alembic.runtime.migration[] Running upgrade 587bdf053233 -&gt; 5e3ec427fdd3, Increase length of email and username in ``ab_user`` and ``ab_register_user`` table to ``256`` characters
INFO  [alembic.runtime.migration[] Running upgrade 5e3ec427fdd3 -&gt; 786e3737b18f, Add ``timetable_description`` column to DagModel for UI.
INFO  [alembic.runtime.migration[] Running upgrade 786e3737b18f -&gt; f9da662e7089, Add ``LogTemplate`` table to track changes to config values ``log_filename_template``
INFO  [alembic.runtime.migration[] Running upgrade f9da662e7089 -&gt; e655c0453f75, Add ``map_index`` column to TaskInstance to identify task-mapping,
and a ``task_map`` table to track mapping values from XCom.
Traceback (most recent call last):
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py&quot;, line 1705, in _execute_context
    self.dialect.do_execute(
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py&quot;, line 716, in do_execute
    cursor.execute(statement, parameters)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/MySQLdb/cursors.py&quot;, line 206, in execute
    res = self._query(query)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/MySQLdb/cursors.py&quot;, line 319, in _query
    db.query(q)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/MySQLdb/connections.py&quot;, line 254, in query
    _mysql.connection.query(self, query)
MySQLdb._exceptions.OperationalError: (3780, &quot;Referencing column 'task_id' and referenced column 'task_id' in foreign key constraint 'task_map_task_instance_fkey' are incompatible.&quot;)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File &quot;/home/airflow/.local/bin/airflow&quot;, line 8, in &lt;module&gt;
    sys.exit(main())
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/airflow/__main__.py&quot;, line 38, in main
    args.func(args)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/cli_parser.py&quot;, line 51, in command
    return func(*args, **kwargs)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/cli.py&quot;, line 99, in wrapper
    return f(*args, **kwargs)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/db_command.py&quot;, line 82, in upgradedb
    db.upgradedb(to_revision=to_revision, from_revision=from_revision, show_sql_only=args.show_sql_only)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/session.py&quot;, line 71, in wrapper
    return func(*args, session=session, **kwargs)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/db.py&quot;, line 1449, in upgradedb
    command.upgrade(config, revision=to_revision or 'heads')
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/alembic/command.py&quot;, line 322, in upgrade
    script.run_env()
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/alembic/script/base.py&quot;, line 569, in run_env
    util.load_python_file(self.dir, &quot;env.py&quot;)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/alembic/util/pyfiles.py&quot;, line 94, in load_python_file
    module = load_module_py(module_id, path)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/alembic/util/pyfiles.py&quot;, line 110, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
  File &quot;&lt;frozen importlib._bootstrap_external&gt;&quot;, line 843, in exec_module
  File &quot;&lt;frozen importlib._bootstrap&gt;&quot;, line 219, in _call_with_frames_removed
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/airflow/migrations/env.py&quot;, line 107, in &lt;module&gt;
    run_migrations_online()
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/airflow/migrations/env.py&quot;, line 101, in run_migrations_online
    context.run_migrations()
  File &quot;&lt;string&gt;&quot;, line 8, in run_migrations
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/alembic/runtime/environment.py&quot;, line 853, in run_migrations
    self.get_context().run_migrations(**kw)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/alembic/runtime/migration.py&quot;, line 623, in run_migrations
    step.migration_fn(**kw)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/airflow/migrations/versions/0100_2_3_0_add_taskmap_and_map_id_on_taskinstance.py&quot;, line 75, in upgrade
    op.create_table(
  File &quot;&lt;string&gt;&quot;, line 8, in create_table
  File &quot;&lt;string&gt;&quot;, line 3, in create_table
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/alembic/operations/ops.py&quot;, line 1254, in create_table
    return operations.invoke(op)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/alembic/operations/base.py&quot;, line 394, in invoke
    return fn(self, operation)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/alembic/operations/toimpl.py&quot;, line 114, in create_table
    operations.impl.create_table(table)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/alembic/ddl/impl.py&quot;, line 354, in create_table
    self._exec(schema.CreateTable(table))
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/alembic/ddl/impl.py&quot;, line 195, in _exec
    return conn.execute(construct, multiparams)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py&quot;, line 1200, in execute
    return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py&quot;, line 77, in _execute_on_connection
    return connection._execute_ddl(
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py&quot;, line 1290, in _execute_ddl
    ret = self._execute_context(
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py&quot;, line 1748, in _execute_context
    self._handle_dbapi_exception(
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py&quot;, line 1929, in _handle_dbapi_exception
    util.raise_(
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py&quot;, line 211, in raise_
    raise exception
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py&quot;, line 1705, in _execute_context
    self.dialect.do_execute(
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py&quot;, line 716, in do_execute
    cursor.execute(statement, parameters)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/MySQLdb/cursors.py&quot;, line 206, in execute
    res = self._query(query)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/MySQLdb/cursors.py&quot;, line 319, in _query
    db.query(q)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/MySQLdb/connections.py&quot;, line 254, in query
    _mysql.connection.query(self, query)
sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (3780, &quot;Referencing column 'task_id' and referenced column 'task_id' in foreign key constraint 'task_map_task_instance_fkey' are incompatible.&quot;)
[SQL: 
CREATE TABLE task_map (
    dag_id VARCHAR(250) COLLATE utf8mb3_bin NOT NULL, 
    task_id VARCHAR(250) COLLATE utf8mb3_bin NOT NULL, 
    run_id VARCHAR(250) COLLATE utf8mb3_bin NOT NULL, 
    map_index INTEGER NOT NULL, 
    length INTEGER NOT NULL, 
    `keys` JSON, 
    PRIMARY KEY (dag_id, task_id, run_id, map_index), 
    CONSTRAINT task_map_length_not_negative CHECK (length &gt;= 0), 
    CONSTRAINT task_map_task_instance_fkey FOREIGN KEY(dag_id, task_id, run_id, map_index) REFERENCES task_instance (dag_id, task_id, run_id, map_index) ON DELETE CASCADE
)
]
(Background on this error at: http://sqlalche.me/e/14/e3q8)"><pre class="notranslate"><code class="notranslate">/home/airflow/.local/lib/python3.8/site-packages/airflow/configuration.py:529: DeprecationWarning: The auth_backend option in [api[] has been renamed to auth_backends - the old setting has been used, but please update your config.
  option = self._get_option_from_config_file(deprecated_key, deprecated_section, key, kwargs, section)
/home/airflow/.local/lib/python3.8/site-packages/airflow/configuration.py:356: FutureWarning: The auth_backends setting in [api[] has had airflow.api.auth.backend.session added in the running config, which is needed by the UI. Please update your config before Apache Airflow 3.0.
  warnings.warn(
DB: mysql+mysqldb://airflow:***@test-airflow2-db-blue.fsgfsdcfds76.eu-central-1.rds.amazonaws.com:3306/airflow
Performing upgrade with database mysql+mysqldb://airflow:***@test-airflow2-db-blue.fsgfsdcfds76.eu-central-1.rds.amazonaws.com:3306/airflow
[2022-06-17 12:19:59,724[] {db.py:920} WARNING - Found 33 duplicates in table task_fail.  Will attempt to move them.
[2022-06-17 12:36:18,813[] {db.py:1448} INFO - Creating tables
INFO  [alembic.runtime.migration[] Context impl MySQLImpl.
INFO  [alembic.runtime.migration[] Will assume non-transactional DDL.
INFO  [alembic.runtime.migration[] Running upgrade be2bfac3da23 -&gt; c381b21cb7e4, Create a ``session`` table to store web session data
INFO  [alembic.runtime.migration[] Running upgrade c381b21cb7e4 -&gt; 587bdf053233, Add index for ``dag_id`` column in ``job`` table.
INFO  [alembic.runtime.migration[] Running upgrade 587bdf053233 -&gt; 5e3ec427fdd3, Increase length of email and username in ``ab_user`` and ``ab_register_user`` table to ``256`` characters
INFO  [alembic.runtime.migration[] Running upgrade 5e3ec427fdd3 -&gt; 786e3737b18f, Add ``timetable_description`` column to DagModel for UI.
INFO  [alembic.runtime.migration[] Running upgrade 786e3737b18f -&gt; f9da662e7089, Add ``LogTemplate`` table to track changes to config values ``log_filename_template``
INFO  [alembic.runtime.migration[] Running upgrade f9da662e7089 -&gt; e655c0453f75, Add ``map_index`` column to TaskInstance to identify task-mapping,
and a ``task_map`` table to track mapping values from XCom.
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1705, in _execute_context
    self.dialect.do_execute(
  File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 716, in do_execute
    cursor.execute(statement, parameters)
  File "/home/airflow/.local/lib/python3.8/site-packages/MySQLdb/cursors.py", line 206, in execute
    res = self._query(query)
  File "/home/airflow/.local/lib/python3.8/site-packages/MySQLdb/cursors.py", line 319, in _query
    db.query(q)
  File "/home/airflow/.local/lib/python3.8/site-packages/MySQLdb/connections.py", line 254, in query
    _mysql.connection.query(self, query)
MySQLdb._exceptions.OperationalError: (3780, "Referencing column 'task_id' and referenced column 'task_id' in foreign key constraint 'task_map_task_instance_fkey' are incompatible.")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/home/airflow/.local/bin/airflow", line 8, in &lt;module&gt;
    sys.exit(main())
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/__main__.py", line 38, in main
    args.func(args)
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/cli_parser.py", line 51, in command
    return func(*args, **kwargs)
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/cli.py", line 99, in wrapper
    return f(*args, **kwargs)
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/db_command.py", line 82, in upgradedb
    db.upgradedb(to_revision=to_revision, from_revision=from_revision, show_sql_only=args.show_sql_only)
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/session.py", line 71, in wrapper
    return func(*args, session=session, **kwargs)
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/db.py", line 1449, in upgradedb
    command.upgrade(config, revision=to_revision or 'heads')
  File "/home/airflow/.local/lib/python3.8/site-packages/alembic/command.py", line 322, in upgrade
    script.run_env()
  File "/home/airflow/.local/lib/python3.8/site-packages/alembic/script/base.py", line 569, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/home/airflow/.local/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 94, in load_python_file
    module = load_module_py(module_id, path)
  File "/home/airflow/.local/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 110, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
  File "&lt;frozen importlib._bootstrap_external&gt;", line 843, in exec_module
  File "&lt;frozen importlib._bootstrap&gt;", line 219, in _call_with_frames_removed
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/migrations/env.py", line 107, in &lt;module&gt;
    run_migrations_online()
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/migrations/env.py", line 101, in run_migrations_online
    context.run_migrations()
  File "&lt;string&gt;", line 8, in run_migrations
  File "/home/airflow/.local/lib/python3.8/site-packages/alembic/runtime/environment.py", line 853, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/home/airflow/.local/lib/python3.8/site-packages/alembic/runtime/migration.py", line 623, in run_migrations
    step.migration_fn(**kw)
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/migrations/versions/0100_2_3_0_add_taskmap_and_map_id_on_taskinstance.py", line 75, in upgrade
    op.create_table(
  File "&lt;string&gt;", line 8, in create_table
  File "&lt;string&gt;", line 3, in create_table
  File "/home/airflow/.local/lib/python3.8/site-packages/alembic/operations/ops.py", line 1254, in create_table
    return operations.invoke(op)
  File "/home/airflow/.local/lib/python3.8/site-packages/alembic/operations/base.py", line 394, in invoke
    return fn(self, operation)
  File "/home/airflow/.local/lib/python3.8/site-packages/alembic/operations/toimpl.py", line 114, in create_table
    operations.impl.create_table(table)
  File "/home/airflow/.local/lib/python3.8/site-packages/alembic/ddl/impl.py", line 354, in create_table
    self._exec(schema.CreateTable(table))
  File "/home/airflow/.local/lib/python3.8/site-packages/alembic/ddl/impl.py", line 195, in _exec
    return conn.execute(construct, multiparams)
  File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1200, in execute
    return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
  File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 77, in _execute_on_connection
    return connection._execute_ddl(
  File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1290, in _execute_ddl
    ret = self._execute_context(
  File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1748, in _execute_context
    self._handle_dbapi_exception(
  File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1929, in _handle_dbapi_exception
    util.raise_(
  File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
    raise exception
  File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1705, in _execute_context
    self.dialect.do_execute(
  File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 716, in do_execute
    cursor.execute(statement, parameters)
  File "/home/airflow/.local/lib/python3.8/site-packages/MySQLdb/cursors.py", line 206, in execute
    res = self._query(query)
  File "/home/airflow/.local/lib/python3.8/site-packages/MySQLdb/cursors.py", line 319, in _query
    db.query(q)
  File "/home/airflow/.local/lib/python3.8/site-packages/MySQLdb/connections.py", line 254, in query
    _mysql.connection.query(self, query)
sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (3780, "Referencing column 'task_id' and referenced column 'task_id' in foreign key constraint 'task_map_task_instance_fkey' are incompatible.")
[SQL: 
CREATE TABLE task_map (
    dag_id VARCHAR(250) COLLATE utf8mb3_bin NOT NULL, 
    task_id VARCHAR(250) COLLATE utf8mb3_bin NOT NULL, 
    run_id VARCHAR(250) COLLATE utf8mb3_bin NOT NULL, 
    map_index INTEGER NOT NULL, 
    length INTEGER NOT NULL, 
    `keys` JSON, 
    PRIMARY KEY (dag_id, task_id, run_id, map_index), 
    CONSTRAINT task_map_length_not_negative CHECK (length &gt;= 0), 
    CONSTRAINT task_map_task_instance_fkey FOREIGN KEY(dag_id, task_id, run_id, map_index) REFERENCES task_instance (dag_id, task_id, run_id, map_index) ON DELETE CASCADE
)
]
(Background on this error at: http://sqlalche.me/e/14/e3q8)
</code></pre></div>
<p dir="auto">Full error Log <strong>after</strong> first execution (should caused by first execution):</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="
/home/airflow/.local/lib/python3.8/site-packages/airflow/configuration.py:529: DeprecationWarning: The auth_backend option in [api[] has been renamed to auth_backends - the old setting has been used, but please update your config.
  option = self._get_option_from_config_file(deprecated_key, deprecated_section, key, kwargs, section)
/home/airflow/.local/lib/python3.8/site-packages/airflow/configuration.py:356: FutureWarning: The auth_backends setting in [api[] has had airflow.api.auth.backend.session added in the running config, which is needed by the UI. Please update your config before Apache Airflow 3.0.
  warnings.warn(
DB: mysql+mysqldb://airflow:***@test-airflow2-db-blue.cndbtlpttl69.eu-central-1.rds.amazonaws.com:3306/airflow
Performing upgrade with database mysql+mysqldb://airflow:***@test-airflow2-db-blue.cndbtlpttl69.eu-central-1.rds.amazonaws.com:3306/airflow
[2022-06-17 12:41:53,882[] {db.py:1448} INFO - Creating tables
INFO  [alembic.runtime.migration[] Context impl MySQLImpl.
INFO  [alembic.runtime.migration[] Will assume non-transactional DDL.
INFO  [alembic.runtime.migration[] Running upgrade f9da662e7089 -&gt; e655c0453f75, Add ``map_index`` column to TaskInstance to identify task-mapping,
and a ``task_map`` table to track mapping values from XCom.
Traceback (most recent call last):
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py&quot;, line 1705, in _execute_context
    self.dialect.do_execute(
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py&quot;, line 716, in do_execute
    cursor.execute(statement, parameters)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/MySQLdb/cursors.py&quot;, line 206, in execute
    res = self._query(query)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/MySQLdb/cursors.py&quot;, line 319, in _query
    db.query(q)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/MySQLdb/connections.py&quot;, line 254, in query
    _mysql.connection.query(self, query)
MySQLdb._exceptions.OperationalError: (1091, &quot;Can't DROP 'task_reschedule_ti_fkey'; check that column/key exists&quot;)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File &quot;/home/airflow/.local/bin/airflow&quot;, line 8, in &lt;module&gt;
    sys.exit(main())
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/airflow/__main__.py&quot;, line 38, in main
    args.func(args)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/cli_parser.py&quot;, line 51, in command
    return func(*args, **kwargs)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/cli.py&quot;, line 99, in wrapper
    return f(*args, **kwargs)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/db_command.py&quot;, line 82, in upgradedb
    db.upgradedb(to_revision=to_revision, from_revision=from_revision, show_sql_only=args.show_sql_only)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/session.py&quot;, line 71, in wrapper
    return func(*args, session=session, **kwargs)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/db.py&quot;, line 1449, in upgradedb
    command.upgrade(config, revision=to_revision or 'heads')
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/alembic/command.py&quot;, line 322, in upgrade
    script.run_env()
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/alembic/script/base.py&quot;, line 569, in run_env
    util.load_python_file(self.dir, &quot;env.py&quot;)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/alembic/util/pyfiles.py&quot;, line 94, in load_python_file
    module = load_module_py(module_id, path)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/alembic/util/pyfiles.py&quot;, line 110, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
  File &quot;&lt;frozen importlib._bootstrap_external&gt;&quot;, line 843, in exec_module
  File &quot;&lt;frozen importlib._bootstrap&gt;&quot;, line 219, in _call_with_frames_removed
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/airflow/migrations/env.py&quot;, line 107, in &lt;module&gt;
    run_migrations_online()
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/airflow/migrations/env.py&quot;, line 101, in run_migrations_online
    context.run_migrations()
  File &quot;&lt;string&gt;&quot;, line 8, in run_migrations
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/alembic/runtime/environment.py&quot;, line 853, in run_migrations
    self.get_context().run_migrations(**kw)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/alembic/runtime/migration.py&quot;, line 623, in run_migrations
    step.migration_fn(**kw)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/airflow/migrations/versions/0100_2_3_0_add_taskmap_and_map_id_on_taskinstance.py&quot;, line 49, in upgrade
    batch_op.drop_index(&quot;idx_task_reschedule_dag_task_run&quot;)
  File &quot;/usr/local/lib/python3.8/contextlib.py&quot;, line 120, in __exit__
    next(self.gen)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/alembic/operations/base.py&quot;, line 376, in batch_alter_table
    impl.flush()
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/alembic/operations/batch.py&quot;, line 111, in flush
    fn(*arg, **kw)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/alembic/ddl/mysql.py&quot;, line 155, in drop_constraint
    super(MySQLImpl, self).drop_constraint(const)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/alembic/ddl/impl.py&quot;, line 338, in drop_constraint
    self._exec(schema.DropConstraint(const))
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/alembic/ddl/impl.py&quot;, line 195, in _exec
    return conn.execute(construct, multiparams)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py&quot;, line 1200, in execute
    return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py&quot;, line 77, in _execute_on_connection
    return connection._execute_ddl(
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py&quot;, line 1290, in _execute_ddl
    ret = self._execute_context(
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py&quot;, line 1748, in _execute_context
    self._handle_dbapi_exception(
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py&quot;, line 1929, in _handle_dbapi_exception
    util.raise_(
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py&quot;, line 211, in raise_
    raise exception
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py&quot;, line 1705, in _execute_context
    self.dialect.do_execute(
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py&quot;, line 716, in do_execute
    cursor.execute(statement, parameters)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/MySQLdb/cursors.py&quot;, line 206, in execute
    res = self._query(query)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/MySQLdb/cursors.py&quot;, line 319, in _query
    db.query(q)
  File &quot;/home/airflow/.local/lib/python3.8/site-packages/MySQLdb/connections.py&quot;, line 254, in query
    _mysql.connection.query(self, query)
sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (1091, &quot;Can't DROP 'task_reschedule_ti_fkey'; check that column/key exists&quot;)
[SQL: ALTER TABLE task_reschedule DROP FOREIGN KEY task_reschedule_ti_fkey[]
(Background on this error at: http://sqlalche.me/e/14/e3q8)"><pre class="notranslate"><code class="notranslate">
/home/airflow/.local/lib/python3.8/site-packages/airflow/configuration.py:529: DeprecationWarning: The auth_backend option in [api[] has been renamed to auth_backends - the old setting has been used, but please update your config.
  option = self._get_option_from_config_file(deprecated_key, deprecated_section, key, kwargs, section)
/home/airflow/.local/lib/python3.8/site-packages/airflow/configuration.py:356: FutureWarning: The auth_backends setting in [api[] has had airflow.api.auth.backend.session added in the running config, which is needed by the UI. Please update your config before Apache Airflow 3.0.
  warnings.warn(
DB: mysql+mysqldb://airflow:***@test-airflow2-db-blue.cndbtlpttl69.eu-central-1.rds.amazonaws.com:3306/airflow
Performing upgrade with database mysql+mysqldb://airflow:***@test-airflow2-db-blue.cndbtlpttl69.eu-central-1.rds.amazonaws.com:3306/airflow
[2022-06-17 12:41:53,882[] {db.py:1448} INFO - Creating tables
INFO  [alembic.runtime.migration[] Context impl MySQLImpl.
INFO  [alembic.runtime.migration[] Will assume non-transactional DDL.
INFO  [alembic.runtime.migration[] Running upgrade f9da662e7089 -&gt; e655c0453f75, Add ``map_index`` column to TaskInstance to identify task-mapping,
and a ``task_map`` table to track mapping values from XCom.
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1705, in _execute_context
    self.dialect.do_execute(
  File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 716, in do_execute
    cursor.execute(statement, parameters)
  File "/home/airflow/.local/lib/python3.8/site-packages/MySQLdb/cursors.py", line 206, in execute
    res = self._query(query)
  File "/home/airflow/.local/lib/python3.8/site-packages/MySQLdb/cursors.py", line 319, in _query
    db.query(q)
  File "/home/airflow/.local/lib/python3.8/site-packages/MySQLdb/connections.py", line 254, in query
    _mysql.connection.query(self, query)
MySQLdb._exceptions.OperationalError: (1091, "Can't DROP 'task_reschedule_ti_fkey'; check that column/key exists")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/home/airflow/.local/bin/airflow", line 8, in &lt;module&gt;
    sys.exit(main())
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/__main__.py", line 38, in main
    args.func(args)
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/cli_parser.py", line 51, in command
    return func(*args, **kwargs)
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/cli.py", line 99, in wrapper
    return f(*args, **kwargs)
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/db_command.py", line 82, in upgradedb
    db.upgradedb(to_revision=to_revision, from_revision=from_revision, show_sql_only=args.show_sql_only)
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/session.py", line 71, in wrapper
    return func(*args, session=session, **kwargs)
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/db.py", line 1449, in upgradedb
    command.upgrade(config, revision=to_revision or 'heads')
  File "/home/airflow/.local/lib/python3.8/site-packages/alembic/command.py", line 322, in upgrade
    script.run_env()
  File "/home/airflow/.local/lib/python3.8/site-packages/alembic/script/base.py", line 569, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/home/airflow/.local/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 94, in load_python_file
    module = load_module_py(module_id, path)
  File "/home/airflow/.local/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 110, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
  File "&lt;frozen importlib._bootstrap_external&gt;", line 843, in exec_module
  File "&lt;frozen importlib._bootstrap&gt;", line 219, in _call_with_frames_removed
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/migrations/env.py", line 107, in &lt;module&gt;
    run_migrations_online()
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/migrations/env.py", line 101, in run_migrations_online
    context.run_migrations()
  File "&lt;string&gt;", line 8, in run_migrations
  File "/home/airflow/.local/lib/python3.8/site-packages/alembic/runtime/environment.py", line 853, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/home/airflow/.local/lib/python3.8/site-packages/alembic/runtime/migration.py", line 623, in run_migrations
    step.migration_fn(**kw)
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/migrations/versions/0100_2_3_0_add_taskmap_and_map_id_on_taskinstance.py", line 49, in upgrade
    batch_op.drop_index("idx_task_reschedule_dag_task_run")
  File "/usr/local/lib/python3.8/contextlib.py", line 120, in __exit__
    next(self.gen)
  File "/home/airflow/.local/lib/python3.8/site-packages/alembic/operations/base.py", line 376, in batch_alter_table
    impl.flush()
  File "/home/airflow/.local/lib/python3.8/site-packages/alembic/operations/batch.py", line 111, in flush
    fn(*arg, **kw)
  File "/home/airflow/.local/lib/python3.8/site-packages/alembic/ddl/mysql.py", line 155, in drop_constraint
    super(MySQLImpl, self).drop_constraint(const)
  File "/home/airflow/.local/lib/python3.8/site-packages/alembic/ddl/impl.py", line 338, in drop_constraint
    self._exec(schema.DropConstraint(const))
  File "/home/airflow/.local/lib/python3.8/site-packages/alembic/ddl/impl.py", line 195, in _exec
    return conn.execute(construct, multiparams)
  File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1200, in execute
    return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
  File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/sql/ddl.py", line 77, in _execute_on_connection
    return connection._execute_ddl(
  File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1290, in _execute_ddl
    ret = self._execute_context(
  File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1748, in _execute_context
    self._handle_dbapi_exception(
  File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1929, in _handle_dbapi_exception
    util.raise_(
  File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
    raise exception
  File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1705, in _execute_context
    self.dialect.do_execute(
  File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 716, in do_execute
    cursor.execute(statement, parameters)
  File "/home/airflow/.local/lib/python3.8/site-packages/MySQLdb/cursors.py", line 206, in execute
    res = self._query(query)
  File "/home/airflow/.local/lib/python3.8/site-packages/MySQLdb/cursors.py", line 319, in _query
    db.query(q)
  File "/home/airflow/.local/lib/python3.8/site-packages/MySQLdb/connections.py", line 254, in query
    _mysql.connection.query(self, query)
sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (1091, "Can't DROP 'task_reschedule_ti_fkey'; check that column/key exists")
[SQL: ALTER TABLE task_reschedule DROP FOREIGN KEY task_reschedule_ti_fkey[]
(Background on this error at: http://sqlalche.me/e/14/e3q8)
</code></pre></div>
<h3 dir="auto">Are you willing to submit PR?</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> Yes I am willing to submit a PR!</li>
</ul>
<h3 dir="auto">Code of Conduct</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I agree to follow this project's <a href="https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md">Code of Conduct</a></li>
</ul>
<h3 dir="auto">Apache Airflow version</h3>
<p dir="auto">2.5.1</p>
<h3 dir="auto">What happened</h3>
<p dir="auto">When declaring a DAG Param, any resulting <code class="notranslate">jsonschema.exceptions.SchemaErrors</code> are not shown in the GUI as a DAG import error, nor visible via <code class="notranslate">airflow dags list-import-errors</code> because the exception is not caught in the <code class="notranslate">models/dagbag.py</code> <code class="notranslate"> _process_modules</code>-function (only <code class="notranslate">ParamValidationError</code> is)</p>
<h3 dir="auto">What you think should happen instead</h3>
<p dir="auto">The function should not only capture <code class="notranslate">ParamValidationError</code>s, but also <code class="notranslate">jsonschema.exceptions.SchemaError</code>. and let you know why the DAG is not being imported.</p>
<h3 dir="auto">How to reproduce</h3>
<p dir="auto">Add a <code class="notranslate">Param</code> to your dag, but use a wrong configuration. In  our case the developer mistakingly used <code class="notranslate">schema</code> instead of <code class="notranslate">type</code>.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="&quot;release&quot;: Param(
  default=None,
  description=&quot;Release to run the update for&quot;,
  schema=&quot;string&quot;
 )"><pre class="notranslate"><code class="notranslate">"release": Param(
  default=None,
  description="Release to run the update for",
  schema="string"
 )
</code></pre></div>
<p dir="auto">The <code class="notranslate">schema</code> line should've been <code class="notranslate">type=["string", "null"]</code>.</p>
<h3 dir="auto">Operating System</h3>
<p dir="auto">Debian</p>
<h3 dir="auto">Versions of Apache Airflow Providers</h3>
<p dir="auto"><em>No response</em></p>
<h3 dir="auto">Deployment</h3>
<p dir="auto">Docker-Compose</p>
<h3 dir="auto">Deployment details</h3>
<p dir="auto">Original airflow docker container (2.5.1 - python 3.10)</p>
<h3 dir="auto">Anything else</h3>
<p dir="auto"><em>No response</em></p>
<h3 dir="auto">Are you willing to submit PR?</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> Yes I am willing to submit a PR!</li>
</ul>
<h3 dir="auto">Code of Conduct</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I agree to follow this project's <a href="https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md">Code of Conduct</a></li>
</ul>