<p dir="auto"><strong>Apache Airflow version</strong>: 2.0.0</p>
<p dir="auto"><strong>Environment</strong>: Docker image <code class="notranslate">apache/airflow:2.0.0-python3.8</code> on Win10 with WSL</p>
<p dir="auto"><strong>What happened</strong>:</p>
<p dir="auto">I built a custom provider with a number of custom connections.</p>
<p dir="auto">This works:</p>
<ul dir="auto">
<li>The connections are properly registered</li>
<li>The UI does not show hidden fields as per <code class="notranslate">get_ui_field_behaviour</code></li>
<li>The UI correctly relabels fields as per <code class="notranslate">get_ui_field_behaviour</code></li>
<li>The UI correctly shows added widgets as per <code class="notranslate">get_connection_form_widgets</code> (well, mostly)</li>
</ul>
<p dir="auto">What does not work:</p>
<ul dir="auto">
<li>The UI does not save values entered for additional widgets</li>
</ul>
<p dir="auto">I used the <a href="https://github.com/apache/airflow/blob/master/airflow/providers/jdbc/hooks/jdbc.py">JDBC example</a> to string myself along by copying it and pasting it as a hook into my custom provider package. (I did not install the JDBC provider package, unless it is installed in the image I use - but if I don't add it in my own provider package, I don't have the connection type in the UI, so I assume it is not). Curiously, The JDBC hook works just fine. I then created the following file:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="&quot;&quot;&quot;
You find two child classes of DbApiHook in here. One is the exact copy of the JDBC
provider hook, minus some irrelevant logic (I only care about the UI stuff here).
The other is the exact same thing, except I added an &quot;x&quot; behind every occurance
of &quot;jdbc&quot; in strings and names.
&quot;&quot;&quot;

from typing import Any, Dict, Optional
from airflow.hooks.dbapi import DbApiHook

class JdbcXHook(DbApiHook):
    &quot;&quot;&quot;
    Copy of JdbcHook below. Added an &quot;x&quot; at various places, including the class name.
    &quot;&quot;&quot;

    conn_name_attr = 'jdbcx_conn_id'  # added x
    default_conn_name = 'jdbcx_default' # added x
    conn_type = 'jdbcx' # added x
    hook_name = 'JDBCx Connection' # added x
    supports_autocommit = True

    @staticmethod
    def get_connection_form_widgets() -&gt; Dict[str, Any]:
        &quot;&quot;&quot;Returns connection widgets to add to connection form&quot;&quot;&quot;
        from flask_appbuilder.fieldwidgets import BS3TextFieldWidget
        from flask_babel import lazy_gettext
        from wtforms import StringField

        # added an x in the keys
        return {
            &quot;extra__jdbcx__drv_path&quot;: StringField(lazy_gettext('Driver Path'), widget=BS3TextFieldWidget()),
            &quot;extra__jdbcx__drv_clsname&quot;: StringField(
                lazy_gettext('Driver Class'), widget=BS3TextFieldWidget()
            ),
        }

    @staticmethod
    def get_ui_field_behaviour() -&gt; Dict:
        &quot;&quot;&quot;Returns custom field behaviour&quot;&quot;&quot;
        return {
            &quot;hidden_fields&quot;: ['port', 'schema', 'extra'],
            &quot;relabeling&quot;: {'host': 'Connection URL'},
        }

class JdbcHook(DbApiHook):
    &quot;&quot;&quot;
    General hook for jdbc db access.
    JDBC URL, username and password will be taken from the predefined connection.
    Note that the whole JDBC URL must be specified in the &quot;host&quot; field in the DB.
    Raises an airflow error if the given connection id doesn't exist.
    &quot;&quot;&quot;

    conn_name_attr = 'jdbc_conn_id'
    default_conn_name = 'jdbc_default'
    conn_type = 'jdbc'
    hook_name = 'JDBC Connection plain'
    supports_autocommit = True

    @staticmethod
    def get_connection_form_widgets() -&gt; Dict[str, Any]:
        &quot;&quot;&quot;Returns connection widgets to add to connection form&quot;&quot;&quot;
        from flask_appbuilder.fieldwidgets import BS3TextFieldWidget
        from flask_babel import lazy_gettext
        from wtforms import StringField

        return {
            &quot;extra__jdbc__drv_path&quot;: StringField(lazy_gettext('Driver Path'), widget=BS3TextFieldWidget()),
            &quot;extra__jdbc__drv_clsname&quot;: StringField(
                lazy_gettext('Driver Class'), widget=BS3TextFieldWidget()
            ),
        }

    @staticmethod
    def get_ui_field_behaviour() -&gt; Dict:
        &quot;&quot;&quot;Returns custom field behaviour&quot;&quot;&quot;
        return {
            &quot;hidden_fields&quot;: ['port', 'schema', 'extra'],
            &quot;relabeling&quot;: {'host': 'Connection URL'},
        }"><pre class="notranslate"><span class="pl-s">"""</span>
<span class="pl-s">You find two child classes of DbApiHook in here. One is the exact copy of the JDBC</span>
<span class="pl-s">provider hook, minus some irrelevant logic (I only care about the UI stuff here).</span>
<span class="pl-s">The other is the exact same thing, except I added an "x" behind every occurance</span>
<span class="pl-s">of "jdbc" in strings and names.</span>
<span class="pl-s">"""</span>

<span class="pl-k">from</span> <span class="pl-s1">typing</span> <span class="pl-k">import</span> <span class="pl-v">Any</span>, <span class="pl-v">Dict</span>, <span class="pl-v">Optional</span>
<span class="pl-k">from</span> <span class="pl-s1">airflow</span>.<span class="pl-s1">hooks</span>.<span class="pl-s1">dbapi</span> <span class="pl-k">import</span> <span class="pl-v">DbApiHook</span>

<span class="pl-k">class</span> <span class="pl-v">JdbcXHook</span>(<span class="pl-v">DbApiHook</span>):
    <span class="pl-s">"""</span>
<span class="pl-s">    Copy of JdbcHook below. Added an "x" at various places, including the class name.</span>
<span class="pl-s">    """</span>

    <span class="pl-s1">conn_name_attr</span> <span class="pl-c1">=</span> <span class="pl-s">'jdbcx_conn_id'</span>  <span class="pl-c"># added x</span>
    <span class="pl-s1">default_conn_name</span> <span class="pl-c1">=</span> <span class="pl-s">'jdbcx_default'</span> <span class="pl-c"># added x</span>
    <span class="pl-s1">conn_type</span> <span class="pl-c1">=</span> <span class="pl-s">'jdbcx'</span> <span class="pl-c"># added x</span>
    <span class="pl-s1">hook_name</span> <span class="pl-c1">=</span> <span class="pl-s">'JDBCx Connection'</span> <span class="pl-c"># added x</span>
    <span class="pl-s1">supports_autocommit</span> <span class="pl-c1">=</span> <span class="pl-c1">True</span>

    <span class="pl-en">@<span class="pl-s1">staticmethod</span></span>
    <span class="pl-k">def</span> <span class="pl-en">get_connection_form_widgets</span>() <span class="pl-c1">-&gt;</span> <span class="pl-v">Dict</span>[<span class="pl-s1">str</span>, <span class="pl-v">Any</span>]:
        <span class="pl-s">"""Returns connection widgets to add to connection form"""</span>
        <span class="pl-k">from</span> <span class="pl-s1">flask_appbuilder</span>.<span class="pl-s1">fieldwidgets</span> <span class="pl-k">import</span> <span class="pl-v">BS3TextFieldWidget</span>
        <span class="pl-k">from</span> <span class="pl-s1">flask_babel</span> <span class="pl-k">import</span> <span class="pl-s1">lazy_gettext</span>
        <span class="pl-k">from</span> <span class="pl-s1">wtforms</span> <span class="pl-k">import</span> <span class="pl-v">StringField</span>

        <span class="pl-c"># added an x in the keys</span>
        <span class="pl-k">return</span> {
            <span class="pl-s">"extra__jdbcx__drv_path"</span>: <span class="pl-v">StringField</span>(<span class="pl-en">lazy_gettext</span>(<span class="pl-s">'Driver Path'</span>), <span class="pl-s1">widget</span><span class="pl-c1">=</span><span class="pl-v">BS3TextFieldWidget</span>()),
            <span class="pl-s">"extra__jdbcx__drv_clsname"</span>: <span class="pl-v">StringField</span>(
                <span class="pl-en">lazy_gettext</span>(<span class="pl-s">'Driver Class'</span>), <span class="pl-s1">widget</span><span class="pl-c1">=</span><span class="pl-v">BS3TextFieldWidget</span>()
            ),
        }

    <span class="pl-en">@<span class="pl-s1">staticmethod</span></span>
    <span class="pl-k">def</span> <span class="pl-en">get_ui_field_behaviour</span>() <span class="pl-c1">-&gt;</span> <span class="pl-v">Dict</span>:
        <span class="pl-s">"""Returns custom field behaviour"""</span>
        <span class="pl-k">return</span> {
            <span class="pl-s">"hidden_fields"</span>: [<span class="pl-s">'port'</span>, <span class="pl-s">'schema'</span>, <span class="pl-s">'extra'</span>],
            <span class="pl-s">"relabeling"</span>: {<span class="pl-s">'host'</span>: <span class="pl-s">'Connection URL'</span>},
        }

<span class="pl-k">class</span> <span class="pl-v">JdbcHook</span>(<span class="pl-v">DbApiHook</span>):
    <span class="pl-s">"""</span>
<span class="pl-s">    General hook for jdbc db access.</span>
<span class="pl-s">    JDBC URL, username and password will be taken from the predefined connection.</span>
<span class="pl-s">    Note that the whole JDBC URL must be specified in the "host" field in the DB.</span>
<span class="pl-s">    Raises an airflow error if the given connection id doesn't exist.</span>
<span class="pl-s">    """</span>

    <span class="pl-s1">conn_name_attr</span> <span class="pl-c1">=</span> <span class="pl-s">'jdbc_conn_id'</span>
    <span class="pl-s1">default_conn_name</span> <span class="pl-c1">=</span> <span class="pl-s">'jdbc_default'</span>
    <span class="pl-s1">conn_type</span> <span class="pl-c1">=</span> <span class="pl-s">'jdbc'</span>
    <span class="pl-s1">hook_name</span> <span class="pl-c1">=</span> <span class="pl-s">'JDBC Connection plain'</span>
    <span class="pl-s1">supports_autocommit</span> <span class="pl-c1">=</span> <span class="pl-c1">True</span>

    <span class="pl-en">@<span class="pl-s1">staticmethod</span></span>
    <span class="pl-k">def</span> <span class="pl-en">get_connection_form_widgets</span>() <span class="pl-c1">-&gt;</span> <span class="pl-v">Dict</span>[<span class="pl-s1">str</span>, <span class="pl-v">Any</span>]:
        <span class="pl-s">"""Returns connection widgets to add to connection form"""</span>
        <span class="pl-k">from</span> <span class="pl-s1">flask_appbuilder</span>.<span class="pl-s1">fieldwidgets</span> <span class="pl-k">import</span> <span class="pl-v">BS3TextFieldWidget</span>
        <span class="pl-k">from</span> <span class="pl-s1">flask_babel</span> <span class="pl-k">import</span> <span class="pl-s1">lazy_gettext</span>
        <span class="pl-k">from</span> <span class="pl-s1">wtforms</span> <span class="pl-k">import</span> <span class="pl-v">StringField</span>

        <span class="pl-k">return</span> {
            <span class="pl-s">"extra__jdbc__drv_path"</span>: <span class="pl-v">StringField</span>(<span class="pl-en">lazy_gettext</span>(<span class="pl-s">'Driver Path'</span>), <span class="pl-s1">widget</span><span class="pl-c1">=</span><span class="pl-v">BS3TextFieldWidget</span>()),
            <span class="pl-s">"extra__jdbc__drv_clsname"</span>: <span class="pl-v">StringField</span>(
                <span class="pl-en">lazy_gettext</span>(<span class="pl-s">'Driver Class'</span>), <span class="pl-s1">widget</span><span class="pl-c1">=</span><span class="pl-v">BS3TextFieldWidget</span>()
            ),
        }

    <span class="pl-en">@<span class="pl-s1">staticmethod</span></span>
    <span class="pl-k">def</span> <span class="pl-en">get_ui_field_behaviour</span>() <span class="pl-c1">-&gt;</span> <span class="pl-v">Dict</span>:
        <span class="pl-s">"""Returns custom field behaviour"""</span>
        <span class="pl-k">return</span> {
            <span class="pl-s">"hidden_fields"</span>: [<span class="pl-s">'port'</span>, <span class="pl-s">'schema'</span>, <span class="pl-s">'extra'</span>],
            <span class="pl-s">"relabeling"</span>: {<span class="pl-s">'host'</span>: <span class="pl-s">'Connection URL'</span>},
        }</pre></div>
<p dir="auto"><strong>What you expected to happen</strong>:</p>
<p dir="auto">After doing the above, I expected</p>
<ul dir="auto">
<li>Seeing both in the add connection UI</li>
<li>Being able to use both the same way</li>
</ul>
<p dir="auto"><strong>What actually happenes</strong>:</p>
<ul dir="auto">
<li>I <em>do</em> see both in the UI (Screenshot 1)</li>
<li>For some reason, the "normal" hook has BOTH extra fields - not just his own two? (Screenshot 2)</li>
<li>If I add the connection as in Screenshot 2, they are saved in the four fields (his own two + the two for the "x" hook) properly as shown in Screenshot 3</li>
<li>If I seek to edit the connection again, they are also they - all four fields - with the correct values in the UI</li>
<li>If I add the connection for the "x" type as in Screenshot 4, it ostensibly saves it - with two fields as defined in the code</li>
<li>You can see in screenshot 5, that the extra is saved as an empty string?!</li>
<li>When trying to edit the connection in the UI, you also see that there is no data saved for two extra widgets?!</li>
<li>I added a few more screenshots of airflow providers CLI command results (note that the package <code class="notranslate">ewah</code> has a number of other custom hooks, and the issue above occurs for <em>all</em> of them)</li>
</ul>
<p dir="auto"><em>Screenshot 1:</em><br>
<a target="_blank" rel="noopener noreferrer nofollow" href="https://user-images.githubusercontent.com/46958547/104121824-9acc6c00-5341-11eb-821c-4bff40a0e7c7.png"><img src="https://user-images.githubusercontent.com/46958547/104121824-9acc6c00-5341-11eb-821c-4bff40a0e7c7.png" alt="image" style="max-width: 100%;"></a></p>
<p dir="auto"><em>Screenshot 2:</em><br>
<a target="_blank" rel="noopener noreferrer nofollow" href="https://user-images.githubusercontent.com/46958547/104121854-c94a4700-5341-11eb-8d3c-80b6380730d9.png"><img src="https://user-images.githubusercontent.com/46958547/104121854-c94a4700-5341-11eb-8d3c-80b6380730d9.png" alt="image" style="max-width: 100%;"></a></p>
<p dir="auto"><em>Screenshot 3:</em><br>
<a target="_blank" rel="noopener noreferrer nofollow" href="https://user-images.githubusercontent.com/46958547/104121912-247c3980-5342-11eb-8030-11c7348309f3.png"><img src="https://user-images.githubusercontent.com/46958547/104121912-247c3980-5342-11eb-8030-11c7348309f3.png" alt="image" style="max-width: 100%;"></a></p>
<p dir="auto"><em>Screenshot 4:</em><br>
<a target="_blank" rel="noopener noreferrer nofollow" href="https://user-images.githubusercontent.com/46958547/104121944-5e4d4000-5342-11eb-83b7-870711ccd367.png"><img src="https://user-images.githubusercontent.com/46958547/104121944-5e4d4000-5342-11eb-83b7-870711ccd367.png" alt="image" style="max-width: 100%;"></a></p>
<p dir="auto"><em>Screenshot 5:</em><br>
<a target="_blank" rel="noopener noreferrer nofollow" href="https://user-images.githubusercontent.com/46958547/104121971-82a91c80-5342-11eb-83b8-fee9386c0c4f.png"><img src="https://user-images.githubusercontent.com/46958547/104121971-82a91c80-5342-11eb-83b8-fee9386c0c4f.png" alt="image" style="max-width: 100%;"></a></p>
<p dir="auto"><em>Screenshot 6 - airflow providers behaviours:</em><br>
<a target="_blank" rel="noopener noreferrer nofollow" href="https://user-images.githubusercontent.com/46958547/104122073-1c70c980-5343-11eb-88f6-6130e5de9e92.png"><img src="https://user-images.githubusercontent.com/46958547/104122073-1c70c980-5343-11eb-88f6-6130e5de9e92.png" alt="image" style="max-width: 100%;"></a></p>
<p dir="auto"><em>Screenshot 7 - airflow providers get:</em><br>
<a target="_blank" rel="noopener noreferrer nofollow" href="https://user-images.githubusercontent.com/46958547/104122092-41fdd300-5343-11eb-9bda-f6849812ba56.png"><img src="https://user-images.githubusercontent.com/46958547/104122092-41fdd300-5343-11eb-9bda-f6849812ba56.png" alt="image" style="max-width: 100%;"></a><br>
(Note: This error occurs with pre-installed providers as well)</p>
<p dir="auto"><em>Screenshot 8 - airflow providers hooks:</em><br>
<a target="_blank" rel="noopener noreferrer nofollow" href="https://user-images.githubusercontent.com/46958547/104122109-65288280-5343-11eb-8322-dda73fef6649.png"><img src="https://user-images.githubusercontent.com/46958547/104122109-65288280-5343-11eb-8322-dda73fef6649.png" alt="image" style="max-width: 100%;"></a></p>
<p dir="auto"><em>Screenshot 9 - aorflow providers list:</em><br>
<a target="_blank" rel="noopener noreferrer nofollow" href="https://user-images.githubusercontent.com/46958547/104122191-c94b4680-5343-11eb-80cf-7f510d4b6e9a.png"><img src="https://user-images.githubusercontent.com/46958547/104122191-c94b4680-5343-11eb-80cf-7f510d4b6e9a.png" alt="image" style="max-width: 100%;"></a></p>
<p dir="auto"><em>Screenshot 10 - airflow providers widgets:</em><br>
<a target="_blank" rel="noopener noreferrer nofollow" href="https://user-images.githubusercontent.com/46958547/104122142-930dc700-5343-11eb-96be-dec43d87a59d.png"><img src="https://user-images.githubusercontent.com/46958547/104122142-930dc700-5343-11eb-96be-dec43d87a59d.png" alt="image" style="max-width: 100%;"></a></p>
<p dir="auto"><strong>How to reproduce it</strong>:</p>
<ul dir="auto">
<li>create a custom provider package</li>
<li>add the code snippet pasted above somewhere</li>
<li>add the two classes to the <code class="notranslate">hook-class-names</code> list in the provider info</li>
<li>install the provider package</li>
<li>do what I described above</li>
</ul>
<h3 dir="auto">Apache Airflow version</h3>
<p dir="auto">2.1.3 (latest released)</p>
<h3 dir="auto">Operating System</h3>
<p dir="auto">Debian GNU/Linux 10 (buster)</p>
<h3 dir="auto">Versions of Apache Airflow Providers</h3>
<p dir="auto">apache-airflow-providers-amazon==2.1.0<br>
apache-airflow-providers-celery==2.0.0<br>
apache-airflow-providers-cncf-kubernetes==2.0.2<br>
apache-airflow-providers-docker==2.1.0<br>
apache-airflow-providers-elasticsearch==2.0.2<br>
apache-airflow-providers-ftp==2.0.0<br>
apache-airflow-providers-google==5.0.0<br>
apache-airflow-providers-grpc==2.0.0<br>
apache-airflow-providers-hashicorp==2.0.0<br>
apache-airflow-providers-http==2.0.0<br>
apache-airflow-providers-imap==2.0.0<br>
apache-airflow-providers-microsoft-azure==3.1.0<br>
apache-airflow-providers-mysql==2.1.0<br>
apache-airflow-providers-postgres==2.0.0<br>
apache-airflow-providers-redis==2.0.0<br>
apache-airflow-providers-sendgrid==2.0.0<br>
apache-airflow-providers-sftp==2.1.0<br>
apache-airflow-providers-slack==4.0.0<br>
apache-airflow-providers-sqlite==2.0.0<br>
apache-airflow-providers-ssh==2.1.0</p>
<h3 dir="auto">Deployment</h3>
<p dir="auto">Other Docker-based deployment</p>
<h3 dir="auto">Deployment details</h3>
<p dir="auto">Base docker image- apache/airflow:2.1.3-python3.7</p>
<h3 dir="auto">What happened</h3>
<p dir="auto">Last Runs for the DAGs on the listing page are not loading because the <code class="notranslate">/last_dagruns</code> endpoint is returning 500 error.</p>
<h3 dir="auto">What you expected to happen</h3>
<p dir="auto">The Last Run for the DAGs should show the latest execution date of each DAG</p>
<h3 dir="auto">How to reproduce</h3>
<ol dir="auto">
<li>Open the DAG listing page</li>
<li>Look at the <code class="notranslate">Last Run</code> column, which would continuously show the loader for each DAG</li>
<li>If you open the Network, you would observe a 500 error for <code class="notranslate">/last_dagruns</code> endpoint</li>
</ol>
<h3 dir="auto">Anything else</h3>
<p dir="auto"><em>No response</em></p>
<h3 dir="auto">Are you willing to submit PR?</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> Yes I am willing to submit a PR!</li>
</ul>
<h3 dir="auto">Code of Conduct</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I agree to follow this project's <a href="https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md">Code of Conduct</a></li>
</ul>