<p dir="auto">After <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="818258641" data-permission-text="Title is private" data-url="https://github.com/apache/airflow/issues/14531" data-hovercard-type="pull_request" data-hovercard-url="/apache/airflow/pull/14531/hovercard" href="https://github.com/apache/airflow/pull/14531">#14531</a> we are running the CI infrastructiure of ours using <code class="notranslate">r5a.2xlarge</code> machines. They are needed for tests (thanks to having 8xCPUs and 64 GB RAM we can run full suite of tests in 15 minutes instead of 1h by running the tests in parallel. Likely (due to ~ 2x less cost of running tests (2x more price/ 4x less time) we are saving money. We also achieve 30%-40% speedups on Pylint, Static Checks (thanks to 8 CPUS available).</p>
<p dir="auto">However not all jobs can use the memory/parallelism and we can use much cheaper machines without impacting CI execution time:</p>
<ul dir="auto">
<li>
<p dir="auto">Docs (not yet but possible)</p>
</li>
<li>
<p dir="auto">K8S tests  (not yet but possible)</p>
</li>
<li>
<p dir="auto">Building  Images</p>
</li>
<li>
<p dir="auto">Providers building and testing</p>
</li>
<li>
<p dir="auto">Image verification</p>
</li>
<li>
<p dir="auto">Image waiting (this one is particularly wasteful)</p>
</li>
</ul>
<p dir="auto">By introducing a smaller type of machines and assigning the "big" machine type only to the jobs that can utilise higher we can drive the cost down significantly.</p>
<p dir="auto">Small machine = r5a.large with 2 CPUs and 16GB memory should likely be enough</p>
<p dir="auto">Spot Price in Frankfurt:</p>
<p dir="auto">r5.2xlarge = $0.1549 per Hour<br>
r5a.large  = $0.0365 per Hour</p>
<p dir="auto">This means that for non-test jobs we can have further 4 x smaller cost if we do it.</p>
<p dir="auto">Greenplum DB is a popular open source relational DBMS[1].</p>
<p dir="auto">I want to implement some database specific behaviour for a common class. It's easy to check connection type for postgres, mysql or oracle. Now I use postgres connection type for Greenplum and cannot explicitly check type.</p>
<p dir="auto">Airflow has <code class="notranslate">PostgresHook</code> based on <code class="notranslate">psycopg2</code>. As Greenplum supports PostgreSQL libpq API [2], <code class="notranslate">PostgresHook</code> could be reused for Greenplum.</p>
<p dir="auto">As I see it's enough to change 2 code line in the <code class="notranslate">connectoin.py</code>.</p>
<p dir="auto">I'm not sure that it's nice to reuse <code class="notranslate">Hook</code>.<br>
Please confirm or reject my idea.</p>
<div class="highlight highlight-source-diff notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="Index: airflow/models/connection.py
===================================================================
--- airflow/models/connection.py  (revision 6416d898060706787861ff8ecbc4363152a35f45)
+++ airflow/models/connection.py  (date 1598717245079)
@@ -109,6 +109,7 @@
         ('grpc', 'GRPC Connection'),
         ('yandexcloud', 'Yandex Cloud'),
         ('spark', 'Spark'),
+        ('greenplum', 'Greenplum'),
     ]
 
     def __init__(
@@ -243,7 +244,7 @@
         elif self.conn_type == 'google_cloud_platform':
             from airflow.contrib.hooks.bigquery_hook import BigQueryHook
             return BigQueryHook(bigquery_conn_id=self.conn_id)
-        elif self.conn_type == 'postgres':
+        elif self.conn_type in ('postgres', 'greenplum'):
             from airflow.hooks.postgres_hook import PostgresHook
             return PostgresHook(postgres_conn_id=self.conn_id)
         elif self.conn_type == 'pig_cli':"><pre class="notranslate">Index: airflow/models/connection.py
<span class="pl-ms">===================================================================</span>
<span class="pl-md">--- airflow/models/connection.py  (revision 6416d898060706787861ff8ecbc4363152a35f45)</span>
<span class="pl-mi1">+++ airflow/models/connection.py  (date 1598717245079)</span>
<span class="pl-mdr">@@ -109,6 +109,7 @@</span>
         ('grpc', 'GRPC Connection'),
         ('yandexcloud', 'Yandex Cloud'),
         ('spark', 'Spark'),
<span class="pl-mi1"><span class="pl-mi1">+</span>        ('greenplum', 'Greenplum'),</span>
     ]
 
     def __init__(
<span class="pl-mdr">@@ -243,7 +244,7 @@</span>
         elif self.conn_type == 'google_cloud_platform':
             from airflow.contrib.hooks.bigquery_hook import BigQueryHook
             return BigQueryHook(bigquery_conn_id=self.conn_id)
<span class="pl-md"><span class="pl-md">-</span>        elif self.conn_type == 'postgres':</span>
<span class="pl-mi1"><span class="pl-mi1">+</span>        elif self.conn_type in ('postgres', 'greenplum'):</span>
             from airflow.hooks.postgres_hook import PostgresHook
             return PostgresHook(postgres_conn_id=self.conn_id)
         elif self.conn_type == 'pig_cli':</pre></div>
<hr>
<ol dir="auto">
<li><a href="https://github.com/greenplum-db/gpdb">https://github.com/greenplum-db/gpdb</a></li>
<li><a href="https://gpdb.docs.pivotal.io/6-10/security-guide/topics/ports_and_protocols.html" rel="nofollow">https://gpdb.docs.pivotal.io/6-10/security-guide/topics/ports_and_protocols.html</a></li>
</ol>