<h3 dir="auto">Description</h3>
<p dir="auto"><code class="notranslate">apache-airflow-providers-google</code> has a lot of dependencies, some pinned to pretty old versions compared to <a href="https://github.com/googleapis/google-cloud-python#libraries">current release versions</a>. In particular, many of these older versions requires <code class="notranslate">protobuf&lt;4.0.0dev</code> whereas if you already have newer versions (say google-cloud-bigquery==3.7.0) it requires newer versions of protobuf and causes incompatibility</p>
<h3 dir="auto">Use case/motivation</h3>
<p dir="auto">Installing <code class="notranslate">dbt-bigquery</code> then <code class="notranslate">apache-airflow-providers-google</code> takes forever to resolve dependencies.</p>
<p dir="auto">Installing <code class="notranslate">apache-airflow-providers-google</code> then <code class="notranslate">dbt-bigquery</code> leads to dependency errors:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
google-cloud-aiplatform 1.16.1 requires google-cloud-bigquery&lt;3.0.0dev,&gt;=1.15.0, but you have google-cloud-bigquery 3.7.0 which is incompatible.
google-ads 18.0.0 requires protobuf!=3.18.*,!=3.19.*,&lt;=3.20.0,&gt;=3.12.0, but you have protobuf 3.20.3 which is incompatible."><pre class="notranslate"><code class="notranslate">ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
google-cloud-aiplatform 1.16.1 requires google-cloud-bigquery&lt;3.0.0dev,&gt;=1.15.0, but you have google-cloud-bigquery 3.7.0 which is incompatible.
google-ads 18.0.0 requires protobuf!=3.18.*,!=3.19.*,&lt;=3.20.0,&gt;=3.12.0, but you have protobuf 3.20.3 which is incompatible.
</code></pre></div>
<h3 dir="auto">Related issues</h3>
<p dir="auto"><em>No response</em></p>
<h3 dir="auto">Are you willing to submit a PR?</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> Yes I am willing to submit a PR!</li>
</ul>
<h3 dir="auto">Code of Conduct</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I agree to follow this project's <a href="https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md">Code of Conduct</a></li>
</ul>
<h3 dir="auto">Apache Airflow Provider(s)</h3>
<p dir="auto">amazon</p>
<h3 dir="auto">Versions of Apache Airflow Providers</h3>
<p dir="auto">apache-airflow-providers-amazon==7.2.0</p>
<h3 dir="auto">Apache Airflow version</h3>
<p dir="auto">2.4.3</p>
<h3 dir="auto">Operating System</h3>
<p dir="auto">Arch Linux</p>
<h3 dir="auto">Deployment</h3>
<p dir="auto">Docker-Compose</p>
<h3 dir="auto">Deployment details</h3>
<p dir="auto">the same for MWAA (aws-managed airflow)</p>
<h3 dir="auto">What happened</h3>
<p dir="auto">exception is raised:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="[2023-02-20, 14:32:02 UTC] {subprocess.py:92} INFO - [�[34m2023-02-20, 14:32:02 UTC�[0m] {�[34mconnectionpool.py:�[0m475} WARNING�[0m - Failed to parse headers url=[https://BUCKET.s3.us-west-2.amazonaws.com:443/object-key.json:[NoBoundaryInMultipartDefect()], unparsed data: ''�[0m
[2023-02-20, 14:32:02 UTC] {subprocess.py:92} INFO - Traceback (most recent call last):
[2023-02-20, 14:32:02 UTC] {subprocess.py:92} INFO -   File &quot;/home/***/.local/lib/python3.7/site-packages/urllib3/connectionpool.py&quot;, line 469, in _make_request
[2023-02-20, 14:32:02 UTC] {subprocess.py:92} INFO -     assert_header_parsing(httplib_response.msg)
[2023-02-20, 14:32:02 UTC] {subprocess.py:92} INFO -   File &quot;/home/***/.local/lib/python3.7/site-packages/urllib3/util/response.py&quot;, line 91, in assert_header_parsing
[2023-02-20, 14:32:02 UTC] {subprocess.py:92} INFO -     raise HeaderParsingError(defects=defects, unparsed_data=unparsed_data)
[2023-02-20, 14:32:02 UTC] {subprocess.py:92} INFO - urllib3.exceptions.HeaderParsingError: [NoBoundaryInMultipartDefect()], unparsed data: ''"><pre class="notranslate"><code class="notranslate">[2023-02-20, 14:32:02 UTC] {subprocess.py:92} INFO - [�[34m2023-02-20, 14:32:02 UTC�[0m] {�[34mconnectionpool.py:�[0m475} WARNING�[0m - Failed to parse headers url=[https://BUCKET.s3.us-west-2.amazonaws.com:443/object-key.json:[NoBoundaryInMultipartDefect()], unparsed data: ''�[0m
[2023-02-20, 14:32:02 UTC] {subprocess.py:92} INFO - Traceback (most recent call last):
[2023-02-20, 14:32:02 UTC] {subprocess.py:92} INFO -   File "/home/***/.local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 469, in _make_request
[2023-02-20, 14:32:02 UTC] {subprocess.py:92} INFO -     assert_header_parsing(httplib_response.msg)
[2023-02-20, 14:32:02 UTC] {subprocess.py:92} INFO -   File "/home/***/.local/lib/python3.7/site-packages/urllib3/util/response.py", line 91, in assert_header_parsing
[2023-02-20, 14:32:02 UTC] {subprocess.py:92} INFO -     raise HeaderParsingError(defects=defects, unparsed_data=unparsed_data)
[2023-02-20, 14:32:02 UTC] {subprocess.py:92} INFO - urllib3.exceptions.HeaderParsingError: [NoBoundaryInMultipartDefect()], unparsed data: ''
</code></pre></div>
<h3 dir="auto">What you think should happen instead</h3>
<p dir="auto">shouldn't be such an exception :)</p>
<h3 dir="auto">How to reproduce</h3>
<p dir="auto">the code that downloads data is simple:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="
def download_from_s3(key: str, bucket_name: str, local_path: str) -&gt; str:
    boto3.set_stream_logger('boto3.resources', logging.DEBUG)
    hook = S3Hook(aws_conn_id='s3_conn')
    file_name = hook.download_file(key=key, bucket_name=bucket_name, preserve_file_name=True)
    return file_name
"><pre class="notranslate"><code class="notranslate">
def download_from_s3(key: str, bucket_name: str, local_path: str) -&gt; str:
    boto3.set_stream_logger('boto3.resources', logging.DEBUG)
    hook = S3Hook(aws_conn_id='s3_conn')
    file_name = hook.download_file(key=key, bucket_name=bucket_name, preserve_file_name=True)
    return file_name

</code></pre></div>
<h3 dir="auto">Anything else</h3>
<p dir="auto">anyway, file is downldaed and looks valid.</p>
<p dir="auto">some logs:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="[�[34m2023-02-20, 15:18:38 UTC�[0m] {�[34mconnection_wrapper.py:�[0m337} INFO�[0m - AWS Connection (conn_id='s3_conn', conn_type='aws') credentials retrieved from login and password.�[0m
2023-02-20, 15:18:38 UTC boto3.resources.factory [DEBUG] Loading s3:s3
[�[34m2023-02-20, 15:18:38 UTC�[0m] {�[34mfactory.py:�[0m66} DEBUG�[0m - Loading s3:s3�[0m
2023-02-20, 15:18:38 UTC boto3.resources.factory [DEBUG] Loading s3:Object
[�[34m2023-02-20, 15:18:38 UTC�[0m] {�[34mfactory.py:�[0m66} DEBUG�[0m - Loading s3:Object�[0m
2023-02-20, 15:18:38 UTC boto3.resources.action [DEBUG] Calling s3:head_object with {'Bucket': 'BUCKET', 'Key': 'object_key.json'}
[�[34m2023-02-20, 15:18:38 UTC�[0m] {�[34maction.py:�[0m85} DEBUG�[0m - Calling s3:head_object with {'Bucket': 'BUCKET', 'Key': 'object_key.json'}�[0m
[�[34m2023-02-20, 15:18:40 UTC�[0m] {�[34mconnectionpool.py:�[0m475} WARNING�[0m - Failed to parse headers (url=https://BUCKET.s3.us-west-2.amazonaws.com:443/object_key.json): [NoBoundaryInMultipartDefect()], unparsed data: ''�[0m
Traceback (most recent call last):
  File &quot;/home/***/.local/lib/python3.7/site-packages/urllib3/connectionpool.py&quot;, line 469, in _make_request
    assert_header_parsing(httplib_response.msg)
  File &quot;/home/***/.local/lib/python3.7/site-packages/urllib3/util/response.py&quot;, line 91, in assert_header_parsing
    raise HeaderParsingError(defects=defects, unparsed_data=unparsed_data)
urllib3.exceptions.HeaderParsingError: [NoBoundaryInMultipartDefect()], unparsed data: ''
2023-02-20, 15:18:40 UTC boto3.resources.action [DEBUG] Response: {'ResponseMetadata': {'RequestId': 'W3J4VRW3WQVV8AV7', 'HostId': 'uRLn/mC6mUAPtgAZRcPbdIlkzWNQ8/AKuPn5HuHjJK1CLNAxfES3DXQsnF7HYSia4guuylFLItY=', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amz-id-2': 'uRLn/mC6mUAPtgAZRcPbdIlkzWNQ8/AKuPn5HuHjJK1CLNAxfES3DXQsnF7HYSia4guuylFLItY=', 'x-amz-request-id': 'W3J4VRW3WQVV8AV7', 'date': 'Mon, 20 Feb 2023 15:18:40 GMT', 'last-modified': 'Thu, 09 Feb 2023 10:34:28 GMT', 'etag': '&quot;e7d2a315e24716624b1085cfa7f31ad8&quot;', 'x-amz-server-side-encryption': 'AES256', 'accept-ranges': 'bytes', 'content-type': 'multipart/form-data', 'server': 'AmazonS3', 'content-length': '7004'}, 'RetryAttempts': 0}, 'AcceptRanges': 'bytes', 'LastModified': datetime.datetime(2023, 2, 9, 10, 34, 28, tzinfo=tzutc()), 'ContentLength': 7004, 'ETag': '&quot;e7d2a315e24716624b1085cfa7f31ad8&quot;', 'ContentType': 'multipart/form-data', 'ServerSideEncryption': 'AES256', 'Metadata': {}}
[�[34m2023-02-20, 15:18:40 UTC�[0m] {�[34maction.py:�[0m90} DEBUG�[0m - Response: {'ResponseMetadata': {'RequestId': 'W3J4VRW3WQVV8AV7', 'HostId': 'uRLn/mC6mUAPtgAZRcPbdIlkzWNQ8/AKuPn5HuHjJK1CLNAxfES3DXQsnF7HYSia4guuylFLItY=', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amz-id-2': 'uRLn/mC6mUAPtgAZRcPbdIlkzWNQ8/AKuPn5HuHjJK1CLNAxfES3DXQsnF7HYSia4guuylFLItY=', 'x-amz-request-id': 'W3J4VRW3WQVV8AV7', 'date': 'Mon, 20 Feb 2023 15:18:40 GMT', 'last-modified': 'Thu, 09 Feb 2023 10:34:28 GMT', 'etag': '&quot;e7d2a315e24716624b1085cfa7f31ad8&quot;', 'x-amz-server-side-encryption': 'AES256', 'accept-ranges': 'bytes', 'content-type': 'multipart/form-data', 'server': 'AmazonS3', 'content-length': '7004'}, 'RetryAttempts': 0}, 'AcceptRanges': 'bytes', 'LastModified': datetime.datetime(2023, 2, 9, 10, 34, 28, tzinfo=tzutc()), 'ContentLength': 7004, 'ETag': '&quot;e7d2a315e24716624b1085cfa7f31ad8&quot;', 'ContentType': 'multipart/form-data', 'ServerSideEncryption': 'AES256', 'Metadata': {}}�[0m
[�[34m2023-02-20, 15:18:40 UTC�[0m] {�[34mconnectionpool.py:�[0m475} WARNING�[0m - Failed to parse headers (url=https://BUCKET.s3.us-west-2.amazonaws.com:443/object_key.json): [NoBoundaryInMultipartDefect()], unparsed data: ''�[0m
Traceback (most recent call last):
  File &quot;/home/***/.local/lib/python3.7/site-packages/urllib3/connectionpool.py&quot;, line 469, in _make_request
    assert_header_parsing(httplib_response.msg)
  File &quot;/home/***/.local/lib/python3.7/site-packages/urllib3/util/response.py&quot;, line 91, in assert_header_parsing
    raise HeaderParsingError(defects=defects, unparsed_data=unparsed_data)
urllib3.exceptions.HeaderParsingError: [NoBoundaryInMultipartDefect()], unparsed data: ''
[�[34m2023-02-20, 15:18:40 UTC�[0m] {�[34mconnectionpool.py:�[0m475} WARNING�[0m - Failed to parse headers (url=https://BUCKET.s3.us-west-2.amazonaws.com:443/object_key.json): [NoBoundaryInMultipartDefect()], unparsed data: ''�[0m
Traceback (most recent call last):
  File &quot;/home/***/.local/lib/python3.7/site-packages/urllib3/connectionpool.py&quot;, line 469, in _make_request
    assert_header_parsing(httplib_response.msg)
  File &quot;/home/***/.local/lib/python3.7/site-packages/urllib3/util/response.py&quot;, line 91, in assert_header_parsing
    raise HeaderParsingError(defects=defects, unparsed_data=unparsed_data)
urllib3.exceptions.HeaderParsingError: [NoBoundaryInMultipartDefect()], unparsed data: ''"><pre class="notranslate"><code class="notranslate">[�[34m2023-02-20, 15:18:38 UTC�[0m] {�[34mconnection_wrapper.py:�[0m337} INFO�[0m - AWS Connection (conn_id='s3_conn', conn_type='aws') credentials retrieved from login and password.�[0m
2023-02-20, 15:18:38 UTC boto3.resources.factory [DEBUG] Loading s3:s3
[�[34m2023-02-20, 15:18:38 UTC�[0m] {�[34mfactory.py:�[0m66} DEBUG�[0m - Loading s3:s3�[0m
2023-02-20, 15:18:38 UTC boto3.resources.factory [DEBUG] Loading s3:Object
[�[34m2023-02-20, 15:18:38 UTC�[0m] {�[34mfactory.py:�[0m66} DEBUG�[0m - Loading s3:Object�[0m
2023-02-20, 15:18:38 UTC boto3.resources.action [DEBUG] Calling s3:head_object with {'Bucket': 'BUCKET', 'Key': 'object_key.json'}
[�[34m2023-02-20, 15:18:38 UTC�[0m] {�[34maction.py:�[0m85} DEBUG�[0m - Calling s3:head_object with {'Bucket': 'BUCKET', 'Key': 'object_key.json'}�[0m
[�[34m2023-02-20, 15:18:40 UTC�[0m] {�[34mconnectionpool.py:�[0m475} WARNING�[0m - Failed to parse headers (url=https://BUCKET.s3.us-west-2.amazonaws.com:443/object_key.json): [NoBoundaryInMultipartDefect()], unparsed data: ''�[0m
Traceback (most recent call last):
  File "/home/***/.local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 469, in _make_request
    assert_header_parsing(httplib_response.msg)
  File "/home/***/.local/lib/python3.7/site-packages/urllib3/util/response.py", line 91, in assert_header_parsing
    raise HeaderParsingError(defects=defects, unparsed_data=unparsed_data)
urllib3.exceptions.HeaderParsingError: [NoBoundaryInMultipartDefect()], unparsed data: ''
2023-02-20, 15:18:40 UTC boto3.resources.action [DEBUG] Response: {'ResponseMetadata': {'RequestId': 'W3J4VRW3WQVV8AV7', 'HostId': 'uRLn/mC6mUAPtgAZRcPbdIlkzWNQ8/AKuPn5HuHjJK1CLNAxfES3DXQsnF7HYSia4guuylFLItY=', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amz-id-2': 'uRLn/mC6mUAPtgAZRcPbdIlkzWNQ8/AKuPn5HuHjJK1CLNAxfES3DXQsnF7HYSia4guuylFLItY=', 'x-amz-request-id': 'W3J4VRW3WQVV8AV7', 'date': 'Mon, 20 Feb 2023 15:18:40 GMT', 'last-modified': 'Thu, 09 Feb 2023 10:34:28 GMT', 'etag': '"e7d2a315e24716624b1085cfa7f31ad8"', 'x-amz-server-side-encryption': 'AES256', 'accept-ranges': 'bytes', 'content-type': 'multipart/form-data', 'server': 'AmazonS3', 'content-length': '7004'}, 'RetryAttempts': 0}, 'AcceptRanges': 'bytes', 'LastModified': datetime.datetime(2023, 2, 9, 10, 34, 28, tzinfo=tzutc()), 'ContentLength': 7004, 'ETag': '"e7d2a315e24716624b1085cfa7f31ad8"', 'ContentType': 'multipart/form-data', 'ServerSideEncryption': 'AES256', 'Metadata': {}}
[�[34m2023-02-20, 15:18:40 UTC�[0m] {�[34maction.py:�[0m90} DEBUG�[0m - Response: {'ResponseMetadata': {'RequestId': 'W3J4VRW3WQVV8AV7', 'HostId': 'uRLn/mC6mUAPtgAZRcPbdIlkzWNQ8/AKuPn5HuHjJK1CLNAxfES3DXQsnF7HYSia4guuylFLItY=', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amz-id-2': 'uRLn/mC6mUAPtgAZRcPbdIlkzWNQ8/AKuPn5HuHjJK1CLNAxfES3DXQsnF7HYSia4guuylFLItY=', 'x-amz-request-id': 'W3J4VRW3WQVV8AV7', 'date': 'Mon, 20 Feb 2023 15:18:40 GMT', 'last-modified': 'Thu, 09 Feb 2023 10:34:28 GMT', 'etag': '"e7d2a315e24716624b1085cfa7f31ad8"', 'x-amz-server-side-encryption': 'AES256', 'accept-ranges': 'bytes', 'content-type': 'multipart/form-data', 'server': 'AmazonS3', 'content-length': '7004'}, 'RetryAttempts': 0}, 'AcceptRanges': 'bytes', 'LastModified': datetime.datetime(2023, 2, 9, 10, 34, 28, tzinfo=tzutc()), 'ContentLength': 7004, 'ETag': '"e7d2a315e24716624b1085cfa7f31ad8"', 'ContentType': 'multipart/form-data', 'ServerSideEncryption': 'AES256', 'Metadata': {}}�[0m
[�[34m2023-02-20, 15:18:40 UTC�[0m] {�[34mconnectionpool.py:�[0m475} WARNING�[0m - Failed to parse headers (url=https://BUCKET.s3.us-west-2.amazonaws.com:443/object_key.json): [NoBoundaryInMultipartDefect()], unparsed data: ''�[0m
Traceback (most recent call last):
  File "/home/***/.local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 469, in _make_request
    assert_header_parsing(httplib_response.msg)
  File "/home/***/.local/lib/python3.7/site-packages/urllib3/util/response.py", line 91, in assert_header_parsing
    raise HeaderParsingError(defects=defects, unparsed_data=unparsed_data)
urllib3.exceptions.HeaderParsingError: [NoBoundaryInMultipartDefect()], unparsed data: ''
[�[34m2023-02-20, 15:18:40 UTC�[0m] {�[34mconnectionpool.py:�[0m475} WARNING�[0m - Failed to parse headers (url=https://BUCKET.s3.us-west-2.amazonaws.com:443/object_key.json): [NoBoundaryInMultipartDefect()], unparsed data: ''�[0m
Traceback (most recent call last):
  File "/home/***/.local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 469, in _make_request
    assert_header_parsing(httplib_response.msg)
  File "/home/***/.local/lib/python3.7/site-packages/urllib3/util/response.py", line 91, in assert_header_parsing
    raise HeaderParsingError(defects=defects, unparsed_data=unparsed_data)
urllib3.exceptions.HeaderParsingError: [NoBoundaryInMultipartDefect()], unparsed data: ''
</code></pre></div>
<h3 dir="auto">Are you willing to submit PR?</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> Yes I am willing to submit a PR!</li>
</ul>
<h3 dir="auto">Code of Conduct</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I agree to follow this project's <a href="https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md">Code of Conduct</a></li>
</ul>