Answers: <p>The best way to have AWS CLI tooling installed to your custom agent is by installing from Visual Studio Marketplace. AWS CLI is part of AWS Tools for Azure DevOps Service and Azure DevOps Server (the on-premise Azure DevOps) under the name of "AWS Tools for Microsoft Visual Studio Team Services".</p>
  
  <p>Installing this AWS Tools will enable AWS tooling on CI (build pipeline) and CD (release pipeline).</p>
  
  <p>This is the official landing page of AWS Tools in Visual Studio Marketplace: 
  <a href="https://marketplace.visualstudio.com/items?itemName=AmazonWebServices.aws-vsts-tools" rel="nofollow noreferrer">https://marketplace.visualstudio.com/items?itemName=AmazonWebServices.aws-vsts-tools</a></p>
  
  <p>Official documentation is available at: <a href="https://docs.aws.amazon.com/vsts/index.html" rel="nofollow noreferrer">https://docs.aws.amazon.com/vsts/index.html</a></p>
  
Answers: CC BY-SA 4.0
Answers: 2019-11-25T22:11:09.527Z
Answers: Thanks - that would be the solution "c" as suggested above. Unfortunately, I need the CLI, because our existing scripts use them. The aws vsts tools do not contain the aws cli. This would therefore mean that we have to migrate everything. Any other suggestion?
Answers: hey
Answers: CC BY-SA 4.0
Answers: 2019-11-26T04:27:06.070Z
Answers: The AWS Toos for Azure DevOps _**does have the AWS CLI**_ as long as your build agent runs on Windows. Please read the description of AWS Tools on the Visual Studio Marketplace link.
Answers: Eriawan Kusumawardhono
Answers: CC BY-SA 4.0
Answers: 2019-11-25T08:47:25.880Z
Answers: 2019-11-25T08:47:25.880Z
Answers: Eriawan Kusumawardhono
Body: <p>I have quite some difficulties installing <strong>aws cli</strong> on a self-hosted Azure DevOps Windows agent as part of a pipeline. Does anyone have a working solution?</p>
  
  <h2>What I have tried so far:</h2>
  
  <h3>a) install aws cli via pip</h3>
  
  <p>Firstly, I install Python (tried both; Python2 and Python3) within the pipeline (Powershell) by downloading and extracting a tar file. Example:</p>
  
  <pre><code>(New-Object System.Net.WebClient).DownloadFile("$(python3_download_url)", ".\Python.tar")
  Expand-7Zip .\Python.tar .
  </code></pre>
  
  <p>Secondly, I use the freshly installed Python for installing aws cli. Example:</p>
  
  <pre><code>$(PYTHON364EXE) -m pip install awscli
  </code></pre>
  
  <p>Aws cli is then installed in the <code>SCRIPTS</code> directory, as expected. </p>
  
  <p>When I try to execute aws.cmd, I receive an error message:</p>
  
  <pre><code>File association not found for extension .py
  '""' is not recognized as an internal or external command,
  operable program or batch file
  </code></pre>
  
  <p>Similar issues are described here: <a href="https://stackoverflow.com/questions/47459536/file-association-not-found-for-extension-py">1</a>, <a href="https://github.com/awslabs/aws-shell/issues/83" rel="nofollow noreferrer">2</a>, <a href="https://github.com/aws/aws-cli/issues/3518" rel="nofollow noreferrer">3</a>. The suggested solution is, to associate python files with the freshly installed python executable:</p>
  
  <pre><code>assoc .py=pyautofile
  ftype pyautofile="$(PYTHON364EXE)" "%1" %*
  </code></pre>
  
  <p>I do not feel this is appropriate for an azure agent:</p>
  
  <ol>
  <li>It does not feel right to use Admin permissions for a simple build</li>
  <li>I am worried that the proposed change would impact <em>all</em> agents and the entire host server, while I need it for only one specific agent.</li>
  </ol>
  
  <p>Another option would be, to install the required tools system-wide (python, aws). That would, however, make it difficult to try out a specific (new?) version of python or aws cli.</p>
  
  <h3>b) install aws cli via extracted msi package</h3>
  
  <p>... extracted the contents of the msi package via <code>lessmsi</code>, then run the aws cli:</p>
  
  <pre><code>C:\AWS&gt; aws
  failed to create process.
  </code></pre>
  
  <p>This obviously does not work either. - It actually does work properly with aws cli version 2, but not with aws cli version 1. Version 2 is not recommended for use in production yet.</p>
  
  <h3>c) Use AWS Tools for Microsoft Visual Studio Team Services</h3>
  
  <p>This is an extension for Azure DevOps, maintained by AWS. It works, but a) we have legacy scripts of a certain complexity using aws cli we cannot migrate, and b) there might be features available in aws cli which are not available in the extension.</p>
  
  <h3>d) call aws as python module</h3>
  
  <pre><code>$(PYTHON364EXE) -m awscli --version
  </code></pre>
  
  <p>This works, but it means I would have to adapt and modify the legacy script.</p>
  
Comments: CC BY-SA 4.0
Comments: 2021-03-01T16:54:11.363Z
Comments: Were you able to come up with a solution? I am trying to run the AWS CLI command `aws s3 sync <s3 source> <azure data lake dest>`. From the research, I've done, it appears that boto3 doesn't support the `sync` command. I have thought about a few different design patterns (including spinning up an Azure Data Science VM), but haven't come up with an elegant, simple, and cost efficient solution.
Comments: datalifenyc
Tags: <azure-devops><python><azure-pipelines><aws-cli>
Title: How to install aws cli on a self-hosted azure DevOps Windows agent?
UserDisplayName: hey
ThreadText: # How to install aws cli on a self-hosted azure DevOps Windows agent?
  
  (Asked by: username_0 on 2019-11-20)
  
  I have quite some difficulties installing **aws cli** on a self-hosted Azure DevOps Windows agent as part of a pipeline. Does anyone have a working solution?
  
  ## What I have tried so far:
  
  ### a) install aws cli via pip
  
  Firstly, I install Python (tried both; Python2 and Python3) within the pipeline (Powershell) by downloading and extracting a tar file. Example:
  
  ```
  (New-Object System.Net.WebClient).DownloadFile("$(python3_download_url)", ".\Python.tar")
  Expand-7Zip .\Python.tar .
  
  ```
  
  Secondly, I use the freshly installed Python for installing aws cli. Example:
  
  ```
  $(PYTHON364EXE) -m pip install awscli
  
  ```
  
  Aws cli is then installed in the `SCRIPTS` directory, as expected.
  
  When I try to execute aws.cmd, I receive an error message:
  
  ```
  File association not found for extension .py
  '""' is not recognized as an internal or external command,
  operable program or batch file
  
  ```
  
  Similar issues are described here: [1](https://stackoverflow.com/questions/47459536/file-association-not-found-for-extension-py), [2](https://github.com/awslabs/aws-shell/issues/83), [3](https://github.com/aws/aws-cli/issues/3518). The suggested solution is, to associate python files with the freshly installed python executable:
  
  ```
  assoc .py=pyautofile
  ftype pyautofile="$(PYTHON364EXE)" "%1" %*
  
  ```
  
  I do not feel this is appropriate for an azure agent:
  
  1. It does not feel right to use Admin permissions for a simple build
  2. I am worried that the proposed change would impact *all* agents and the entire host server, while I need it for only one specific agent.
  
  Another option would be, to install the required tools system-wide (python, aws). That would, however, make it difficult to try out a specific (new?) version of python or aws cli.
  
  ### b) install aws cli via extracted msi package
  
  ... extracted the contents of the msi package via `lessmsi`, then run the aws cli:
  
  ```
  C:\AWS> aws
  failed to create process.
  
  ```
  
  This obviously does not work either. - It actually does work properly with aws cli version 2, but not with aws cli version 1. Version 2 is not recommended for use in production yet.
  
  ### c) Use AWS Tools for Microsoft Visual Studio Team Services
  
  This is an extension for Azure DevOps, maintained by AWS. It works, but a) we have legacy scripts of a certain complexity using aws cli we cannot migrate, and b) there might be features available in aws cli which are not available in the extension.
  
  ### d) call aws as python module
  
  ```
  $(PYTHON364EXE) -m awscli --version
  
  ```
  
  This works, but it means I would have to adapt and modify the legacy script.
  
  **Tags:** azure-devops, python, azure-pipelines, aws-cli
  
  > **Comments:**
  > **username_1**: Were you able to come up with a solution? I am trying to run the AWS CLI command `aws s3 sync <s3 source> <azure data lake dest>`. From the research, I've done, it appears that boto3 doesn't support the `sync` command. I have thought about a few different design patterns (including spinning up an Azure Data Science VM), but haven't come up with an elegant, simple, and cost efficient solution.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2019-11-25. Score: 3
  
  The best way to have AWS CLI tooling installed to your custom agent is by installing from Visual Studio Marketplace. AWS CLI is part of AWS Tools for Azure DevOps Service and Azure DevOps Server (the on-premise Azure DevOps) under the name of "AWS Tools for Microsoft Visual Studio Team Services".
  
  Installing this AWS Tools will enable AWS tooling on CI (build pipeline) and CD (release pipeline).
  
  This is the official landing page of AWS Tools in Visual Studio Marketplace:
  <https://marketplace.visualstudio.com/items?itemName=AmazonWebServices.aws-vsts-tools>
  
  Official documentation is available at: <https://docs.aws.amazon.com/vsts/index.html>
  
  > **Comments:**
  > **username_0**: Thanks - that would be the solution "c" as suggested above. Unfortunately, I need the CLI, because our existing scripts use them. The aws vsts tools do not contain the aws cli. This would therefore mean that we have to migrate everything. Any other suggestion?  
  > **username_2**: The AWS Toos for Azure DevOps _**does have the AWS CLI**_ as long as your build agent runs on Windows. Please read the description of AWS Tools on the Visual Studio Marketplace link.  
  
  ---
  
  