Body: <p>I am currently creating a concept for a multi-stage release pipeline using Azure DevOps release pipelines. The software stack that should be deployed is a rather bulky PHP application that uses lots of external libraries. As far as this questions goes, using Docker to ship the application is (sadly) not an option.</p>
  <p>In this context, I wonder how to best publish some commit to lots of linux servers (dozenzs). The servers themselves are set up using Ansible and are virtually the same. The last part I don't know how to do is how to push the PHP codebase (approx. 700MB including external libs used) to all these servers.</p>
  <p>Some ideas that all seem kind of &quot;hacky&quot;:</p>
  <ul>
  <li>Zip the application, copy via SSH, unzip, done</li>
  <li>Configure a Git connection on each of the servers. Then, simply trigger a <code>git checkout commit</code> or something.</li>
  <li>Just publish the application in the release pipeline on some semi-public repo and have all servers periodically do a <code>git pull</code></li>
  <li><em>Edit</em>: Somehow use ansible?</li>
  </ul>
  <p>Maybe someone has an idea on how to handle this in a more or less stable and performant way.</p>
  
Comments: CC BY-SA 4.0
Comments: 2021-10-30T00:16:41.413Z
Comments: The idea of cloning a Git repo on the production(?) servers and update them by fetching just deltas from a dev repo is nice. But it may raise a security issue: If you can _fetch/pull from_ a remote repo you usually can also _push to_ a remote repo. So there has to be, for instance, a firewall rule that blocks inbound traffic to the dev repo server(s) from the prod servers content-based.
Comments: Gerold Broser
Comments: CC BY-SA 4.0
Comments: 2021-10-30T10:49:04.440Z
Comments: Can you elaborate why containerization is not an option?
Comments: taleodor
Comments: CC BY-SA 4.0
Comments: 2021-10-31T23:20:07.937Z
Comments: Have you considered simply having a staging location, which could be a git repo, then [`rsync`](https://linux.die.net/man/1/rsync) (or [`rdist`](https://linux.die.net/man/1/rdist)) to the target servers? I assume most of your files don't change, so only the initial sync would take a lot of time. rsync becomes inefficient if it's 100Ks or 1Ms of files tho. Piping `tar` is also an option.
Comments: Ian W
Comments: CC BY-SA 4.0
Comments: 2021-11-01T10:40:45.070Z
Comments: @taledor Trust me, I'd love to go with containers, but right now company does not have enough resources to establish that :/
  
  edit: That staging repo would be deployed near the actual deployment, right? Either manually or via a pipeline commits that would be pushed to that git repo and a job on deployment machines would trigger a pull on the staging repo. Is that apporximately the idea?
Comments: FabianTe
Comments: CC BY-SA 4.0
Comments: 2021-11-19T14:08:50.377Z
Comments: For your 3rd idea you could publish your app to Azure Artifacts which is a part of Azure DevOps and then use Ansible to call Azure CLI to fetch the package and then install it. This SO answer may help with using Azure CLI from Ansible https://stackoverflow.com/a/51318002/184220
Comments: Aquila Sands
Tags: <azure-devops><git>
Title: How to push large codebase to lots of servers
UserDisplayName: FabianTe
ThreadText: # How to push large codebase to lots of servers
  
  (Asked by: username_0 on 2021-10-28)
  
  I am currently creating a concept for a multi-stage release pipeline using Azure DevOps release pipelines. The software stack that should be deployed is a rather bulky PHP application that uses lots of external libraries. As far as this questions goes, using Docker to ship the application is (sadly) not an option.
  
  In this context, I wonder how to best publish some commit to lots of linux servers (dozenzs). The servers themselves are set up using Ansible and are virtually the same. The last part I don't know how to do is how to push the PHP codebase (approx. 700MB including external libs used) to all these servers.
  
  Some ideas that all seem kind of "hacky":
  
  * Zip the application, copy via SSH, unzip, done
  * Configure a Git connection on each of the servers. Then, simply trigger a `git checkout commit` or something.
  * Just publish the application in the release pipeline on some semi-public repo and have all servers periodically do a `git pull`
  * *Edit*: Somehow use ansible?
  
  Maybe someone has an idea on how to handle this in a more or less stable and performant way.
  
  **Tags:** azure-devops, git
  
  > **Comments:**
  > **username_1**: The idea of cloning a Git repo on the production(?) servers and update them by fetching just deltas from a dev repo is nice. But it may raise a security issue: If you can _fetch/pull from_ a remote repo you usually can also _push to_ a remote repo. So there has to be, for instance, a firewall rule that blocks inbound traffic to the dev repo server(s) from the prod servers content-based.  
  > **username_2**: Can you elaborate why containerization is not an option?  
  > **username_3**: Have you considered simply having a staging location, which could be a git repo, then [`rsync`](https://linux.die.net/man/1/rsync) (or [`rdist`](https://linux.die.net/man/1/rdist)) to the target servers? I assume most of your files don't change, so only the initial sync would take a lot of time. rsync becomes inefficient if it's 100Ks or 1Ms of files tho. Piping `tar` is also an option.  
  > **username_0**: @taledor Trust me, I'd love to go with containers, but right now company does not have enough resources to establish that :/  edit: That staging repo would be deployed near the actual deployment, right? Either manually or via a pipeline commits that would be pushed to that git repo and a job on deployment machines would trigger a pull on the staging repo. Is that apporximately the idea?  
  > **username_4**: For your 3rd idea you could publish your app to Azure Artifacts which is a part of Azure DevOps and then use Ansible to call Azure CLI to fetch the package and then install it. This SO answer may help with using Azure CLI from Ansible https://stackoverflow.com/a/51318002/184220  
  
  ---
  
  ## 0 answers
  
  