Answers: <p>I found out of what happening here. I hit the limit of the IP amount in the ingress network. Default ingress network for Docker Swarm when you create it is 254 IP's its landing in the IP range /24</p>
  <p>So what I do its change this IP range to /16 and now get 65k IP's</p>
  <p>Every published port you are using multiply with replicates take a single IP so in my case when I got 109 services with avg. replicates its taking 218 IP's out of the default IP range.</p>
  <p>So after this change everything are working again in my Docker Swarm setup.</p>
  <p>Here is the command to change your ingress network from /24 to /16</p>
  <pre><code>docker network rm ingress
  docker network create \
    --driver overlay \
    --ingress \
    --subnet=10.11.0.0/16 \
    --gateway=10.11.0.2 \
    --opt com.docker.network.driver.mtu=1200 \
    ingress
  </code></pre>
  
Answers: CC BY-SA 4.0
Answers: 2021-08-18T13:02:02.250Z
Answers: 2021-08-18T13:02:02.250Z
Answers: ParisNakitaKejser
Body: <p>I got a Docker Swarm with 3 nodes, and contain around 40-50 stack on it, now its random stop working to redeploy my new changes its just say in <code>new</code> mode, and really don't know what going one here.</p>
  <pre><code>ID             NAME                            IMAGE                                                               NODE                                DESIRED STATE   CURRENT STATE                ERROR
                          PORTS
  1wyc57lkgebc   translation-manager_web.1       {images}                                       Running         New 6 minutes ago
  
  6xthicsqmkst    \_ translation-manager_web.1   {images}   docker-swarm-02   Shutdown        Shutdown 6 minutes ago
  </code></pre>
  <p>Normal its taking like 20-30sec to redeploy new changes to my Docker Swarm setup.</p>
  <p>If I'm running the <code>service update</code> command its just stay like this and never ending</p>
  <pre><code>translation-manager_web
  overall progress: 0 out of 1 tasks
  1/1: new       [=====&gt;                                             ]
  </code></pre>
  <p>Its look like its random witch service / container there hit this error.</p>
  <p>I got 50% free disk space, lots of memory and free CPU that's the first I think its wrong.</p>
  <p>Swarm node info:</p>
  <pre><code>ID                            HOSTNAME                                    STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSION
  6n19d0ryoon2zy4s295s928pp     staging-docker-swarm-01           Ready     Active                          20.10.8
  8se5pzi4znzde6qtlmu0yt0og     staging-docker-swarm-02           Ready     Active                          20.10.8
  p7srx7ojo50rmdxtk6ppriomd     staging-docker-swarm-03           Ready     Active                          20.10.8
  yf8ubbk650rrkmrwkyk3no8zb     staging-docker-swarm-04           Ready     Active                          20.10.8
  f3acixkd61015h0kqgp61ilav *   staging-docker-swarm-manager-01   Ready     Active         Leader           20.10.8
  </code></pre>
  <p><strong>-- Update: 18/08/2021</strong></p>
  <p>I have added one more worker to my swarm, and 3 new manager to it, and its its still the same issue.</p>
  <p>I have 62 Stacks with 109 services running in my swarm right now if the numbers helping.</p>
  
Tags: <docker><containers><docker-swarm>
Title: Docker Swarm stop working with redeploy service, require system reboot
UserDisplayName: ParisNakitaKejser
ThreadText: # Docker Swarm stop working with redeploy service, require system reboot
  
  (Asked by: username_0 on 2021-08-17)
  
  I got a Docker Swarm with 3 nodes, and contain around 40-50 stack on it, now its random stop working to redeploy my new changes its just say in `new` mode, and really don't know what going one here.
  
  ```
  ID             NAME                            IMAGE                                                               NODE                                DESIRED STATE   CURRENT STATE                ERROR
                          PORTS
  1wyc57lkgebc   translation-manager_web.1       {images}                                       Running         New 6 minutes ago
  
  6xthicsqmkst    \_ translation-manager_web.1   {images}   docker-swarm-02   Shutdown        Shutdown 6 minutes ago
  
  ```
  
  Normal its taking like 20-30sec to redeploy new changes to my Docker Swarm setup.
  
  If I'm running the `service update` command its just stay like this and never ending
  
  ```
  translation-manager_web
  overall progress: 0 out of 1 tasks
  1/1: new       [=====>                                             ]
  
  ```
  
  Its look like its random witch service / container there hit this error.
  
  I got 50% free disk space, lots of memory and free CPU that's the first I think its wrong.
  
  Swarm node info:
  
  ```
  ID                            HOSTNAME                                    STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSION
  6n19d0ryoon2zy4s295s928pp     staging-docker-swarm-01           Ready     Active                          20.10.8
  8se5pzi4znzde6qtlmu0yt0og     staging-docker-swarm-02           Ready     Active                          20.10.8
  p7srx7ojo50rmdxtk6ppriomd     staging-docker-swarm-03           Ready     Active                          20.10.8
  yf8ubbk650rrkmrwkyk3no8zb     staging-docker-swarm-04           Ready     Active                          20.10.8
  f3acixkd61015h0kqgp61ilav *   staging-docker-swarm-manager-01   Ready     Active         Leader           20.10.8
  
  ```
  
  **-- Update: 18/08/2021**
  
  I have added one more worker to my swarm, and 3 new manager to it, and its its still the same issue.
  
  I have 62 Stacks with 109 services running in my swarm right now if the numbers helping.
  
  **Tags:** docker, containers, docker-swarm
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2021-08-18. Score: 1 | Accepted answer
  
  I found out of what happening here. I hit the limit of the IP amount in the ingress network. Default ingress network for Docker Swarm when you create it is 254 IP's its landing in the IP range /24
  
  So what I do its change this IP range to /16 and now get 65k IP's
  
  Every published port you are using multiply with replicates take a single IP so in my case when I got 109 services with avg. replicates its taking 218 IP's out of the default IP range.
  
  So after this change everything are working again in my Docker Swarm setup.
  
  Here is the command to change your ingress network from /24 to /16
  
  ```
  docker network rm ingress
  docker network create \
    --driver overlay \
    --ingress \
    --subnet=10.11.0.0/16 \
    --gateway=10.11.0.2 \
    --opt com.docker.network.driver.mtu=1200 \
    ingress
  
  ```
  
  
  ---
  
  