Answers: <p>After some more time I found my answer.</p>
  
  <p>The service I wanted to deploy was a slightly different version of a service we already have. Stack file for the "original" service:</p>
  
  <pre><code>version: '3.7'
  
  services:
  my-service:
      image: my-repo:port/company/my-service
      ports:
      - 81:81
      networks:
      - my-network
      deploy:
      replicas: 1
      restart_policy:
          condition: on-failure
          delay: 5s
          max_attempts: 3
          window: 120s
  networks:
  my-network:
      driver: overlay
      internal: false
      attachable: true
      ipam:
      driver: default
      config:
          - subnet: 10.1.99.0/24
  </code></pre>
  
  <p>We took <strong>the same file</strong> but on a different git branch to deploy the slightly altered version (as makes sense from a "code" perspective).</p>
  
  <p>From a Docker perspective this meant however that the stack with the original service with the <strong>exact same network definition</strong> was still running and we deployed another stack with <strong>the exact same network definition</strong> again, which lead to the service never being scheduled as, apparently, you must not re-declare the same network twice.</p>
  
  <p>Of course it will get a different network name, but the overlay/subnet config will be identical.</p>
  
  <p>It would be nice of Docker to inform you of this when you're trying to do that, but, as it so often is: Crap in, crap out. Our fault!</p>
  
  <p>For future readers who do not have the exact same problem but their containers are getting stuck in "new" state: It might be safe to assume that it has something to do with your stack-file, your network definition or even something like correct YAML indentation. It is most certainly something "meta" and not something related to your Docker host and specifically not your image or your container (as when the task is stuck in "new" state, there is not even a container in existence as it never got scheduled).</p>
  
Answers: CC BY-SA 4.0
Answers: 2019-10-10T19:40:34.530Z
Answers: 2019-10-10T19:40:34.530Z
Answers: Worp
Body: <p>I am deploying a service stack to a Docker swarm like so:</p>
  
  <pre><code>$ dt stack deploy --with-registry-auth -c tmp.stack.docker-compose.yml tmp
  Creating network tmp_my-network
  Creating service tmp_my-service
  
  $ dt stack ps tmp
  ID                  NAME                IMAGE               NODE    DESIRED STATE       CURRENT STATE       ERROR               PORTS
  yyjue6fdqu5u        tmp_my-service.1    nginx:latest                            Running             New 7 minutes ago
  </code></pre>
  
  <p>As you can see it's just an nginx, but the state of the service never leaves "new".</p>
  
  <p>I can not stack deploy any other service either. No matter what I write into my compose file, they are all stuck in state "NEW".</p>
  
  <p>I am not sure how to produce the following output from commandline, but in a running portainer I was able to get the following information:</p>
  
  <pre><code>Services &gt; tmp_my-service &gt; qlvqjofehvqys8kje3b32y50v &gt; Logs
  "Error grabbing logs: rpc error: code = Unknown desc = warning: incomplete log stream. some logs could not be retrieved for the following reasons: task qlvqjofehvqys8kje3b32y50v has not been scheduled"
  </code></pre>
  
  <p>This makes sense, as it looks like the service's tasks are never being scheduled, nothing is actually ever started. So I am assuming something (at orchestrator level) stops Docker swarm from carrying out the request to get any service started.</p>
  
  <p>Where might I find more information about what could be wrong?</p>
  
  <p>I have tried:</p>
  
  <pre><code>$ docker events
  network create bcqcoakf5worjmfpelb199xrp (name=my-network)
  node update rddzqxvlu2v2ylbcbywpa8kze (name=node-name)
  service create x549akxlbonn9zvtxgh8oq2mm (name=my-service)
  </code></pre>
  
  <p><a href="https://success.docker.com/article/why-do-my-services-stay-pending-when-trying-to-schedule-them-with-placement-contraints" rel="nofollow noreferrer">https://success.docker.com/article/why-do-my-services-stay-pending-when-trying-to-schedule-them-with-placement-contraints</a></p>
  
  <p><a href="https://success.docker.com/article/swarm-troubleshooting-methodology" rel="nofollow noreferrer">https://success.docker.com/article/swarm-troubleshooting-methodology</a></p>
  
  <p>Details of the task:</p>
  
  <pre><code>$ dt stack services tmp
  dID                  NAME                MODE                REPLICAS            IMAGE               PORTS
  x549akxlbonn        my-service      replicated          0/1                 nginx:latest
  
  $ dt service ps my-service
  ID                  NAME                IMAGE               NODE                DESIRED STATE       CURRENT STATE       ERROR               PORTS
  m4d929n5su7r        my-service.1    nginx:latest                            Running             New 4 minutes ago
  
  $ dt inspect --type task m4d929n5su7r
  [
      {
          "ID": "m4d929n5su7re5igit3w2tnvg",
          "Version": {
              "Index": 2397643
          },
          "CreatedAt": "2019-10-10T15:58:33.416210977Z",
          "UpdatedAt": "2019-10-10T15:58:33.416210977Z",
          "Labels": {},
          "Spec": {
              "ContainerSpec": {
                  "Image": "nginx:latest@sha256:aeded0f2a861747f43a01cf1018cf9efe2bdd02afd57d2b11fcc7fcadc16ccd1",
                  "Labels": {
                      "com.docker.stack.namespace": "tmp"
                  },
                  "Command": [
                      "tail",
                      "-f",
                      "/dev/null"
                  ],
                  "Privileges": {
                      "CredentialSpec": null,
                      "SELinuxContext": null
                  },
                  "Isolation": "default"
              },
              "Resources": {},
              "RestartPolicy": {
                  "Condition": "on-failure",
                  "Delay": 5000000000,
                  "MaxAttempts": 3,
                  "Window": 120000000000
              },
              "Placement": {
                  "Constraints": [
                      "node.role == manager"
                  ],
                  "Platforms": [
                      {
                          "Architecture": "amd64",
                          "OS": "linux"
                      },
                      {
                          "OS": "linux"
                      },
                      {
                          "Architecture": "arm64",
                          "OS": "linux"
                      },
                      {
                          "Architecture": "386",
                          "OS": "linux"
                      },
                      {
                          "Architecture": "ppc64le",
                          "OS": "linux"
                      },
                      {
                          "Architecture": "s390x",
                          "OS": "linux"
                      }
                  ]
              },
              "Networks": [
                  {
                      "Target": "bcqcoakf5worjmfpelb199xrp",
                      "Aliases": [
                          "my-service"
                      ]
                  }
              ],
              "ForceUpdate": 0
          },
          "ServiceID": "x549akxlbonn9zvtxgh8oq2mm",
          "Slot": 1,
          "Status": {
              "Timestamp": "2019-10-10T15:58:33.41619488Z",
              "State": "new",
              "Message": "created",
              "PortStatus": {}
          },
          "DesiredState": "running"
      }
  ]
  </code></pre>
  
  <p>It all gives me the same info: Desired is "running", actual state is "new" but it won't give me more info on why.</p>
  
  <p><strong>edit:</strong> Under "placement > Constraints" it says "node.role == manager" in the ouput. This is just temporary, as I have been trying to get it to run on individual nodes by constraining it. It does not work with this constraint, nor any other constraint, nor with no constraint at all.</p>
  
Comments: CC BY-SA 4.0
Comments: 2020-01-08T08:55:17.673Z
Comments: Refer https://stackoverflow.com/a/52547670/2137210 for solution
Comments: Pratik
Tags: <docker><containers><docker-swarm>
Title: Docker container stuck in "new" state - Docker swarm - service stack
UserDisplayName: Worp
ThreadText: # Docker container stuck in "new" state - Docker swarm - service stack
  
  (Asked by: username_0 on 2019-10-10)
  
  I am deploying a service stack to a Docker swarm like so:
  
  ```
  $ dt stack deploy --with-registry-auth -c tmp.stack.docker-compose.yml tmp
  Creating network tmp_my-network
  Creating service tmp_my-service
  
  $ dt stack ps tmp
  ID                  NAME                IMAGE               NODE    DESIRED STATE       CURRENT STATE       ERROR               PORTS
  yyjue6fdqu5u        tmp_my-service.1    nginx:latest                            Running             New 7 minutes ago
  
  ```
  
  As you can see it's just an nginx, but the state of the service never leaves "new".
  
  I can not stack deploy any other service either. No matter what I write into my compose file, they are all stuck in state "NEW".
  
  I am not sure how to produce the following output from commandline, but in a running portainer I was able to get the following information:
  
  ```
  Services > tmp_my-service > qlvqjofehvqys8kje3b32y50v > Logs
  "Error grabbing logs: rpc error: code = Unknown desc = warning: incomplete log stream. some logs could not be retrieved for the following reasons: task qlvqjofehvqys8kje3b32y50v has not been scheduled"
  
  ```
  
  This makes sense, as it looks like the service's tasks are never being scheduled, nothing is actually ever started. So I am assuming something (at orchestrator level) stops Docker swarm from carrying out the request to get any service started.
  
  Where might I find more information about what could be wrong?
  
  I have tried:
  
  ```
  $ docker events
  network create bcqcoakf5worjmfpelb199xrp (name=my-network)
  node update rddzqxvlu2v2ylbcbywpa8kze (name=node-name)
  service create x549akxlbonn9zvtxgh8oq2mm (name=my-service)
  
  ```
  
  <https://success.docker.com/article/why-do-my-services-stay-pending-when-trying-to-schedule-them-with-placement-contraints>
  
  <https://success.docker.com/article/swarm-troubleshooting-methodology>
  
  Details of the task:
  
  ```
  $ dt stack services tmp
  dID                  NAME                MODE                REPLICAS            IMAGE               PORTS
  x549akxlbonn        my-service      replicated          0/1                 nginx:latest
  
  $ dt service ps my-service
  ID                  NAME                IMAGE               NODE                DESIRED STATE       CURRENT STATE       ERROR               PORTS
  m4d929n5su7r        my-service.1    nginx:latest                            Running             New 4 minutes ago
  
  $ dt inspect --type task m4d929n5su7r
  [
      {
          "ID": "m4d929n5su7re5igit3w2tnvg",
          "Version": {
              "Index": 2397643
          },
          "CreatedAt": "2019-10-10T15:58:33.416210977Z",
          "UpdatedAt": "2019-10-10T15:58:33.416210977Z",
          "Labels": {},
          "Spec": {
              "ContainerSpec": {
                  "Image": "nginx:latest@sha256:aeded0f2a861747f43a01cf1018cf9efe2bdd02afd57d2b11fcc7fcadc16ccd1",
                  "Labels": {
                      "com.docker.stack.namespace": "tmp"
                  },
                  "Command": [
                      "tail",
                      "-f",
                      "/dev/null"
                  ],
                  "Privileges": {
                      "CredentialSpec": null,
                      "SELinuxContext": null
                  },
                  "Isolation": "default"
              },
              "Resources": {},
              "RestartPolicy": {
                  "Condition": "on-failure",
                  "Delay": 5000000000,
                  "MaxAttempts": 3,
                  "Window": 120000000000
              },
              "Placement": {
                  "Constraints": [
                      "node.role == manager"
                  ],
                  "Platforms": [
                      {
                          "Architecture": "amd64",
                          "OS": "linux"
                      },
                      {
                          "OS": "linux"
                      },
                      {
                          "Architecture": "arm64",
                          "OS": "linux"
                      },
                      {
                          "Architecture": "386",
                          "OS": "linux"
                      },
                      {
                          "Architecture": "ppc64le",
                          "OS": "linux"
                      },
                      {
                          "Architecture": "s390x",
                          "OS": "linux"
                      }
                  ]
              },
              "Networks": [
                  {
                      "Target": "bcqcoakf5worjmfpelb199xrp",
                      "Aliases": [
                          "my-service"
                      ]
                  }
              ],
              "ForceUpdate": 0
          },
          "ServiceID": "x549akxlbonn9zvtxgh8oq2mm",
          "Slot": 1,
          "Status": {
              "Timestamp": "2019-10-10T15:58:33.41619488Z",
              "State": "new",
              "Message": "created",
              "PortStatus": {}
          },
          "DesiredState": "running"
      }
  ]
  
  ```
  
  It all gives me the same info: Desired is "running", actual state is "new" but it won't give me more info on why.
  
  **edit:** Under "placement > Constraints" it says "node.role == manager" in the ouput. This is just temporary, as I have been trying to get it to run on individual nodes by constraining it. It does not work with this constraint, nor any other constraint, nor with no constraint at all.
  
  **Tags:** docker, containers, docker-swarm
  
  > **Comments:**
  > **username_1**: Refer https://stackoverflow.com/a/52547670/2137210 for solution  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2019-10-10. Score: 1 | Accepted answer
  
  After some more time I found my answer.
  
  The service I wanted to deploy was a slightly different version of a service we already have. Stack file for the "original" service:
  
  ```
  version: '3.7'
  
  services:
  my-service:
      image: my-repo:port/company/my-service
      ports:
      - 81:81
      networks:
      - my-network
      deploy:
      replicas: 1
      restart_policy:
          condition: on-failure
          delay: 5s
          max_attempts: 3
          window: 120s
  networks:
  my-network:
      driver: overlay
      internal: false
      attachable: true
      ipam:
      driver: default
      config:
          - subnet: 10.1.99.0/24
  
  ```
  
  We took **the same file** but on a different git branch to deploy the slightly altered version (as makes sense from a "code" perspective).
  
  From a Docker perspective this meant however that the stack with the original service with the **exact same network definition** was still running and we deployed another stack with **the exact same network definition** again, which lead to the service never being scheduled as, apparently, you must not re-declare the same network twice.
  
  Of course it will get a different network name, but the overlay/subnet config will be identical.
  
  It would be nice of Docker to inform you of this when you're trying to do that, but, as it so often is: Crap in, crap out. Our fault!
  
  For future readers who do not have the exact same problem but their containers are getting stuck in "new" state: It might be safe to assume that it has something to do with your stack-file, your network definition or even something like correct YAML indentation. It is most certainly something "meta" and not something related to your Docker host and specifically not your image or your container (as when the task is stuck in "new" state, there is not even a container in existence as it never got scheduled).
  
  
  ---
  
  