<p dir="auto"><strong>Is this a request for help?</strong> (If yes, you should use our troubleshooting guide and community support channels, see <a href="http://kubernetes.io/docs/troubleshooting/" rel="nofollow">http://kubernetes.io/docs/troubleshooting/</a>.):</p>
<p dir="auto">BUG REPORT</p>
<p dir="auto"><strong>What keywords did you search in Kubernetes issues before filing this one?</strong> (If you have found any duplicates, you should instead reply there.):<br>
kube-dns kubernetes setupnetworkerror flannel subnet.env no such file</p>
<hr>
<p dir="auto"><strong>Is this a BUG REPORT or FEATURE REQUEST?</strong> (choose one):</p>
<p dir="auto">BUG REPORT</p>

<p dir="auto"><strong>Kubernetes version</strong> (use <code class="notranslate">kubectl version</code>):</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Client Version: version.Info{Major:&quot;1&quot;, Minor:&quot;4&quot;, GitVersion:&quot;v1.4.4&quot;, GitCommit:&quot;3b417cc4ccd1b8f38ff9ec96bb50a81ca0ea9d56&quot;, GitTreeState:&quot;clean&quot;, BuildDate:&quot;2016-10-21T02:48:38Z&quot;, GoVersion:&quot;go1.6.3&quot;, Compiler:&quot;gc&quot;, Platform:&quot;linux/amd64&quot;}
Server Version: version.Info{Major:&quot;1&quot;, Minor:&quot;4&quot;, GitVersion:&quot;v1.4.4&quot;, GitCommit:&quot;3b417cc4ccd1b8f38ff9ec96bb50a81ca0ea9d56&quot;, GitTreeState:&quot;clean&quot;, BuildDate:&quot;2016-10-21T02:42:39Z&quot;, GoVersion:&quot;go1.6.3&quot;, Compiler:&quot;gc&quot;, Platform:&quot;linux/amd64&quot;}"><pre class="notranslate"><code class="notranslate">Client Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.4", GitCommit:"3b417cc4ccd1b8f38ff9ec96bb50a81ca0ea9d56", GitTreeState:"clean", BuildDate:"2016-10-21T02:48:38Z", GoVersion:"go1.6.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.4", GitCommit:"3b417cc4ccd1b8f38ff9ec96bb50a81ca0ea9d56", GitTreeState:"clean", BuildDate:"2016-10-21T02:42:39Z", GoVersion:"go1.6.3", Compiler:"gc", Platform:"linux/amd64"}
</code></pre></div>
<p dir="auto"><strong>Environment</strong>:</p>
<ul dir="auto">
<li><strong>Cloud provider or hardware configuration</strong>:</li>
</ul>
<p dir="auto">VMWare Fusion for Mac</p>
<ul dir="auto">
<li><strong>OS</strong> (e.g. from /etc/os-release):</li>
</ul>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="NAME=&quot;Ubuntu&quot;
VERSION=&quot;16.04.1 LTS (Xenial Xerus)&quot;
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME=&quot;Ubuntu 16.04.1 LTS&quot;
VERSION_ID=&quot;16.04&quot;
HOME_URL=&quot;http://www.ubuntu.com/&quot;
SUPPORT_URL=&quot;http://help.ubuntu.com/&quot;
BUG_REPORT_URL=&quot;http://bugs.launchpad.net/ubuntu/&quot;
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial"><pre class="notranslate"><code class="notranslate">NAME="Ubuntu"
VERSION="16.04.1 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.1 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial
</code></pre></div>
<ul dir="auto">
<li><strong>Kernel</strong> (e.g. <code class="notranslate">uname -a</code>):</li>
</ul>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Linux ubuntu-master 4.4.0-47-generic #68-Ubuntu SMP Wed Oct 26 19:39:52 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux"><pre class="notranslate"><code class="notranslate">Linux ubuntu-master 4.4.0-47-generic #68-Ubuntu SMP Wed Oct 26 19:39:52 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
</code></pre></div>
<ul dir="auto">
<li>
<p dir="auto"><strong>Install tools</strong>:</p>
</li>
<li>
<p dir="auto"><strong>Others</strong>:</p>
</li>
</ul>
<p dir="auto"><strong>What happened</strong>:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="kube-system   kube-dns-654381707-w4mpg                0/3       ContainerCreating   0          2m"><pre class="notranslate"><code class="notranslate">kube-system   kube-dns-654381707-w4mpg                0/3       ContainerCreating   0          2m
</code></pre></div>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="FirstSeen     LastSeen        Count   From                    SubobjectPath   Type            Reason          Message
  ---------     --------        -----   ----                    -------------   --------        ------          -------
  3m            3m              1       {default-scheduler }                    Normal          Scheduled       Successfully assigned kube-dns-654381707-w4mpg to ubuntu-master
  2m            1s              177     {kubelet ubuntu-master}                 Warning         FailedSync      Error syncing pod, skipping: failed to &quot;SetupNetwork&quot; for &quot;kube-dns-654381707-w4mpg_kube-system&quot; with SetupNetworkError: &quot;Failed to setup network for pod \&quot;kube-dns-654381707-w4mpg_kube-system(8ffe3172-a739-11e6-871f-000c2912631c)\&quot; using network plugins \&quot;cni\&quot;: open /run/flannel/subnet.env: no such file or directory; Skipping pod&quot;"><pre class="notranslate"><code class="notranslate">FirstSeen     LastSeen        Count   From                    SubobjectPath   Type            Reason          Message
  ---------     --------        -----   ----                    -------------   --------        ------          -------
  3m            3m              1       {default-scheduler }                    Normal          Scheduled       Successfully assigned kube-dns-654381707-w4mpg to ubuntu-master
  2m            1s              177     {kubelet ubuntu-master}                 Warning         FailedSync      Error syncing pod, skipping: failed to "SetupNetwork" for "kube-dns-654381707-w4mpg_kube-system" with SetupNetworkError: "Failed to setup network for pod \"kube-dns-654381707-w4mpg_kube-system(8ffe3172-a739-11e6-871f-000c2912631c)\" using network plugins \"cni\": open /run/flannel/subnet.env: no such file or directory; Skipping pod"
</code></pre></div>
<p dir="auto"><strong>What you expected to happen</strong>:</p>
<p dir="auto">kube-dns Running</p>
<p dir="auto"><strong>How to reproduce it</strong> (as minimally and precisely as possible):</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="root@ubuntu-master:~# kubeadm init
Running pre-flight checks
&lt;master/tokens&gt; generated token: &quot;247a8e.b7c8c1a7685bf204&quot;
&lt;master/pki&gt; generated Certificate Authority key and certificate:
Issuer: CN=kubernetes | Subject: CN=kubernetes | CA: true
Not before: 2016-11-10 11:40:21 +0000 UTC Not After: 2026-11-08 11:40:21 +0000 UTC
Public: /etc/kubernetes/pki/ca-pub.pem
Private: /etc/kubernetes/pki/ca-key.pem
Cert: /etc/kubernetes/pki/ca.pem
&lt;master/pki&gt; generated API Server key and certificate:
Issuer: CN=kubernetes | Subject: CN=kube-apiserver | CA: false
Not before: 2016-11-10 11:40:21 +0000 UTC Not After: 2017-11-10 11:40:21 +0000 UTC
Alternate Names: [172.20.10.4 10.96.0.1 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local]
Public: /etc/kubernetes/pki/apiserver-pub.pem
Private: /etc/kubernetes/pki/apiserver-key.pem
Cert: /etc/kubernetes/pki/apiserver.pem
&lt;master/pki&gt; generated Service Account Signing keys:
Public: /etc/kubernetes/pki/sa-pub.pem
Private: /etc/kubernetes/pki/sa-key.pem
&lt;master/pki&gt; created keys and certificates in &quot;/etc/kubernetes/pki&quot;
&lt;util/kubeconfig&gt; created &quot;/etc/kubernetes/kubelet.conf&quot;
&lt;util/kubeconfig&gt; created &quot;/etc/kubernetes/admin.conf&quot;
&lt;master/apiclient&gt; created API client configuration
&lt;master/apiclient&gt; created API client, waiting for the control plane to become ready
&lt;master/apiclient&gt; all control plane components are healthy after 14.053453 seconds
&lt;master/apiclient&gt; waiting for at least one node to register and become ready
&lt;master/apiclient&gt; first node is ready after 0.508561 seconds
&lt;master/apiclient&gt; attempting a test deployment
&lt;master/apiclient&gt; test deployment succeeded
&lt;master/discovery&gt; created essential addon: kube-discovery, waiting for it to become ready
&lt;master/discovery&gt; kube-discovery is ready after 1.503838 seconds
&lt;master/addons&gt; created essential addon: kube-proxy
&lt;master/addons&gt; created essential addon: kube-dns

Kubernetes master initialised successfully!

You can now join any number of machines by running the following on each node:

kubeadm join --token=247a8e.b7c8c1a7685bf204 172.20.10.4
root@ubuntu-master:~# 
root@ubuntu-master:~# 
root@ubuntu-master:~# 
root@ubuntu-master:~# kubectl get pods --all-namespaces
NAMESPACE     NAME                                    READY     STATUS              RESTARTS   AGE
kube-system   dummy-2088944543-eo1ua                  1/1       Running             0          47s
kube-system   etcd-ubuntu-master                      1/1       Running             3          51s
kube-system   kube-apiserver-ubuntu-master            1/1       Running             0          49s
kube-system   kube-controller-manager-ubuntu-master   1/1       Running             3          51s
kube-system   kube-discovery-1150918428-qmu0b         1/1       Running             0          46s
kube-system   kube-dns-654381707-mv47d                0/3       ContainerCreating   0          44s
kube-system   kube-proxy-k0k9q                        1/1       Running             0          44s
kube-system   kube-scheduler-ubuntu-master            1/1       Running             3          51s
root@ubuntu-master:~# 
root@ubuntu-master:~# 
root@ubuntu-master:~# 
root@ubuntu-master:~# kubectl apply -f https://git.io/weave-kube
daemonset &quot;weave-net&quot; created
root@ubuntu-master:~# 
root@ubuntu-master:~# 
root@ubuntu-master:~# "><pre class="notranslate"><code class="notranslate">root@ubuntu-master:~# kubeadm init
Running pre-flight checks
&lt;master/tokens&gt; generated token: "247a8e.b7c8c1a7685bf204"
&lt;master/pki&gt; generated Certificate Authority key and certificate:
Issuer: CN=kubernetes | Subject: CN=kubernetes | CA: true
Not before: 2016-11-10 11:40:21 +0000 UTC Not After: 2026-11-08 11:40:21 +0000 UTC
Public: /etc/kubernetes/pki/ca-pub.pem
Private: /etc/kubernetes/pki/ca-key.pem
Cert: /etc/kubernetes/pki/ca.pem
&lt;master/pki&gt; generated API Server key and certificate:
Issuer: CN=kubernetes | Subject: CN=kube-apiserver | CA: false
Not before: 2016-11-10 11:40:21 +0000 UTC Not After: 2017-11-10 11:40:21 +0000 UTC
Alternate Names: [172.20.10.4 10.96.0.1 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local]
Public: /etc/kubernetes/pki/apiserver-pub.pem
Private: /etc/kubernetes/pki/apiserver-key.pem
Cert: /etc/kubernetes/pki/apiserver.pem
&lt;master/pki&gt; generated Service Account Signing keys:
Public: /etc/kubernetes/pki/sa-pub.pem
Private: /etc/kubernetes/pki/sa-key.pem
&lt;master/pki&gt; created keys and certificates in "/etc/kubernetes/pki"
&lt;util/kubeconfig&gt; created "/etc/kubernetes/kubelet.conf"
&lt;util/kubeconfig&gt; created "/etc/kubernetes/admin.conf"
&lt;master/apiclient&gt; created API client configuration
&lt;master/apiclient&gt; created API client, waiting for the control plane to become ready
&lt;master/apiclient&gt; all control plane components are healthy after 14.053453 seconds
&lt;master/apiclient&gt; waiting for at least one node to register and become ready
&lt;master/apiclient&gt; first node is ready after 0.508561 seconds
&lt;master/apiclient&gt; attempting a test deployment
&lt;master/apiclient&gt; test deployment succeeded
&lt;master/discovery&gt; created essential addon: kube-discovery, waiting for it to become ready
&lt;master/discovery&gt; kube-discovery is ready after 1.503838 seconds
&lt;master/addons&gt; created essential addon: kube-proxy
&lt;master/addons&gt; created essential addon: kube-dns

Kubernetes master initialised successfully!

You can now join any number of machines by running the following on each node:

kubeadm join --token=247a8e.b7c8c1a7685bf204 172.20.10.4
root@ubuntu-master:~# 
root@ubuntu-master:~# 
root@ubuntu-master:~# 
root@ubuntu-master:~# kubectl get pods --all-namespaces
NAMESPACE     NAME                                    READY     STATUS              RESTARTS   AGE
kube-system   dummy-2088944543-eo1ua                  1/1       Running             0          47s
kube-system   etcd-ubuntu-master                      1/1       Running             3          51s
kube-system   kube-apiserver-ubuntu-master            1/1       Running             0          49s
kube-system   kube-controller-manager-ubuntu-master   1/1       Running             3          51s
kube-system   kube-discovery-1150918428-qmu0b         1/1       Running             0          46s
kube-system   kube-dns-654381707-mv47d                0/3       ContainerCreating   0          44s
kube-system   kube-proxy-k0k9q                        1/1       Running             0          44s
kube-system   kube-scheduler-ubuntu-master            1/1       Running             3          51s
root@ubuntu-master:~# 
root@ubuntu-master:~# 
root@ubuntu-master:~# 
root@ubuntu-master:~# kubectl apply -f https://git.io/weave-kube
daemonset "weave-net" created
root@ubuntu-master:~# 
root@ubuntu-master:~# 
root@ubuntu-master:~# 
</code></pre></div>
<p dir="auto"><strong>Anything else do we need to know</strong>:</p>
<p dir="auto">First time I ran</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="root@ubuntu-master:~# kubeadm init"><pre class="notranslate"><code class="notranslate">root@ubuntu-master:~# kubeadm init
</code></pre></div>
<p dir="auto">and then downloaded kube-flannel.yml and apply</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="root@ubuntu-master:~# kubectl apply -f kube-flannel.yml"><pre class="notranslate"><code class="notranslate">root@ubuntu-master:~# kubectl apply -f kube-flannel.yml
</code></pre></div>
<p dir="auto">And then tried to join nodes. Then I reset configuration by</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="root@ubuntu-master:~# kubeadm reset
root@ubuntu-master:~# rm -rf .kube/"><pre class="notranslate"><code class="notranslate">root@ubuntu-master:~# kubeadm reset
root@ubuntu-master:~# rm -rf .kube/
</code></pre></div>
<p dir="auto">Then tried to initialize kubernetes again using weave.</p>
<p dir="auto">Most applications should be running in multiple zones to increase availability. Kubernetes should support it. I imagine this to work in the following way:</p>
<ol dir="auto">
<li>User sets up cluster in a region in a way that minions are spread evenly across all available zones</li>
<li>User creates replication controller for the application with size &gt; 1</li>
<li>Scheduler spreads pods within the same replication controller across available zones</li>
</ol>
<p dir="auto">That way user gets regional availability for free.</p>
<p dir="auto">AFAIU this will mostly require changes in how we create cluster and how we schedule pods.</p>
<p dir="auto">cc <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/wojtek-t/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/wojtek-t">@wojtek-t</a></p>