<p dir="auto"><strong>Is this a request for help?</strong> (If yes, you should use our troubleshooting guide and community support channels, see <a href="http://kubernetes.io/docs/troubleshooting/" rel="nofollow">http://kubernetes.io/docs/troubleshooting/</a>.):</p>
<p dir="auto"><strong>What keywords did you search in Kubernetes issues before filing this one?</strong> (If you have found any duplicates, you should instead reply there.):</p>
<hr>
<p dir="auto"><strong>Is this a BUG REPORT or FEATURE REQUEST?</strong> (choose one):</p>

<p dir="auto"><strong>Kubernetes version</strong> (use <code class="notranslate">kubectl version</code>):</p>
<p dir="auto"><strong>Environment</strong>:</p>
<ul dir="auto">
<li><strong>Cloud provider or hardware configuration</strong>:</li>
<li><strong>OS</strong> (e.g. from /etc/os-release):</li>
<li><strong>Kernel</strong> (e.g. <code class="notranslate">uname -a</code>):</li>
<li><strong>Install tools</strong>:</li>
<li><strong>Others</strong>:</li>
</ul>
<p dir="auto"><strong>What happened</strong>:</p>
<ol dir="auto">
<li>Using Mac OS 10.10.5</li>
<li>Run a cluster locally:</li>
</ol>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="sudo ./hack/local-up-cluster.sh
Password:
+++ [1121 17:25:35] Generating bindata:
    /Users/gyliu/git/kubernetes/test/e2e/framework/gobindata_util.go
+++ [1121 17:25:36] Building the toolchain targets:
    k8s.io/kubernetes/hack/cmd/teststale
+++ [1121 17:25:36] Building go targets for darwin/amd64:
    cmd/kubectl
    cmd/hyperkube
API SERVER insecure port is free, proceeding...
API SERVER secure port is free, proceeding...
Detected host and ready to start services.  Doing some housekeeping first...
Using GO_OUT /Users/gyliu/git/kubernetes/_output/local/bin/darwin/amd64
Starting services now!
Starting etcd
etcd --advertise-client-urls http://127.0.0.1:2379 --data-dir /tmp/test-etcd.XXXXXX.LwKYdriJ --listen-client-urls http://127.0.0.1:2379 --debug &gt; &quot;/dev/null&quot; 2&gt;/dev/null
Waiting for etcd to come up.
+++ [1121 17:27:36] On try 2, etcd: : http://127.0.0.1:2379
{&quot;action&quot;:&quot;set&quot;,&quot;node&quot;:{&quot;key&quot;:&quot;/_test&quot;,&quot;value&quot;:&quot;&quot;,&quot;modifiedIndex&quot;:4,&quot;createdIndex&quot;:4}}
Waiting for apiserver to come up
+++ [1121 17:27:37] On try 2, apiserver: : {
  &quot;major&quot;: &quot;1&quot;,
  &quot;minor&quot;: &quot;6+&quot;,
  &quot;gitVersion&quot;: &quot;v1.6.0-alpha.0.708+0042ce56848e5d&quot;,
  &quot;gitCommit&quot;: &quot;0042ce56848e5d7363f3995a155df3435c9b532d&quot;,
  &quot;gitTreeState&quot;: &quot;clean&quot;,
  &quot;buildDate&quot;: &quot;2016-11-21T09:25:35Z&quot;,
  &quot;goVersion&quot;: &quot;go1.7.3&quot;,
  &quot;compiler&quot;: &quot;gc&quot;,
  &quot;platform&quot;: &quot;darwin/amd64&quot;
}
Local Kubernetes cluster is running. Press Ctrl-C to shut it down.

Logs:
  /tmp/kube-apiserver.log
  /tmp/kube-controller-manager.log
  /tmp/kube-proxy.log
  /tmp/kube-scheduler.log
  /tmp/kubelet.log

To start using your cluster, open up another terminal/tab and run:

  export KUBERNETES_PROVIDER=local

  cluster/kubectl.sh config set-cluster local --server=https://localhost:6443 --certificate-authority=/var/run/kubernetes/apiserver.crt
  cluster/kubectl.sh config set-credentials myself --username=admin --password=admin
  cluster/kubectl.sh config set-context local --cluster=local --user=myself
  cluster/kubectl.sh config use-context local
  cluster/kubectl.sh"><pre class="notranslate"><code class="notranslate">sudo ./hack/local-up-cluster.sh
Password:
+++ [1121 17:25:35] Generating bindata:
    /Users/gyliu/git/kubernetes/test/e2e/framework/gobindata_util.go
+++ [1121 17:25:36] Building the toolchain targets:
    k8s.io/kubernetes/hack/cmd/teststale
+++ [1121 17:25:36] Building go targets for darwin/amd64:
    cmd/kubectl
    cmd/hyperkube
API SERVER insecure port is free, proceeding...
API SERVER secure port is free, proceeding...
Detected host and ready to start services.  Doing some housekeeping first...
Using GO_OUT /Users/gyliu/git/kubernetes/_output/local/bin/darwin/amd64
Starting services now!
Starting etcd
etcd --advertise-client-urls http://127.0.0.1:2379 --data-dir /tmp/test-etcd.XXXXXX.LwKYdriJ --listen-client-urls http://127.0.0.1:2379 --debug &gt; "/dev/null" 2&gt;/dev/null
Waiting for etcd to come up.
+++ [1121 17:27:36] On try 2, etcd: : http://127.0.0.1:2379
{"action":"set","node":{"key":"/_test","value":"","modifiedIndex":4,"createdIndex":4}}
Waiting for apiserver to come up
+++ [1121 17:27:37] On try 2, apiserver: : {
  "major": "1",
  "minor": "6+",
  "gitVersion": "v1.6.0-alpha.0.708+0042ce56848e5d",
  "gitCommit": "0042ce56848e5d7363f3995a155df3435c9b532d",
  "gitTreeState": "clean",
  "buildDate": "2016-11-21T09:25:35Z",
  "goVersion": "go1.7.3",
  "compiler": "gc",
  "platform": "darwin/amd64"
}
Local Kubernetes cluster is running. Press Ctrl-C to shut it down.

Logs:
  /tmp/kube-apiserver.log
  /tmp/kube-controller-manager.log
  /tmp/kube-proxy.log
  /tmp/kube-scheduler.log
  /tmp/kubelet.log

To start using your cluster, open up another terminal/tab and run:

  export KUBERNETES_PROVIDER=local

  cluster/kubectl.sh config set-cluster local --server=https://localhost:6443 --certificate-authority=/var/run/kubernetes/apiserver.crt
  cluster/kubectl.sh config set-credentials myself --username=admin --password=admin
  cluster/kubectl.sh config set-context local --cluster=local --user=myself
  cluster/kubectl.sh config use-context local
  cluster/kubectl.sh
</code></pre></div>
<p dir="auto">Check agents:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="cluster/kubectl.sh get nodes
No resources found."><pre class="notranslate"><code class="notranslate">cluster/kubectl.sh get nodes
No resources found.
</code></pre></div>
<p dir="auto">Check kubelet log:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="I1121 17:27:38.113443   42083 feature_gate.go:181] feature gates: map[ExperimentalHostUserNamespaceDefaulting:true DynamicKubeletConfig:true AllAlpha:true DynamicVolumeProvisioning:true StreamingProxyRedirects:true]
I1121 17:27:38.146640   42083 server.go:217] Starting Kubelet configuration sync loop
I1121 17:27:38.146671   42083 plugins.go:94] No cloud provider specified.
I1121 17:27:38.146679   42083 server.go:367] No cloud provider specified: &quot;&quot; from the config file: &quot;&quot;
I1121 17:27:38.536045   42083 docker.go:356] Connecting to docker on unix:///var/run/docker.sock
I1121 17:27:38.536077   42083 docker.go:376] Start docker client with request timeout=2m0s
E1121 17:27:38.540852   42083 cni.go:163] error updating cni config: No networks found in /etc/cni/net.d
W1121 17:27:38.540911   42083 server.go:469] setting OOM scores is unsupported in this build
I1121 17:27:38.540973   42083 server.go:700] Using root directory: /var/lib/kubelet
I1121 17:27:38.541384   42083 kubelet.go:252] Watching apiserver
I1121 17:27:38.541426   42083 reflector.go:185] Starting reflector *api.Pod (0s) from pkg/kubelet/config/apiserver.go:44
I1121 17:27:38.541471   42083 reflector.go:234] Listing and watching *api.Pod from pkg/kubelet/config/apiserver.go:44
I1121 17:27:38.541717   42083 reflector.go:185] Starting reflector *api.Service (0s) from pkg/kubelet/kubelet.go:377
I1121 17:27:38.541755   42083 reflector.go:185] Starting reflector *api.Node (0s) from pkg/kubelet/kubelet.go:385
I1121 17:27:38.541764   42083 reflector.go:234] Listing and watching *api.Service from pkg/kubelet/kubelet.go:377
I1121 17:27:38.541801   42083 reflector.go:234] Listing and watching *api.Node from pkg/kubelet/kubelet.go:385
W1121 17:27:38.541810   42083 iptables.go:143] Error checking iptables version, assuming version at least 1.4.11: executable file not found in $PATH
I1121 17:27:38.541897   42083 iptables.go:176] Could not connect to D-Bus system bus: dial unix /var/run/dbus/system_bus_socket: connect: no such file or directory
I1121 17:27:38.541934   42083 kubelet.go:466] Experimental host user namespace defaulting is enabled.
W1121 17:27:38.541941   42083 kubelet_network.go:69] Hairpin mode set to &quot;promiscuous-bridge&quot; but kubenet is not enabled, falling back to &quot;hairpin-veth&quot;
I1121 17:27:38.541955   42083 kubelet.go:476] Hairpin mode set to &quot;hairpin-veth&quot;
W1121 17:27:38.542020   42083 plugins.go:210] can't set sysctl net/bridge/bridge-nf-call-iptables: open /proc/sys/net/bridge/bridge-nf-call-iptables: no such file or directory
Error: failed to run Kubelet: failed to create kubelet: cAdvisor is unsupported in this build
failed to run Kubelet: failed to create kubelet: cAdvisor is unsupported in this build"><pre class="notranslate"><code class="notranslate">I1121 17:27:38.113443   42083 feature_gate.go:181] feature gates: map[ExperimentalHostUserNamespaceDefaulting:true DynamicKubeletConfig:true AllAlpha:true DynamicVolumeProvisioning:true StreamingProxyRedirects:true]
I1121 17:27:38.146640   42083 server.go:217] Starting Kubelet configuration sync loop
I1121 17:27:38.146671   42083 plugins.go:94] No cloud provider specified.
I1121 17:27:38.146679   42083 server.go:367] No cloud provider specified: "" from the config file: ""
I1121 17:27:38.536045   42083 docker.go:356] Connecting to docker on unix:///var/run/docker.sock
I1121 17:27:38.536077   42083 docker.go:376] Start docker client with request timeout=2m0s
E1121 17:27:38.540852   42083 cni.go:163] error updating cni config: No networks found in /etc/cni/net.d
W1121 17:27:38.540911   42083 server.go:469] setting OOM scores is unsupported in this build
I1121 17:27:38.540973   42083 server.go:700] Using root directory: /var/lib/kubelet
I1121 17:27:38.541384   42083 kubelet.go:252] Watching apiserver
I1121 17:27:38.541426   42083 reflector.go:185] Starting reflector *api.Pod (0s) from pkg/kubelet/config/apiserver.go:44
I1121 17:27:38.541471   42083 reflector.go:234] Listing and watching *api.Pod from pkg/kubelet/config/apiserver.go:44
I1121 17:27:38.541717   42083 reflector.go:185] Starting reflector *api.Service (0s) from pkg/kubelet/kubelet.go:377
I1121 17:27:38.541755   42083 reflector.go:185] Starting reflector *api.Node (0s) from pkg/kubelet/kubelet.go:385
I1121 17:27:38.541764   42083 reflector.go:234] Listing and watching *api.Service from pkg/kubelet/kubelet.go:377
I1121 17:27:38.541801   42083 reflector.go:234] Listing and watching *api.Node from pkg/kubelet/kubelet.go:385
W1121 17:27:38.541810   42083 iptables.go:143] Error checking iptables version, assuming version at least 1.4.11: executable file not found in $PATH
I1121 17:27:38.541897   42083 iptables.go:176] Could not connect to D-Bus system bus: dial unix /var/run/dbus/system_bus_socket: connect: no such file or directory
I1121 17:27:38.541934   42083 kubelet.go:466] Experimental host user namespace defaulting is enabled.
W1121 17:27:38.541941   42083 kubelet_network.go:69] Hairpin mode set to "promiscuous-bridge" but kubenet is not enabled, falling back to "hairpin-veth"
I1121 17:27:38.541955   42083 kubelet.go:476] Hairpin mode set to "hairpin-veth"
W1121 17:27:38.542020   42083 plugins.go:210] can't set sysctl net/bridge/bridge-nf-call-iptables: open /proc/sys/net/bridge/bridge-nf-call-iptables: no such file or directory
Error: failed to run Kubelet: failed to create kubelet: cAdvisor is unsupported in this build
failed to run Kubelet: failed to create kubelet: cAdvisor is unsupported in this build
</code></pre></div>
<p dir="auto"><strong>What you expected to happen</strong>:</p>
<p dir="auto"><strong>How to reproduce it</strong> (as minimally and precisely as possible):</p>
<p dir="auto"><strong>Anything else do we need to know</strong>:</p>
<p dir="auto">Currently if you have a large replication controller running and stop said controller, kubernetes will report that no pods are running, when indeed there are currently pods on the cluster.</p>
<p dir="auto">This occurs because the entries are deleted from etcd and eventually the kubelets will catch up.  Perhaps it makes more sense to update the phase to "vacating" and only delete once the final event has been sent.</p>
<p dir="auto"><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/dchen1107/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/dchen1107">@dchen1107</a> , <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/wojtek-t/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/wojtek-t">@wojtek-t</a> , <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/fgrzadkowski/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/fgrzadkowski">@fgrzadkowski</a></p>