<p dir="auto">Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see <a href="http://kubernetes.io/docs/troubleshooting/" rel="nofollow">http://kubernetes.io/docs/troubleshooting/</a>.):<br>
No</p>
<h2 dir="auto">What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.):<br>
kubernetes, openstack, cloud-config, loadbalancer</h2>
<p dir="auto"><strong>Is this a BUG REPORT or FEATURE REQUEST?</strong> (choose one):<br>
BUG REPORT</p>
<p dir="auto"><strong>Kubernetes version</strong> (use <code class="notranslate">kubectl version</code>):</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="stack@kube-2:~$ kubectl version
Client Version: version.Info{Major:&quot;1&quot;, Minor:&quot;4&quot;, GitVersion:&quot;v1.4.4&quot;, GitCommit:&quot;3b417cc4ccd1b8f38ff9ec96bb50a81ca0ea9d56&quot;, GitTreeState:&quot;clean&quot;, BuildDate:&quot;2016-10-21T02:48:38Z&quot;, GoVersion:&quot;go1.6.3&quot;, Compiler:&quot;gc&quot;, Platform:&quot;linux/amd64&quot;}
Server Version: version.Info{Major:&quot;1&quot;, Minor:&quot;4&quot;, GitVersion:&quot;v1.4.4&quot;, GitCommit:&quot;3b417cc4ccd1b8f38ff9ec96bb50a81ca0ea9d56&quot;, GitTreeState:&quot;clean&quot;, BuildDate:&quot;2016-10-21T02:42:39Z&quot;, GoVersion:&quot;go1.6.3&quot;, Compiler:&quot;gc&quot;, Platform:&quot;linux/amd64&quot;}"><pre class="notranslate"><code class="notranslate">stack@kube-2:~$ kubectl version
Client Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.4", GitCommit:"3b417cc4ccd1b8f38ff9ec96bb50a81ca0ea9d56", GitTreeState:"clean", BuildDate:"2016-10-21T02:48:38Z", GoVersion:"go1.6.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.4", GitCommit:"3b417cc4ccd1b8f38ff9ec96bb50a81ca0ea9d56", GitTreeState:"clean", BuildDate:"2016-10-21T02:42:39Z", GoVersion:"go1.6.3", Compiler:"gc", Platform:"linux/amd64"}
</code></pre></div>
<p dir="auto"><strong>Environment</strong>:</p>
<ul dir="auto">
<li>
<p dir="auto"><strong>Cloud provider or hardware configuration</strong>:<br>
VM</p>
</li>
<li>
<p dir="auto"><strong>OS</strong> (e.g. from /etc/os-release):<br>
Ubuntu 16.04</p>
</li>
<li>
<p dir="auto"><strong>Kernel</strong> (e.g. <code class="notranslate">uname -a</code>):</p>
</li>
</ul>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="stack@kube-2:~$ uname -a
Linux kube-2 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:07:12 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux"><pre class="notranslate"><code class="notranslate">stack@kube-2:~$ uname -a
Linux kube-2 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:07:12 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
</code></pre></div>
<ul dir="auto">
<li>
<p dir="auto"><strong>Install tools</strong>:<br>
Installed K8S cluster using kubeadm</p>
</li>
<li>
<p dir="auto"><strong>Others</strong>:<br>
I am trying to use openstack's load balancer as an external load balancer for k8s services(type:loadbalancer). Currently my setup is as follows:<br>
One node k8s cluster installed on a VM with Ubuntu 16.04. I have another node where devstack is installed. I have the following details in the cloud config file:</p>
</li>
</ul>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="[Global]
auth-url=http://10.10.40.152/identity_admin
username=admin
password=password
tenant-name=admin
region=regionOne"><pre class="notranslate"><code class="notranslate">[Global]
auth-url=http://10.10.40.152/identity_admin
username=admin
password=password
tenant-name=admin
region=regionOne
</code></pre></div>
<p dir="auto">I have updated the kube-controller-manager.json and kube-apiserver.json with the following configuration:<br>
"--cloud-provider=openstack",<br>
"--cloud-config=/etc/sysconfig/kube_openstack_config",</p>
<p dir="auto"><strong>What happened</strong>:<br>
As soon as I updated the above config files, kube-controller-manager container crashes.</p>
<p dir="auto"><strong>What you expected to happen</strong>:<br>
kube-controller-manager is supposed to read the cloud config file and listen to kubernetes api calls and create load balancer on openstack.</p>
<p dir="auto"><strong>How to reproduce it</strong> (as minimally and precisely as possible):<br>
Update the above files with cloud-provider and cloud-config options.</p>
<p dir="auto"><strong>Anything else do we need to know</strong>:<br>
The k8s cluster is not running inside openstack/devstack. One node runs k8s cluster and another node is running devstack.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="stack@kube-2:~$ kubectl get pods --all-namespaces -o wide
NAMESPACE     NAME                              READY     STATUS             RESTARTS   AGE       IP                NODE
kube-system   calico-node-k9wsj                 2/2       Running            0          1h        10.10.40.151      kube-2
kube-system   calico-policy-controller-2snnc    1/1       Running            0          1h        10.10.40.151      kube-2
kube-system   dummy-2088944543-l5cmf            1/1       Running            0          1h        10.10.40.151      kube-2
kube-system   etcd-kube-2                       1/1       Running            0          1h        10.10.40.151      kube-2
kube-system   kube-apiserver-kube-2             1/1       Running            2          1h        10.10.40.151      kube-2
**kube-system   kube-controller-manager-kube-2    0/1       CrashLoopBackOff   28         3s        10.10.40.151      kube-2**
kube-system   kube-discovery-1150918428-iwg0p   1/1       Running            0          1h        10.10.40.151      kube-2
kube-system   kube-dns-654381707-cv1f1          3/3       Running            0          1h        192.168.212.198   kube-2
kube-system   kube-proxy-32pls                  1/1       Running            0          1h        10.10.40.151      kube-2
kube-system   kube-scheduler-kube-2             1/1       Running            0          1h        10.10.40.151      kube-2"><pre class="notranslate"><code class="notranslate">stack@kube-2:~$ kubectl get pods --all-namespaces -o wide
NAMESPACE     NAME                              READY     STATUS             RESTARTS   AGE       IP                NODE
kube-system   calico-node-k9wsj                 2/2       Running            0          1h        10.10.40.151      kube-2
kube-system   calico-policy-controller-2snnc    1/1       Running            0          1h        10.10.40.151      kube-2
kube-system   dummy-2088944543-l5cmf            1/1       Running            0          1h        10.10.40.151      kube-2
kube-system   etcd-kube-2                       1/1       Running            0          1h        10.10.40.151      kube-2
kube-system   kube-apiserver-kube-2             1/1       Running            2          1h        10.10.40.151      kube-2
**kube-system   kube-controller-manager-kube-2    0/1       CrashLoopBackOff   28         3s        10.10.40.151      kube-2**
kube-system   kube-discovery-1150918428-iwg0p   1/1       Running            0          1h        10.10.40.151      kube-2
kube-system   kube-dns-654381707-cv1f1          3/3       Running            0          1h        192.168.212.198   kube-2
kube-system   kube-proxy-32pls                  1/1       Running            0          1h        10.10.40.151      kube-2
kube-system   kube-scheduler-kube-2             1/1       Running            0          1h        10.10.40.151      kube-2
</code></pre></div>
<p dir="auto">Possibly a duplicate of an earlier issue. I can't delete pods and I have to keep re-trying <code class="notranslate">kubectl delete</code> which often fails. GCE.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
pods/serve-hostname-0-0
pods/serve-hostname-0-1
pods/serve-hostname-0-10
F0331 17:06:00.464445   15992 delete.go:67] Client error: 501: All the given peers are not reachable (failed to propose on members [http://127.0.0.1:4001] twice [last error: dial tcp 127.0.0.1:4001: too many open files]) [0]
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
pods/serve-hostname-0-11
pods/serve-hostname-0-12
pods/serve-hostname-0-13
pods/serve-hostname-0-14
pods/serve-hostname-0-15
pods/serve-hostname-0-16
pods/serve-hostname-0-17
F0331 17:06:21.511183   16000 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-0-18?quorum=false&amp;recursive=false&amp;sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
pods/serve-hostname-0-18
pods/serve-hostname-0-19
pods/serve-hostname-0-2
pods/serve-hostname-0-20
pods/serve-hostname-0-21
pods/serve-hostname-0-22
pods/serve-hostname-0-23
pods/serve-hostname-0-24
pods/serve-hostname-0-25
pods/serve-hostname-0-26
pods/serve-hostname-0-27
pods/serve-hostname-0-28
pods/serve-hostname-0-29
pods/serve-hostname-0-3
pods/serve-hostname-0-4
pods/serve-hostname-0-5
pods/serve-hostname-0-6
pods/serve-hostname-0-7
pods/serve-hostname-0-8
pods/serve-hostname-0-9
pods/serve-hostname-1-0
pods/serve-hostname-1-1
pods/serve-hostname-1-10
pods/serve-hostname-1-11
pods/serve-hostname-1-12
pods/serve-hostname-1-13
F0331 17:06:50.483268   16008 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-1-14?quorum=false&amp;recursive=false&amp;sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
pods/serve-hostname-1-14
pods/serve-hostname-1-15
pods/serve-hostname-1-16
pods/serve-hostname-1-17
pods/serve-hostname-1-18
pods/serve-hostname-1-19
pods/serve-hostname-1-2
pods/serve-hostname-1-20
pods/serve-hostname-1-21
pods/serve-hostname-1-22
pods/serve-hostname-1-23
pods/serve-hostname-1-24
pods/serve-hostname-1-25
pods/serve-hostname-1-26
pods/serve-hostname-1-27
pods/serve-hostname-1-28
pods/serve-hostname-1-29
F0331 17:07:17.744491   16017 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-1-3?quorum=false&amp;recursive=false&amp;sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
F0331 17:07:36.299212   16025 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-1-3?quorum=false&amp;recursive=false&amp;sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
7pods/serve-hostname-1-3
pods/serve-hostname-1-4                                                                                                                                                                                  pods/serve-hostname-1-5
pods/serve-hostname-1-6
pods/serve-hostname-1-7
pods/serve-hostname-1-8
F0331 17:08:21.421952   16049 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-1-9?quorum=false&amp;recursive=false&amp;sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
F0331 17:08:51.879079   16057 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506?quorum=false&amp;recursive=true&amp;sorted=true: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
F0331 17:08:55.015498   16068 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506?quorum=false&amp;recursive=true&amp;sorted=true: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
F0331 17:09:16.905368   16077 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-1-9?quorum=false&amp;recursive=false&amp;sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
pods/serve-hostname-1-9
F0331 17:14:20.492245   16139 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-10-0?quorum=false&amp;recursive=false&amp;sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
F0331 17:15:04.572098   16149 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-10-0?quorum=false&amp;recursive=false&amp;sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
pods/serve-hostname-10-0
pods/serve-hostname-10-1
pods/serve-hostname-10-10
pods/serve-hostname-10-11
pods/serve-hostname-10-12
pods/serve-hostname-10-13
pods/serve-hostname-10-14
pods/serve-hostname-10-15
pods/serve-hostname-10-16
pods/serve-hostname-10-17
pods/serve-hostname-10-18
pods/serve-hostname-10-19
pods/serve-hostname-10-2
pods/serve-hostname-10-20
pods/serve-hostname-10-21
pods/serve-hostname-10-22
pods/serve-hostname-10-23
F0331 17:17:15.837176   16174 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-10-24?quorum=false&amp;recursive=false&amp;sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$"><pre class="notranslate"><code class="notranslate">satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
pods/serve-hostname-0-0
pods/serve-hostname-0-1
pods/serve-hostname-0-10
F0331 17:06:00.464445   15992 delete.go:67] Client error: 501: All the given peers are not reachable (failed to propose on members [http://127.0.0.1:4001] twice [last error: dial tcp 127.0.0.1:4001: too many open files]) [0]
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
pods/serve-hostname-0-11
pods/serve-hostname-0-12
pods/serve-hostname-0-13
pods/serve-hostname-0-14
pods/serve-hostname-0-15
pods/serve-hostname-0-16
pods/serve-hostname-0-17
F0331 17:06:21.511183   16000 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-0-18?quorum=false&amp;recursive=false&amp;sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
pods/serve-hostname-0-18
pods/serve-hostname-0-19
pods/serve-hostname-0-2
pods/serve-hostname-0-20
pods/serve-hostname-0-21
pods/serve-hostname-0-22
pods/serve-hostname-0-23
pods/serve-hostname-0-24
pods/serve-hostname-0-25
pods/serve-hostname-0-26
pods/serve-hostname-0-27
pods/serve-hostname-0-28
pods/serve-hostname-0-29
pods/serve-hostname-0-3
pods/serve-hostname-0-4
pods/serve-hostname-0-5
pods/serve-hostname-0-6
pods/serve-hostname-0-7
pods/serve-hostname-0-8
pods/serve-hostname-0-9
pods/serve-hostname-1-0
pods/serve-hostname-1-1
pods/serve-hostname-1-10
pods/serve-hostname-1-11
pods/serve-hostname-1-12
pods/serve-hostname-1-13
F0331 17:06:50.483268   16008 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-1-14?quorum=false&amp;recursive=false&amp;sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
pods/serve-hostname-1-14
pods/serve-hostname-1-15
pods/serve-hostname-1-16
pods/serve-hostname-1-17
pods/serve-hostname-1-18
pods/serve-hostname-1-19
pods/serve-hostname-1-2
pods/serve-hostname-1-20
pods/serve-hostname-1-21
pods/serve-hostname-1-22
pods/serve-hostname-1-23
pods/serve-hostname-1-24
pods/serve-hostname-1-25
pods/serve-hostname-1-26
pods/serve-hostname-1-27
pods/serve-hostname-1-28
pods/serve-hostname-1-29
F0331 17:07:17.744491   16017 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-1-3?quorum=false&amp;recursive=false&amp;sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
F0331 17:07:36.299212   16025 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-1-3?quorum=false&amp;recursive=false&amp;sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
7pods/serve-hostname-1-3
pods/serve-hostname-1-4                                                                                                                                                                                  pods/serve-hostname-1-5
pods/serve-hostname-1-6
pods/serve-hostname-1-7
pods/serve-hostname-1-8
F0331 17:08:21.421952   16049 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-1-9?quorum=false&amp;recursive=false&amp;sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
F0331 17:08:51.879079   16057 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506?quorum=false&amp;recursive=true&amp;sorted=true: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
F0331 17:08:55.015498   16068 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506?quorum=false&amp;recursive=true&amp;sorted=true: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
F0331 17:09:16.905368   16077 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-1-9?quorum=false&amp;recursive=false&amp;sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
pods/serve-hostname-1-9
F0331 17:14:20.492245   16139 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-10-0?quorum=false&amp;recursive=false&amp;sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
F0331 17:15:04.572098   16149 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-10-0?quorum=false&amp;recursive=false&amp;sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$ kubectl delete pods --all --namespace=serve-hostnames-3506
pods/serve-hostname-10-0
pods/serve-hostname-10-1
pods/serve-hostname-10-10
pods/serve-hostname-10-11
pods/serve-hostname-10-12
pods/serve-hostname-10-13
pods/serve-hostname-10-14
pods/serve-hostname-10-15
pods/serve-hostname-10-16
pods/serve-hostname-10-17
pods/serve-hostname-10-18
pods/serve-hostname-10-19
pods/serve-hostname-10-2
pods/serve-hostname-10-20
pods/serve-hostname-10-21
pods/serve-hostname-10-22
pods/serve-hostname-10-23
F0331 17:17:15.837176   16174 delete.go:67] Client error: Get http://127.0.0.1:4001/v2/keys/registry/pods/serve-hostnames-3506/serve-hostname-10-24?quorum=false&amp;recursive=false&amp;sorted=false: dial tcp 127.0.0.1:4001: too many open files
satnam@asia-east1-b:~/gocode/src/github.com/GoogleCloudPlatform/kubernetes/test/soak/serve_hostnames$
</code></pre></div>