<p dir="auto">(k8s 1.1.4)</p>
<p dir="auto">After restarting my master server components, 2 out of 5 of my DaemonSet pods were terminated (why?) and never recreated. It seems like all <code class="notranslate">ds</code> pods were requeued during this event, however the IDs of the currently running pods do not even match what is reported by the DaemonSet.</p>
<ul dir="auto">
<li>Restarting the <code class="notranslate">kubelet</code> on faulty nodes does nothing but re-adding a SyncLoop</li>
<li>Restarting the <code class="notranslate">kube-controller-manager</code> again shows <code class="notranslate">Waiting for pods controller to sync, requeuing ds ingress-apis</code></li>
<li><code class="notranslate">kube-scheduler</code> shows no relevant information</li>
</ul>
<p dir="auto">The first message in the kubelet log ("... The image pull may not succeed.") makes me think it <em>might</em> be related to <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="123204322" data-permission-text="Title is private" data-url="https://github.com/kubernetes/kubernetes/issues/18947" data-hovercard-type="issue" data-hovercard-url="/kubernetes/kubernetes/issues/18947/hovercard" href="https://github.com/kubernetes/kubernetes/issues/18947">#18947</a>, however:</p>
<ul dir="auto">
<li>a pull should not have been triggered since the imagePullPolicy is <code class="notranslate">IfNotPresent</code>, and the image was present before the restart</li>
<li><code class="notranslate">SyncLoop (ADD)</code> shouldn't have happened in the first place since 5 pods were already running before the restart</li>
</ul>
<h2 dir="auto"></h2>
<p dir="auto">Current state of the ds:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Name:       ingress-apis
Image(s):   nginx:1.9.7
Selector:   app=ingress,tier=backend
Node-Selector:  &lt;none&gt;
Labels:     app=ingress,tier=backend
Desired Number of Nodes Scheduled: 5
Current Number of Nodes Scheduled: 5
Number of Nodes Misscheduled: 0
Pods Status:    3 Running / 0 Waiting / 0 Succeeded / 2 Failed
Events:
  FirstSeen LastSeen    Count   From        SubobjectPath   Reason          Message
  ─────────   ────────    ───── ────        ───────────── ──────          ───────
  40m       40m     1   {daemon-set }           SuccessfulCreate    Created pod: ingress-apis-0svoi
  40m       40m     1   {daemon-set }           SuccessfulCreate    Created pod: ingress-apis-qn8tk
  39m       39m     1   {daemon-set }           SuccessfulCreate    Created pod: ingress-apis-4p04e
  39m       39m     1   {daemon-set }           SuccessfulCreate    Created pod: ingress-apis-2ncfy
  39m       39m     1   {daemon-set }           SuccessfulCreate    Created pod: ingress-apis-v6iuf"><pre class="notranslate"><code class="notranslate">Name:       ingress-apis
Image(s):   nginx:1.9.7
Selector:   app=ingress,tier=backend
Node-Selector:  &lt;none&gt;
Labels:     app=ingress,tier=backend
Desired Number of Nodes Scheduled: 5
Current Number of Nodes Scheduled: 5
Number of Nodes Misscheduled: 0
Pods Status:    3 Running / 0 Waiting / 0 Succeeded / 2 Failed
Events:
  FirstSeen LastSeen    Count   From        SubobjectPath   Reason          Message
  ─────────   ────────    ───── ────        ───────────── ──────          ───────
  40m       40m     1   {daemon-set }           SuccessfulCreate    Created pod: ingress-apis-0svoi
  40m       40m     1   {daemon-set }           SuccessfulCreate    Created pod: ingress-apis-qn8tk
  39m       39m     1   {daemon-set }           SuccessfulCreate    Created pod: ingress-apis-4p04e
  39m       39m     1   {daemon-set }           SuccessfulCreate    Created pod: ingress-apis-2ncfy
  39m       39m     1   {daemon-set }           SuccessfulCreate    Created pod: ingress-apis-v6iuf
</code></pre></div>
<h2 dir="auto"></h2>
<p dir="auto">Current pods:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="NAME                 READY     STATUS    RESTARTS   AGE
ingress-apis-jwlqo   1/1       Running   0          19h
ingress-apis-mabp8   1/1       Running   0          17h
ingress-apis-rte9q   1/1       Running   0          19h"><pre class="notranslate"><code class="notranslate">NAME                 READY     STATUS    RESTARTS   AGE
ingress-apis-jwlqo   1/1       Running   0          19h
ingress-apis-mabp8   1/1       Running   0          17h
ingress-apis-rte9q   1/1       Running   0          19h
</code></pre></div>
<h2 dir="auto"></h2>
<p dir="auto"><code class="notranslate">kubelet</code> logs on a faulty node:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="--Jan 20 09:44:11--
kubelet.go:1477] Unable to retrieve pull secret default/docker-registry for default/ingress-apis-mensq due to Get https://k8s.freeletics.com/api/v1/namespaces/default/secrets/docker-registry: dial tcp 10.0.31.59:443: connection refused.  The image pull may not succeed.
kubelet.go:2012] SyncLoop (ADD): &quot;ingress-apis-0svoi_default&quot;
kubelet.go:1862] Pod &quot;ingress-apis-0svoi_default&quot;: HostPort is already allocated, ignoring: [[0].port: duplicate value '9080/TCP']
kubelet.go:2015] SyncLoop (UPDATE): &quot;ingress-apis-mensq_default&quot;
manager.go:1404] Killing container &quot;bac67ab2623eb60b6d23e714feee4d74be6b5c3e7417f07b08bd874d2183245f nginx default/ingress-apis-mensq&quot; with 30 second grace period
manager.go:1438] Container &quot;bac67ab2623eb60b6d23e714feee4d74be6b5c3e7417f07b08bd874d2183245f nginx default/ingress-apis-mensq&quot; exited after 68.511739ms
manager.go:1444] No ref for pod '&quot;bac67ab2623eb60b6d23e714feee4d74be6b5c3e7417f07b08bd874d2183245f nginx default/ingress-apis-mensq&quot;'
manager.go:1404] Killing container &quot;eb0b4d358a9afe7aa2ac18d7f6eec57a6d2cd54506e4ae18096c7996c011de77 default/ingress-apis-mensq&quot; with 30 second grace period
manager.go:1438] Container &quot;eb0b4d358a9afe7aa2ac18d7f6eec57a6d2cd54506e4ae18096c7996c011de77 default/ingress-apis-mensq&quot; exited after 213.225172ms
manager.go:1444] No ref for pod '&quot;eb0b4d358a9afe7aa2ac18d7f6eec57a6d2cd54506e4ae18096c7996c011de77 default/ingress-apis-mensq&quot;'
kubelet.go:2015] SyncLoop (UPDATE): &quot;ingress-apis-mensq_default&quot;
kubelet.go:2018] SyncLoop (REMOVE): &quot;ingress-apis-mensq_default&quot;
kubelet.go:2108] Failed to delete pod &quot;ingress-apis-mensq_default&quot;, err: pod not found
--Jan 20 09:44:22--"><pre class="notranslate"><code class="notranslate">--Jan 20 09:44:11--
kubelet.go:1477] Unable to retrieve pull secret default/docker-registry for default/ingress-apis-mensq due to Get https://k8s.freeletics.com/api/v1/namespaces/default/secrets/docker-registry: dial tcp 10.0.31.59:443: connection refused.  The image pull may not succeed.
kubelet.go:2012] SyncLoop (ADD): "ingress-apis-0svoi_default"
kubelet.go:1862] Pod "ingress-apis-0svoi_default": HostPort is already allocated, ignoring: [[0].port: duplicate value '9080/TCP']
kubelet.go:2015] SyncLoop (UPDATE): "ingress-apis-mensq_default"
manager.go:1404] Killing container "bac67ab2623eb60b6d23e714feee4d74be6b5c3e7417f07b08bd874d2183245f nginx default/ingress-apis-mensq" with 30 second grace period
manager.go:1438] Container "bac67ab2623eb60b6d23e714feee4d74be6b5c3e7417f07b08bd874d2183245f nginx default/ingress-apis-mensq" exited after 68.511739ms
manager.go:1444] No ref for pod '"bac67ab2623eb60b6d23e714feee4d74be6b5c3e7417f07b08bd874d2183245f nginx default/ingress-apis-mensq"'
manager.go:1404] Killing container "eb0b4d358a9afe7aa2ac18d7f6eec57a6d2cd54506e4ae18096c7996c011de77 default/ingress-apis-mensq" with 30 second grace period
manager.go:1438] Container "eb0b4d358a9afe7aa2ac18d7f6eec57a6d2cd54506e4ae18096c7996c011de77 default/ingress-apis-mensq" exited after 213.225172ms
manager.go:1444] No ref for pod '"eb0b4d358a9afe7aa2ac18d7f6eec57a6d2cd54506e4ae18096c7996c011de77 default/ingress-apis-mensq"'
kubelet.go:2015] SyncLoop (UPDATE): "ingress-apis-mensq_default"
kubelet.go:2018] SyncLoop (REMOVE): "ingress-apis-mensq_default"
kubelet.go:2108] Failed to delete pod "ingress-apis-mensq_default", err: pod not found
--Jan 20 09:44:22--
</code></pre></div>
<h2 dir="auto"></h2>
<p dir="auto"><code class="notranslate">kube-controller-manager</code> logs:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="--Jan 20 09:44:14--
controller.go:493] Waiting for pods controller to sync, requeuing ds ingress-apis
controller.go:493] Waiting for pods controller to sync, requeuing ds ingress-apis
controller.go:493] Waiting for pods controller to sync, requeuing ds ingress-apis
controller.go:493] Waiting for pods controller to sync, requeuing ds ingress-apis
controller.go:493] Waiting for pods controller to sync, requeuing ds ingress-apis
event.go:206] Event(api.ObjectReference{Kind:&quot;DaemonSet&quot;, Namespace:&quot;default&quot;, Name:&quot;ingress-apis&quot;, UID:&quot;4b05bf3e-9c18-11e5-b4d2-0a59d1e77755&quot;, APIVersion:&quot;extensions&quot;, ResourceVersion:&quot;63618203&quot;, FieldPath:&quot;&quot;}): reason: 'SuccessfulCreate' Created pod: ingress-apis-0svoi
event.go:206] Event(api.ObjectReference{Kind:&quot;DaemonSet&quot;, Namespace:&quot;default&quot;, Name:&quot;ingress-apis&quot;, UID:&quot;4b05bf3e-9c18-11e5-b4d2-0a59d1e77755&quot;, APIVersion:&quot;extensions&quot;, ResourceVersion:&quot;63618203&quot;, FieldPath:&quot;&quot;}): reason: 'SuccessfulCreate' Created pod: ingress-apis-qn8tk
event.go:206] Event(api.ObjectReference{Kind:&quot;DaemonSet&quot;, Namespace:&quot;default&quot;, Name:&quot;ingress-apis&quot;, UID:&quot;4b05bf3e-9c18-11e5-b4d2-0a59d1e77755&quot;, APIVersion:&quot;extensions&quot;, ResourceVersion:&quot;63618203&quot;, FieldPath:&quot;&quot;}): reason: 'SuccessfulCreate' Created pod: ingress-apis-4p04e
event.go:206] Event(api.ObjectReference{Kind:&quot;DaemonSet&quot;, Namespace:&quot;default&quot;, Name:&quot;ingress-apis&quot;, UID:&quot;4b05bf3e-9c18-11e5-b4d2-0a59d1e77755&quot;, APIVersion:&quot;extensions&quot;, ResourceVersion:&quot;63618203&quot;, FieldPath:&quot;&quot;}): reason: 'SuccessfulCreate' Created pod: ingress-apis-2ncfy
event.go:206] Event(api.ObjectReference{Kind:&quot;DaemonSet&quot;, Namespace:&quot;default&quot;, Name:&quot;ingress-apis&quot;, UID:&quot;4b05bf3e-9c18-11e5-b4d2-0a59d1e77755&quot;, APIVersion:&quot;extensions&quot;, ResourceVersion:&quot;63618203&quot;, FieldPath:&quot;&quot;}): reason: 'SuccessfulCreate' Created pod: ingress-apis-v6iuf
--Jan 20 09:44:20--"><pre class="notranslate"><code class="notranslate">--Jan 20 09:44:14--
controller.go:493] Waiting for pods controller to sync, requeuing ds ingress-apis
controller.go:493] Waiting for pods controller to sync, requeuing ds ingress-apis
controller.go:493] Waiting for pods controller to sync, requeuing ds ingress-apis
controller.go:493] Waiting for pods controller to sync, requeuing ds ingress-apis
controller.go:493] Waiting for pods controller to sync, requeuing ds ingress-apis
event.go:206] Event(api.ObjectReference{Kind:"DaemonSet", Namespace:"default", Name:"ingress-apis", UID:"4b05bf3e-9c18-11e5-b4d2-0a59d1e77755", APIVersion:"extensions", ResourceVersion:"63618203", FieldPath:""}): reason: 'SuccessfulCreate' Created pod: ingress-apis-0svoi
event.go:206] Event(api.ObjectReference{Kind:"DaemonSet", Namespace:"default", Name:"ingress-apis", UID:"4b05bf3e-9c18-11e5-b4d2-0a59d1e77755", APIVersion:"extensions", ResourceVersion:"63618203", FieldPath:""}): reason: 'SuccessfulCreate' Created pod: ingress-apis-qn8tk
event.go:206] Event(api.ObjectReference{Kind:"DaemonSet", Namespace:"default", Name:"ingress-apis", UID:"4b05bf3e-9c18-11e5-b4d2-0a59d1e77755", APIVersion:"extensions", ResourceVersion:"63618203", FieldPath:""}): reason: 'SuccessfulCreate' Created pod: ingress-apis-4p04e
event.go:206] Event(api.ObjectReference{Kind:"DaemonSet", Namespace:"default", Name:"ingress-apis", UID:"4b05bf3e-9c18-11e5-b4d2-0a59d1e77755", APIVersion:"extensions", ResourceVersion:"63618203", FieldPath:""}): reason: 'SuccessfulCreate' Created pod: ingress-apis-2ncfy
event.go:206] Event(api.ObjectReference{Kind:"DaemonSet", Namespace:"default", Name:"ingress-apis", UID:"4b05bf3e-9c18-11e5-b4d2-0a59d1e77755", APIVersion:"extensions", ResourceVersion:"63618203", FieldPath:""}): reason: 'SuccessfulCreate' Created pod: ingress-apis-v6iuf
--Jan 20 09:44:20--
</code></pre></div>
<hr>
<p dir="auto">Before someone asks, although this is obvious, I solved it by deleting and recreating the <code class="notranslate">ds</code> (identically):</p>
<h2 dir="auto"></h2>
<p dir="auto"><code class="notranslate">kubelet</code> logs on a previously faulty node:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="--Jan 20 10:52:33--
kubelet.go:2015] SyncLoop (UPDATE): &quot;ingress-apis-0svoi_default&quot;
kubelet.go:2018] SyncLoop (REMOVE): &quot;ingress-apis-0svoi_default&quot;
kubelet.go:2108] Failed to delete pod &quot;ingress-apis-0svoi_default&quot;, err: pod not found
kubelet.go:2012] SyncLoop (ADD): &quot;ingress-apis-9us1a_default&quot;
manager.go:1707] Need to restart pod infra container for &quot;ingress-apis-9us1a_default&quot; because it is not found
--Jan 20 10:52:34--"><pre class="notranslate"><code class="notranslate">--Jan 20 10:52:33--
kubelet.go:2015] SyncLoop (UPDATE): "ingress-apis-0svoi_default"
kubelet.go:2018] SyncLoop (REMOVE): "ingress-apis-0svoi_default"
kubelet.go:2108] Failed to delete pod "ingress-apis-0svoi_default", err: pod not found
kubelet.go:2012] SyncLoop (ADD): "ingress-apis-9us1a_default"
manager.go:1707] Need to restart pod infra container for "ingress-apis-9us1a_default" because it is not found
--Jan 20 10:52:34--
</code></pre></div>
<h2 dir="auto"></h2>
<p dir="auto"><code class="notranslate">kube-controller-manager</code> logs:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="--Jan 20 10:52:40--
event.go:206] Event(api.ObjectReference{Kind:&quot;DaemonSet&quot;, Namespace:&quot;default&quot;, Name:&quot;ingress-apis&quot;, UID:&quot;ed06f7f4-bf63-11e5-91dc-0a59d1e77755&quot;, APIVersion:&quot;extensions&quot;, ResourceVersion:&quot;64118991&quot;, FieldPath:&quot;&quot;}): reason: 'SuccessfulCreate' Created pod: ingress-apis-hgcn1
event.go:206] Event(api.ObjectReference{Kind:&quot;DaemonSet&quot;, Namespace:&quot;default&quot;, Name:&quot;ingress-apis&quot;, UID:&quot;ed06f7f4-bf63-11e5-91dc-0a59d1e77755&quot;, APIVersion:&quot;extensions&quot;, ResourceVersion:&quot;64118991&quot;, FieldPath:&quot;&quot;}): reason: 'SuccessfulCreate' Created pod: ingress-apis-9us1a
event.go:206] Event(api.ObjectReference{Kind:&quot;DaemonSet&quot;, Namespace:&quot;default&quot;, Name:&quot;ingress-apis&quot;, UID:&quot;ed06f7f4-bf63-11e5-91dc-0a59d1e77755&quot;, APIVersion:&quot;extensions&quot;, ResourceVersion:&quot;64118991&quot;, FieldPath:&quot;&quot;}): reason: 'SuccessfulCreate' Created pod: ingress-apis-lsobz
event.go:206] Event(api.ObjectReference{Kind:&quot;DaemonSet&quot;, Namespace:&quot;default&quot;, Name:&quot;ingress-apis&quot;, UID:&quot;ed06f7f4-bf63-11e5-91dc-0a59d1e77755&quot;, APIVersion:&quot;extensions&quot;, ResourceVersion:&quot;64118991&quot;, FieldPath:&quot;&quot;}): reason: 'SuccessfulCreate' Created pod: ingress-apis-ogjrw
event.go:206] Event(api.ObjectReference{Kind:&quot;DaemonSet&quot;, Namespace:&quot;default&quot;, Name:&quot;ingress-apis&quot;, UID:&quot;ed06f7f4-bf63-11e5-91dc-0a59d1e77755&quot;, APIVersion:&quot;extensions&quot;, ResourceVersion:&quot;64118991&quot;, FieldPath:&quot;&quot;}): reason: 'SuccessfulCreate' Created pod: ingress-apis-9tae0
--Jan 20 10:52:41--"><pre class="notranslate"><code class="notranslate">--Jan 20 10:52:40--
event.go:206] Event(api.ObjectReference{Kind:"DaemonSet", Namespace:"default", Name:"ingress-apis", UID:"ed06f7f4-bf63-11e5-91dc-0a59d1e77755", APIVersion:"extensions", ResourceVersion:"64118991", FieldPath:""}): reason: 'SuccessfulCreate' Created pod: ingress-apis-hgcn1
event.go:206] Event(api.ObjectReference{Kind:"DaemonSet", Namespace:"default", Name:"ingress-apis", UID:"ed06f7f4-bf63-11e5-91dc-0a59d1e77755", APIVersion:"extensions", ResourceVersion:"64118991", FieldPath:""}): reason: 'SuccessfulCreate' Created pod: ingress-apis-9us1a
event.go:206] Event(api.ObjectReference{Kind:"DaemonSet", Namespace:"default", Name:"ingress-apis", UID:"ed06f7f4-bf63-11e5-91dc-0a59d1e77755", APIVersion:"extensions", ResourceVersion:"64118991", FieldPath:""}): reason: 'SuccessfulCreate' Created pod: ingress-apis-lsobz
event.go:206] Event(api.ObjectReference{Kind:"DaemonSet", Namespace:"default", Name:"ingress-apis", UID:"ed06f7f4-bf63-11e5-91dc-0a59d1e77755", APIVersion:"extensions", ResourceVersion:"64118991", FieldPath:""}): reason: 'SuccessfulCreate' Created pod: ingress-apis-ogjrw
event.go:206] Event(api.ObjectReference{Kind:"DaemonSet", Namespace:"default", Name:"ingress-apis", UID:"ed06f7f4-bf63-11e5-91dc-0a59d1e77755", APIVersion:"extensions", ResourceVersion:"64118991", FieldPath:""}): reason: 'SuccessfulCreate' Created pod: ingress-apis-9tae0
--Jan 20 10:52:41--
</code></pre></div>
<p dir="auto"><a href="https://travis-ci.org/GoogleCloudPlatform/kubernetes/jobs/69776608" rel="nofollow">https://travis-ci.org/GoogleCloudPlatform/kubernetes/jobs/69776608</a></p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="panic: test timed out after 5m0s
goroutine 37 [running]:
runtime.panic(0xaef900, 0xc2080aa020)
    /usr/local/go/src/pkg/runtime/panic.c:279 +0xf5
testing.func·008()
    /usr/local/go/src/pkg/testing/testing.go:629 +0x12e
created by time.goFunc
    /usr/local/go/src/pkg/time/sleep.go:121 +0x55
goroutine 16 [chan receive, 2 minutes]:
testing.RunTests(0xee2130, 0x13b34a0, 0x4, 0x4, 0xe3fe01)
    /usr/local/go/src/pkg/testing/testing.go:505 +0xbb8
testing.Main(0xee2130, 0x13b34a0, 0x4, 0x4, 0x13e7900, 0x0, 0x0, 0x13e7900, 0x0, 0x0)
    /usr/local/go/src/pkg/testing/testing.go:435 +0xa3
main.main()
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_testmain.go:105 +0x18c
goroutine 19 [finalizer wait, 5 minutes]:
runtime.park(0x435ff0, 0x13de610, 0x13b8349)
    /usr/local/go/src/pkg/runtime/proc.c:1369 +0x89
runtime.parkunlock(0x13de610, 0x13b8349)
    /usr/local/go/src/pkg/runtime/proc.c:1385 +0x3b
runfinq()
    /usr/local/go/src/pkg/runtime/mgc0.c:2644 +0xcf
runtime.goexit()
    /usr/local/go/src/pkg/runtime/proc.c:1445
goroutine 33 [chan receive]:
github.com/golang/glog.(*loggingT).flushDaemon(0x13e1280)
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/Godeps/_workspace/src/github.com/golang/glog/glog.go:879 +0x96
created by github.com/golang/glog.init·1
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/Godeps/_workspace/src/github.com/golang/glog/glog.go:410 +0x2d6
goroutine 17 [syscall, 5 minutes]:
runtime.goexit()
    /usr/local/go/src/pkg/runtime/proc.c:1445
goroutine 52 [select, 2 minutes]:
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.(*etcdMasterElector).run(0xc20804d1a0, 0xd24260, 0x3, 0xd21a40, 0x3)
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/etcd_master.go:49 +0x3db
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.func·001()
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/etcd_master.go:37 +0xb9
github.com/GoogleCloudPlatform/kubernetes/pkg/util.func·004()
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/util/util.go:112 +0x5f
github.com/GoogleCloudPlatform/kubernetes/pkg/util.Until(0xc20804d1c0, 0x12a05f200, 0x0)
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/util/util.go:113 +0xbc
github.com/GoogleCloudPlatform/kubernetes/pkg/util.Forever(0xc20804d1c0, 0x12a05f200)
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/util/util.go:94 +0x48
created by github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.(*etcdMasterElector).Elect
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/etcd_master.go:37 +0x1f6
goroutine 49 [select, 2 minutes]:
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.(*etcdMasterElector).run(0xc20804d0e0, 0xd24260, 0x3, 0xd21a40, 0x3)
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/etcd_master.go:49 +0x3db
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.func·001()
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/etcd_master.go:37 +0xb9
github.com/GoogleCloudPlatform/kubernetes/pkg/util.func·004()
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/util/util.go:112 +0x5f
github.com/GoogleCloudPlatform/kubernetes/pkg/util.Until(0xc20804d100, 0x12a05f200, 0x0)
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/util/util.go:113 +0xbc
github.com/GoogleCloudPlatform/kubernetes/pkg/util.Forever(0xc20804d100, 0x12a05f200)
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/util/util.go:94 +0x48
created by github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.(*etcdMasterElector).Elect
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/etcd_master.go:37 +0x1f6
goroutine 55 [select, 2 minutes]:
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.(*etcdMasterElector).run(0xc20804d320, 0xd24260, 0x3, 0xd21a40, 0x3)
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/etcd_master.go:49 +0x3db
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.func·001()
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/etcd_master.go:37 +0xb9
github.com/GoogleCloudPlatform/kubernetes/pkg/util.func·004()
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/util/util.go:112 +0x5f
github.com/GoogleCloudPlatform/kubernetes/pkg/util.Until(0xc20804d340, 0x12a05f200, 0x0)
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/util/util.go:113 +0xbc
github.com/GoogleCloudPlatform/kubernetes/pkg/util.Forever(0xc20804d340, 0x12a05f200)
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/util/util.go:94 +0x48
created by github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.(*etcdMasterElector).Elect
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/etcd_master.go:37 +0x1f6
goroutine 58 [select, 2 minutes]:
github.com/GoogleCloudPlatform/kubernetes/pkg/watch.(*Broadcaster).distribute(0xc2080b1110, 0xd372d0, 0x8, 0x7f1745fd6ef0, 0xc208132d20)
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/watch/mux.go:182 +0x3bd
github.com/GoogleCloudPlatform/kubernetes/pkg/watch.(*Broadcaster).loop(0xc2080b1110)
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/watch/mux.go:163 +0xc3
created by github.com/GoogleCloudPlatform/kubernetes/pkg/watch.NewBroadcaster
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/watch/mux.go:70 +0x127
goroutine 57 [chan receive, 2 minutes]:
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.Test(0xc2080a6750)
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/master_test.go:83 +0x306
testing.tRunner(0xc2080a6750, 0x13b34e8)
    /usr/local/go/src/pkg/testing/testing.go:422 +0x110
created by testing.RunTests
    /usr/local/go/src/pkg/testing/testing.go:504 +0xb46
goroutine 59 [select, 2 minutes]:
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.(*notifier).serviceLoop(0xc208093b30, 0xc2080977a0)
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/master.go:105 +0x363
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.func·005()
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/master.go:87 +0x90
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime.func·008()
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime/util.go:115 +0x5f
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime.Until(0x7f174580bec0, 0x0, 0xc2080976e0)
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime/util.go:116 +0xd3
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.Notify(0x7f1745fd7030, 0xc20804d460, 0xd16620, 0x0, 0xd26520, 0x2, 0x7f1745fd7058, 0xc20804d480, 0xc2080976e0)
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/master.go:87 +0x429
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.func·006()
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/master_test.go:72 +0x145
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime.func·007()
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime/util.go:95 +0xaa
created by github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime.After
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime/util.go:97 +0xf7
goroutine 61 [chan send, 2 minutes]:
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.(*Fake).Elect(0xc20804d460, 0xd16620, 0x0, 0xd26520, 0x2, 0x0, 0x0)
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/fake.go:38 +0x23e
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.func·003()
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/master.go:45 +0x13c
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime.func·008()
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime/util.go:115 +0x5f
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime.Until(0x7f1745807f50, 0x0, 0xc2080976e0)
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime/util.go:116 +0xd3
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.func·004()
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/master.go:84 +0x14f
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime.func·007()
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime/util.go:95 +0xaa
created by github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime.After
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime/util.go:97 +0xf7
goroutine 62 [semacquire, 2 minutes]:
sync.runtime_Syncsemacquire(0xc2080a54d0)
    /usr/local/go/src/pkg/runtime/sema.goc:257 +0xc0
sync.(*Cond).Wait(0xc2080a54c0)
    /usr/local/go/src/pkg/sync/cond.go:62 +0xb5
sync.*Cond.Wait·fm()
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/master.go:104 +0x34
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime.func·007()
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime/util.go:95 +0xaa
created by github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime.After
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime/util.go:97 +0xf7
FAIL    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election    300.121s"><pre class="notranslate"><code class="notranslate">panic: test timed out after 5m0s
goroutine 37 [running]:
runtime.panic(0xaef900, 0xc2080aa020)
    /usr/local/go/src/pkg/runtime/panic.c:279 +0xf5
testing.func·008()
    /usr/local/go/src/pkg/testing/testing.go:629 +0x12e
created by time.goFunc
    /usr/local/go/src/pkg/time/sleep.go:121 +0x55
goroutine 16 [chan receive, 2 minutes]:
testing.RunTests(0xee2130, 0x13b34a0, 0x4, 0x4, 0xe3fe01)
    /usr/local/go/src/pkg/testing/testing.go:505 +0xbb8
testing.Main(0xee2130, 0x13b34a0, 0x4, 0x4, 0x13e7900, 0x0, 0x0, 0x13e7900, 0x0, 0x0)
    /usr/local/go/src/pkg/testing/testing.go:435 +0xa3
main.main()
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_testmain.go:105 +0x18c
goroutine 19 [finalizer wait, 5 minutes]:
runtime.park(0x435ff0, 0x13de610, 0x13b8349)
    /usr/local/go/src/pkg/runtime/proc.c:1369 +0x89
runtime.parkunlock(0x13de610, 0x13b8349)
    /usr/local/go/src/pkg/runtime/proc.c:1385 +0x3b
runfinq()
    /usr/local/go/src/pkg/runtime/mgc0.c:2644 +0xcf
runtime.goexit()
    /usr/local/go/src/pkg/runtime/proc.c:1445
goroutine 33 [chan receive]:
github.com/golang/glog.(*loggingT).flushDaemon(0x13e1280)
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/Godeps/_workspace/src/github.com/golang/glog/glog.go:879 +0x96
created by github.com/golang/glog.init·1
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/Godeps/_workspace/src/github.com/golang/glog/glog.go:410 +0x2d6
goroutine 17 [syscall, 5 minutes]:
runtime.goexit()
    /usr/local/go/src/pkg/runtime/proc.c:1445
goroutine 52 [select, 2 minutes]:
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.(*etcdMasterElector).run(0xc20804d1a0, 0xd24260, 0x3, 0xd21a40, 0x3)
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/etcd_master.go:49 +0x3db
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.func·001()
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/etcd_master.go:37 +0xb9
github.com/GoogleCloudPlatform/kubernetes/pkg/util.func·004()
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/util/util.go:112 +0x5f
github.com/GoogleCloudPlatform/kubernetes/pkg/util.Until(0xc20804d1c0, 0x12a05f200, 0x0)
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/util/util.go:113 +0xbc
github.com/GoogleCloudPlatform/kubernetes/pkg/util.Forever(0xc20804d1c0, 0x12a05f200)
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/util/util.go:94 +0x48
created by github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.(*etcdMasterElector).Elect
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/etcd_master.go:37 +0x1f6
goroutine 49 [select, 2 minutes]:
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.(*etcdMasterElector).run(0xc20804d0e0, 0xd24260, 0x3, 0xd21a40, 0x3)
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/etcd_master.go:49 +0x3db
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.func·001()
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/etcd_master.go:37 +0xb9
github.com/GoogleCloudPlatform/kubernetes/pkg/util.func·004()
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/util/util.go:112 +0x5f
github.com/GoogleCloudPlatform/kubernetes/pkg/util.Until(0xc20804d100, 0x12a05f200, 0x0)
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/util/util.go:113 +0xbc
github.com/GoogleCloudPlatform/kubernetes/pkg/util.Forever(0xc20804d100, 0x12a05f200)
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/util/util.go:94 +0x48
created by github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.(*etcdMasterElector).Elect
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/etcd_master.go:37 +0x1f6
goroutine 55 [select, 2 minutes]:
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.(*etcdMasterElector).run(0xc20804d320, 0xd24260, 0x3, 0xd21a40, 0x3)
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/etcd_master.go:49 +0x3db
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.func·001()
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/etcd_master.go:37 +0xb9
github.com/GoogleCloudPlatform/kubernetes/pkg/util.func·004()
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/util/util.go:112 +0x5f
github.com/GoogleCloudPlatform/kubernetes/pkg/util.Until(0xc20804d340, 0x12a05f200, 0x0)
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/util/util.go:113 +0xbc
github.com/GoogleCloudPlatform/kubernetes/pkg/util.Forever(0xc20804d340, 0x12a05f200)
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/util/util.go:94 +0x48
created by github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.(*etcdMasterElector).Elect
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/etcd_master.go:37 +0x1f6
goroutine 58 [select, 2 minutes]:
github.com/GoogleCloudPlatform/kubernetes/pkg/watch.(*Broadcaster).distribute(0xc2080b1110, 0xd372d0, 0x8, 0x7f1745fd6ef0, 0xc208132d20)
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/watch/mux.go:182 +0x3bd
github.com/GoogleCloudPlatform/kubernetes/pkg/watch.(*Broadcaster).loop(0xc2080b1110)
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/watch/mux.go:163 +0xc3
created by github.com/GoogleCloudPlatform/kubernetes/pkg/watch.NewBroadcaster
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/watch/mux.go:70 +0x127
goroutine 57 [chan receive, 2 minutes]:
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.Test(0xc2080a6750)
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/master_test.go:83 +0x306
testing.tRunner(0xc2080a6750, 0x13b34e8)
    /usr/local/go/src/pkg/testing/testing.go:422 +0x110
created by testing.RunTests
    /usr/local/go/src/pkg/testing/testing.go:504 +0xb46
goroutine 59 [select, 2 minutes]:
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.(*notifier).serviceLoop(0xc208093b30, 0xc2080977a0)
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/master.go:105 +0x363
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.func·005()
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/master.go:87 +0x90
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime.func·008()
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime/util.go:115 +0x5f
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime.Until(0x7f174580bec0, 0x0, 0xc2080976e0)
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime/util.go:116 +0xd3
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.Notify(0x7f1745fd7030, 0xc20804d460, 0xd16620, 0x0, 0xd26520, 0x2, 0x7f1745fd7058, 0xc20804d480, 0xc2080976e0)
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/master.go:87 +0x429
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.func·006()
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/master_test.go:72 +0x145
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime.func·007()
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime/util.go:95 +0xaa
created by github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime.After
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime/util.go:97 +0xf7
goroutine 61 [chan send, 2 minutes]:
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.(*Fake).Elect(0xc20804d460, 0xd16620, 0x0, 0xd26520, 0x2, 0x0, 0x0)
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/fake.go:38 +0x23e
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.func·003()
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/master.go:45 +0x13c
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime.func·008()
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime/util.go:115 +0x5f
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime.Until(0x7f1745807f50, 0x0, 0xc2080976e0)
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime/util.go:116 +0xd3
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election.func·004()
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/master.go:84 +0x14f
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime.func·007()
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime/util.go:95 +0xaa
created by github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime.After
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime/util.go:97 +0xf7
goroutine 62 [semacquire, 2 minutes]:
sync.runtime_Syncsemacquire(0xc2080a54d0)
    /usr/local/go/src/pkg/runtime/sema.goc:257 +0xc0
sync.(*Cond).Wait(0xc2080a54c0)
    /usr/local/go/src/pkg/sync/cond.go:62 +0xb5
sync.*Cond.Wait·fm()
    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election/_test/_obj_test/master.go:104 +0x34
github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime.func·007()
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime/util.go:95 +0xaa
created by github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime.After
    /home/travis/gopath/src/github.com/GoogleCloudPlatform/kubernetes/_output/local/go/src/github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/runtime/util.go:97 +0xf7
FAIL    github.com/GoogleCloudPlatform/kubernetes/contrib/mesos/pkg/election    300.121s
</code></pre></div>
<p dir="auto">/cc <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/jdef/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/jdef">@jdef</a>, <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/davidopp/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/davidopp">@davidopp</a></p>