<p dir="auto">Bringing up a cluster using the new Multiple-AZ Support fails on AWS.  The first part of the cluster comes up but we fail on bringing up the second set of nodes.  Below is the config to start up:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="# this part succeeds
export KUBERNETES_PROVIDER=aws
KUBE_AWS_ZONE=us-west-2a kube-up.sh

# this command fails with: Could not detect Kubernetes master node IP.  Make sure you've launched a cluster with 'kube-up.sh'
KUBE_AWS_ZONE=us-west-2b KUBE_SUBNET_CIDR=172.20.1.0/24 KUBE_USE_EXISTING_MASTER=true kube-up.sh"><pre class="notranslate"><code class="notranslate"># this part succeeds
export KUBERNETES_PROVIDER=aws
KUBE_AWS_ZONE=us-west-2a kube-up.sh

# this command fails with: Could not detect Kubernetes master node IP.  Make sure you've launched a cluster with 'kube-up.sh'
KUBE_AWS_ZONE=us-west-2b KUBE_SUBNET_CIDR=172.20.1.0/24 KUBE_USE_EXISTING_MASTER=true kube-up.sh
</code></pre></div>
<p dir="auto">And here is the log of the failure to bring up the second set of nodes:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Creating autoscaling group
 0 minions started; waiting
 0 minions started; waiting
 0 minions started; waiting
 0 minions started; waiting
 2 minions started; ready
Sanity checking cluster...
Attempt 1 to check Docker on node @ &lt;someIP&gt; ...not working yet
Attempt 2 to check Docker on node @ &lt;someIP&gt; ...not working yet


Attempt 3 to check Docker on node @ &lt;someIP&gt; ...not working yet
Attempt 4 to check Docker on node @ &lt;someIP&gt; ...not working yet
Attempt 5 to check Docker on node @ &lt;someIP&gt; ...not working yet
Attempt 6 to check Docker on node @ &lt;someIP&gt; ...not working yet


Attempt 7 to check Docker on node @ &lt;someIP&gt; ...not working yet
Attempt 8 to check Docker on node @ &lt;someIP&gt; ...not working yet
Attempt 9 to check Docker on node @ &lt;someIP&gt; ...not working yet
Attempt 10 to check Docker on node @ &lt;someIP&gt; ...not working yet
Attempt 11 to check Docker on node @ &lt;someIP&gt; ...not working yet

Your cluster is unlikely to work correctly.
Please run ./cluster/kube-down.sh and re-create the
cluster. (sorry!)"><pre class="notranslate"><code class="notranslate">Creating autoscaling group
 0 minions started; waiting
 0 minions started; waiting
 0 minions started; waiting
 0 minions started; waiting
 2 minions started; ready
Sanity checking cluster...
Attempt 1 to check Docker on node @ &lt;someIP&gt; ...not working yet
Attempt 2 to check Docker on node @ &lt;someIP&gt; ...not working yet


Attempt 3 to check Docker on node @ &lt;someIP&gt; ...not working yet
Attempt 4 to check Docker on node @ &lt;someIP&gt; ...not working yet
Attempt 5 to check Docker on node @ &lt;someIP&gt; ...not working yet
Attempt 6 to check Docker on node @ &lt;someIP&gt; ...not working yet


Attempt 7 to check Docker on node @ &lt;someIP&gt; ...not working yet
Attempt 8 to check Docker on node @ &lt;someIP&gt; ...not working yet
Attempt 9 to check Docker on node @ &lt;someIP&gt; ...not working yet
Attempt 10 to check Docker on node @ &lt;someIP&gt; ...not working yet
Attempt 11 to check Docker on node @ &lt;someIP&gt; ...not working yet

Your cluster is unlikely to work correctly.
Please run ./cluster/kube-down.sh and re-create the
cluster. (sorry!)
</code></pre></div>
<p dir="auto">The Kubernetes 1.3 release tarball is 1.4G.  That is insane.  While we know that much of this is due to the fact that we include all architectures, it make kubernetes feel heavier than it is.</p>
<p dir="auto">Much of the space is the fact that 4 server platforms are now created and packaged with everything else.  Each target is a ~325M tarball.  These tarballs include both individual binaries along with Docker images that include those binaries.  They include kubectl (which is available naked in the release).</p>
<p dir="auto">We also have 9 copies of kubectl for various targets totaling 450MB.</p>
<p dir="auto">Things we can do to slim this down:</p>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> Start building release tars per-architecture.  Either include non-arch specific stuff in every tar or break that out too. (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="164399443" data-permission-text="Title is private" data-url="https://github.com/kubernetes/kubernetes/issues/28629" data-hovercard-type="issue" data-hovercard-url="/kubernetes/kubernetes/issues/28629/hovercard" href="https://github.com/kubernetes/kubernetes/issues/28629">#28629</a>, owner?)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> Finish transition to hyperkube and deprecate the individual binaries.  . We could move the individual binaries into a different tar for those that still need them.  For <code class="notranslate">amd64</code> hyperkube is 132M uncompressed and 26M compressed.  This means we could go from a 350M compressed tar for the amd64 target down to 26M. (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="114091814" data-permission-text="Title is private" data-url="https://github.com/kubernetes/kubernetes/issues/16508" data-hovercard-type="issue" data-hovercard-url="/kubernetes/kubernetes/issues/16508/hovercard" href="https://github.com/kubernetes/kubernetes/issues/16508">#16508</a>, owner?)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> Have the ability to build the Docker images on the fly.  It should be easy to automate (as part of the binary) creating a Dockerfile, referencing the running binary and driving docker.  <code class="notranslate">hyperkube admin create-container-image</code> could do this. We could also support <code class="notranslate">hyperkube admin create-container-image --aci</code> and <code class="notranslate">hyperkube admin create-container-image --docker --just-dockerfile</code>. (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="164399974" data-permission-text="Title is private" data-url="https://github.com/kubernetes/kubernetes/issues/28630" data-hovercard-type="issue" data-hovercard-url="/kubernetes/kubernetes/issues/28630/hovercard" href="https://github.com/kubernetes/kubernetes/issues/28630">#28630</a>, owner?)
<ul dir="auto">
<li>I understand that gzip does a decent job of de-duping stuff here.  But the uncompressed footprint balloons and looks bad.</li>
</ul>
</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> Break out kubectl as a separate download from the server components.  <code class="notranslate">gcloud</code> already ships it separately.  The user cases here differ dramatically. (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="164400426" data-permission-text="Title is private" data-url="https://github.com/kubernetes/kubernetes/issues/28631" data-hovercard-type="issue" data-hovercard-url="/kubernetes/kubernetes/issues/28631/hovercard" href="https://github.com/kubernetes/kubernetes/issues/28631">#28631</a>, owner?)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> Reevaluate all of our targets.  Do we really need 32bit Windows? (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="164400843" data-permission-text="Title is private" data-url="https://github.com/kubernetes/kubernetes/issues/28632" data-hovercard-type="issue" data-hovercard-url="/kubernetes/kubernetes/issues/28632/hovercard" href="https://github.com/kubernetes/kubernetes/issues/28632">#28632</a>, owner?)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> Why are the federation targets new binaries? They add 150M uncompressed for amd64. (I don't know enough to say here but could these be folded in to hyperkube?  I'm guessing they could.) (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="164401146" data-permission-text="Title is private" data-url="https://github.com/kubernetes/kubernetes/issues/28633" data-hovercard-type="issue" data-hovercard-url="/kubernetes/kubernetes/issues/28633/hovercard" href="https://github.com/kubernetes/kubernetes/issues/28633">#28633</a>, <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/madhusudancs/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/madhusudancs">@madhusudancs</a>)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> Don't ship binaries that we don't really need.  We included <code class="notranslate">kubemark</code> here.  No end users will actually use that.  This is 100M uncompressed. (<a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="164401535" data-permission-text="Title is private" data-url="https://github.com/kubernetes/kubernetes/issues/28635" data-hovercard-type="issue" data-hovercard-url="/kubernetes/kubernetes/issues/28635/hovercard" href="https://github.com/kubernetes/kubernetes/issues/28635">#28635</a>, owner?)</li>
</ul>
<p dir="auto">Also related: <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="160465886" data-permission-text="Title is private" data-url="https://github.com/kubernetes/kubernetes/issues/27444" data-hovercard-type="issue" data-hovercard-url="/kubernetes/kubernetes/issues/27444/hovercard" href="https://github.com/kubernetes/kubernetes/issues/27444">#27444</a> (Builds are taking too long)</p>