<p dir="auto">The new <code class="notranslate">ansible_ssh_common_args</code> lets me add a <code class="notranslate">ProxyCommand</code><br>
argument to the <code class="notranslate">ssh</code> invocation for entire inventory groups, which<br>
simplifies use of an ssh jump host.</p>
<p dir="auto">My understanding (from reading tickets) is that this was implemented<br>
to permit use of ProxyCommand: (1) without having to override the<br>
whole <code class="notranslate">ssh_args</code>, or alternatively (2) without needing to use the<br>
<code class="notranslate">.ssh/config</code> of the invoking user.</p>
<p dir="auto">However, when using <code class="notranslate">ProxyCommand</code> this way, the control socket<br>
in the filesystem is keyed off the far hop (inner tunnel) of the jump,<br>
which means the near hop does <em>not</em> use a persistent socket.  If<br>
another host needs to be reached via the jump gate, an entirely<br>
new socket will be set up, even though we could reuse it for the<br>
inner portion (first hop) of the jump.  If reaching 1000 remote hosts<br>
via a jump gate, we will make 1000 separate ssh invocations, each<br>
with its own control socket, and no re-use (it only reuses if same<br>
far-host is destination)</p>
<p dir="auto">This could be optimized by using ControlPath to describe only the<br>
near hop, and then use <em>another</em> ssh with ControlPath on the far<br>
side.  I don't know if this is a real issue or not, but thought it was<br>
worth submitting a ticket for.  I think it would involve making a<br>
jump-host scenario a "first class" connection property that was<br>
implemented in code that could then be "nested" and duplicated<br>
on the remote end (rather than just a string-append to ssh args).</p>
<p dir="auto">If this is just totally crazy, by all means close ticket.  I think it could<br>
help some scenarios though, it would avoid doing the first-half of the<br>
jump circuit 999 times in the case of 1000 jump targets on the far<br>
side.  So it would have to do only 1001 total ssh setups, rather than<br>
2000, which could really be a big win...</p>
<p dir="auto">While running an Ansible playbook, it will intermittently hang during a yum install.  It doesn't happen all the time, but it happens somewhat frequently.  It looks like it is waiting on a socket response that never comes.</p>
<p dir="auto">The environment is</p>
<p dir="auto">Boot2Docker version 1.7.0, build master : 7960f90<br>
Docker version 1.7.0, build 0baf609<br>
Container is based on centos:centos6<br>
Ansible version is 1.9.2</p>
<p dir="auto">When I go into the actual box that Ansible is trying to install on, this is what I see:<br>
[root@um-apache tmp]# ps -efL | grep repoquery<br>
root       637   512   637  0    2 21:17 pts/0    00:00:00 /usr/bin/python -tt /usr/bin/repoquery --show-duplicates --plugins --quiet -q --disablerepo  --enablerepo  --qf %{name}-%{version}-%{release}.%{arch} unzip<br>
root       637   512   638  0    2 21:17 pts/0    00:00:00 /usr/bin/python -tt /usr/bin/repoquery --show-duplicates --plugins --quiet -q --disablerepo  --enablerepo  --qf %{name}-%{version}-%{release}.%{arch} unzip</p>
<p dir="auto">That just sits around forever unless I manually kill it.</p>
<p dir="auto">When trying to debug the problem, first I was checking to see if repoquery might be conflicted over a lock, and this is what I see:</p>
<p dir="auto">[root@um-apache tmp]# lsof | grep 637<br>
repoquery  637     root  cwd    DIR               0,66     4096    206 /home/callidus<br>
repoquery  637     root  rtd    DIR               0,66     4096      2 /<br>
repoquery  637     root  txt    REG               0,66     4864    236 /usr/bin/python<br>
repoquery  637     root  mem    REG               0,66    27424    186 /lib64/libnss_dns-2.12.so<br>
repoquery  637     root  mem    REG               0,66    50280    661 /usr/lib64/python2.6/lib-dynload/pyexpat.so<br>
repoquery  637     root  mem    REG               0,66   165264    656 /lib64/libexpat.so.1.5.2<br>
repoquery  637     root  mem    REG               0,66    42392    653 /usr/lib64/python2.6/lib-dynload/_elementtree.so<br>
repoquery  637     root  mem    REG               0,66    25160    565 /usr/lib64/python2.6/lib-dynload/termios.so<br>
repoquery  637     root  mem    REG               0,66  1382496    558 /usr/lib64/libxml2.so.2.7.6<br>
repoquery  637     root  mem    REG               0,66  1140496    556 /lib64/libglib-2.0.so.0.2800.8<br>
repoquery  637     root  mem    REG               0,66    45320    271 /usr/lib64/python2.6/site-packages/_sqlitecache.so<br>
repoquery  637     root  mem    REG               0,66    77104    554 /usr/lib64/python2.6/lib-dynload/_sqlite3.so<br>
repoquery  637     root  mem    REG               0,66    81256    553 /usr/lib64/python2.6/lib-dynload/datetime.so<br>
repoquery  637     root  mem    REG               0,66     7208    546 /usr/lib64/python2.6/lib-dynload/_weakref.so<br>
repoquery  637     root  mem    REG               0,66     9080    527 /usr/lib64/python2.6/lib-dynload/grpmodule.so<br>
repoquery  637     root  mem    REG               0,66    14288    512 /lib64/libgpg-error.so.0.5.0<br>
repoquery  637     root  mem    REG               0,66   197792    510 /usr/lib64/libgpgme.so.11.6.6<br>
repoquery  637     root  mem    REG               0,66    78016    508 /usr/lib64/python2.6/site-packages/gpgme/_gpgme.so<br>
repoquery  637     root  mem    REG               0,66    35696    503 /usr/lib64/python2.6/lib-dynload/bz2.so<br>
repoquery  637     root  mem    REG               0,66    23784    496 /usr/lib64/python2.6/lib-dynload/zlibmodule.so<br>
repoquery  637     root  mem    REG               0,66    10312    144 /lib64/libfreebl3.so<br>
repoquery  637     root  mem    REG               0,66    40400    131 /lib64/libcrypt-2.12.so<br>
repoquery  637     root  mem    REG               0,66   106160    465 /usr/lib64/libsasl2.so.2.0.23<br>
repoquery  637     root  mem    REG               0,66    60512    463 /lib64/liblber-2.4.so.2.10.3<br>
repoquery  637     root  mem    REG               0,66   161856    461 /usr/lib64/libssh2.so.1.0.1<br>
repoquery  637     root  mem    REG               0,66   181176    459 /usr/lib64/libsmime3.so<br>
repoquery  637     root  mem    REG               0,66   254328    458 /usr/lib64/libssl3.so<br>
repoquery  637     root  mem    REG               0,66   326768    457 /lib64/libldap-2.4.so.2.10.3<br>
repoquery  637     root  mem    REG               0,66   206672    455 /lib64/libidn.so.11.6.1<br>
repoquery  637     root  mem    REG               0,66   347536    453 /usr/lib64/libcurl.so.4.1.1<br>
repoquery  637     root  mem    REG               0,66    69504    268 /usr/lib64/python2.6/site-packages/pycurl.so<br>
repoquery  637     root  mem    REG               0,66     9872    451 /usr/lib64/python2.6/lib-dynload/_bisectmodule.so<br>
repoquery  637     root  mem    REG               0,66    41408    444 /usr/lib64/python2.6/lib-dynload/arraymodule.so<br>
repoquery  637     root  mem    REG               0,66    15432    441 /usr/lib64/python2.6/lib-dynload/_hashlib.so<br>
repoquery  637     root  mem    REG               0,66    14632    402 /usr/lib64/python2.6/lib-dynload/fcntlmodule.so<br>
repoquery  637     root  mem    REG               0,66    12680    401 /usr/lib64/python2.6/lib-dynload/_randommodule.so<br>
repoquery  637     root  mem    REG               0,66    20976    400 /usr/lib64/python2.6/lib-dynload/binascii.so<br>
repoquery  637     root  mem    REG               0,66    26408    399 /usr/lib64/python2.6/lib-dynload/mathmodule.so<br>
repoquery  637     root  mem    REG               0,66    24432    392 /usr/lib64/python2.6/lib-dynload/selectmodule.so<br>
repoquery  637     root  mem    REG               0,66    75664    386 /usr/lib64/python2.6/lib-dynload/cPickle.so<br>
repoquery  637     root  mem    REG               0,66   110960    133 /lib64/libresolv-2.12.so<br>
repoquery  637     root  mem    REG               0,66    10192    148 /lib64/libkeyutils.so.1.3<br>
repoquery  637     root  mem    REG               0,66    43728    146 /lib64/libkrb5support.so.0.1<br>
repoquery  637     root  mem    REG               0,66   174840    139 /lib64/libk5crypto.so.3.1<br>
repoquery  637     root  mem    REG               0,66    14664    141 /lib64/libcom_err.so.2.1<br>
repoquery  637     root  mem    REG               0,66   946048    137 /lib64/libkrb5.so.3.3<br>
repoquery  637     root  mem    REG               0,66   277704    135 /lib64/libgssapi_krb5.so.2.2<br>
repoquery  637     root  mem    REG               0,66  1963296    123 /usr/lib64/libcrypto.so.1.0.1e<br>
repoquery  637     root  mem    REG               0,66   441256    385 /usr/lib64/libssl.so.1.0.1e<br>
repoquery  637     root  mem    REG               0,66    34112    383 /usr/lib64/python2.6/lib-dynload/_ssl.so<br>
repoquery  637     root  mem    REG               0,66    60752    382 /usr/lib64/python2.6/lib-dynload/_socketmodule.so<br>
repoquery  637     root  mem    REG               0,66   115768    339 /usr/lib64/libmagic.so.1.0.0<br>
repoquery  637     root  mem    REG               0,66   155344    337 /usr/lib64/librpmbuild.so.1.0.0<br>
repoquery  637     root  mem    REG               0,66    12560    335 /usr/lib64/python2.6/site-packages/rpm/_rpmbmodule.so<br>
repoquery  637     root  mem    REG               0,66   477712    211 /lib64/libfreeblpriv3.so<br>
repoquery  637     root  mem    REG               0,66   583552    320 /usr/lib64/libsqlite3.so.0.8.6<br>
repoquery  637     root  mem    REG               0,66   262168    318 /usr/lib64/libsoftokn3.so<br>
repoquery  637     root  mem    REG               0,66    65928     37 /lib64/libnss_files-2.12.so<br>
repoquery  637     root  mem    REG               0,66   244624    151 /lib64/libnspr4.so<br>
repoquery  637     root  mem    REG               0,66    14560    150 /lib64/libplds4.so<br>
repoquery  637     root  mem    REG               0,66    18720    143 /lib64/libplc4.so<br>
repoquery  637     root  mem    REG               0,66   179416    149 /usr/lib64/libnssutil3.so<br>
repoquery  637     root  mem    REG               0,66    90880    108 /lib64/libgcc_s-4.4.7-20120601.so.1<br>
repoquery  637     root  mem    REG               0,66    18712    231 /lib64/libattr.so.1.1.0<br>
repoquery  637     root  mem    REG               0,66    43880    106 /lib64/librt-2.12.so<br>
repoquery  637     root  mem    REG               0,66   183920    317 /usr/lib64/liblua-5.1.so<br>
repoquery  637     root  mem    REG               0,66   134920    316 /usr/lib64/liblzma.so.0.0.0<br>
repoquery  637     root  mem    REG               0,66    36360    314 /lib64/libpopt.so.0.0.0<br>
repoquery  637     root  mem    REG               0,66    88440    312 /usr/lib64/libelf-0.161.so<br>
repoquery  637     root  mem    REG               0,66    88600    127 /lib64/libz.so.1.2.3<br>
repoquery  637     root  mem    REG               0,66    67592    310 /lib64/libbz2.so.1.0.4<br>
repoquery  637     root  mem    REG               0,66  1300048    142 /usr/lib64/libnss3.so<br>
repoquery  637     root  mem    REG               0,66   183968    308 /usr/lib64/librpmio.so.1.0.0<br>
repoquery  637     root  mem    REG               0,66  1523064    306 /lib64/libdb-4.7.so<br>
repoquery  637     root  mem    REG               0,66    31280    229 /lib64/libacl.so.1.1.0<br>
repoquery  637     root  mem    REG               0,66    16600    305 /lib64/libcap.so.2.16<br>
repoquery  637     root  mem    REG               0,66   122040    152 /lib64/libselinux.so.1<br>
repoquery  637     root  mem    REG               0,66   435400    303 /usr/lib64/librpm.so.1.0.0<br>
repoquery  637     root  mem    REG               0,66    92872    301 /usr/lib64/python2.6/site-packages/rpm/_rpmmodule.so<br>
repoquery  637     root  mem    REG               0,66    19248    359 /usr/lib64/python2.6/lib-dynload/cStringIO.so<br>
repoquery  637     root  mem    REG               0,66    37840    389 /usr/lib64/python2.6/lib-dynload/_struct.so<br>
repoquery  637     root  mem    REG               0,66    21608    410 /usr/lib64/python2.6/lib-dynload/_localemodule.so<br>
repoquery  637     root  mem    REG               0,66    12256    366 /usr/lib64/python2.6/lib-dynload/_functoolsmodule.so<br>
repoquery  637     root  mem    REG               0,66    25288    358 /usr/lib64/python2.6/lib-dynload/stropmodule.so<br>
repoquery  637     root  mem    REG               0,66    54896    373 /usr/lib64/python2.6/lib-dynload/itertoolsmodule.so<br>
repoquery  637     root  mem    REG               0,66    38608    370 /usr/lib64/python2.6/lib-dynload/operator.so<br>
repoquery  637     root  mem    REG               0,66    28112    369 /usr/lib64/python2.6/lib-dynload/_collectionsmodule.so<br>
repoquery  637     root  mem    REG               0,66    20328    348 /usr/lib64/python2.6/lib-dynload/timemodule.so<br>
repoquery  637     root  mem    REG               0,66  1920896     32 /lib64/libc-2.12.so<br>
repoquery  637     root  mem    REG               0,66   596272    239 /lib64/libm-2.12.so<br>
repoquery  637     root  mem    REG               0,66    14584    125 /lib64/libutil-2.12.so<br>
repoquery  637     root  mem    REG               0,66    19536     30 /lib64/libdl-2.12.so<br>
repoquery  637     root  mem    REG               0,66   142640    110 /lib64/libpthread-2.12.so<br>
repoquery  637     root  mem    REG               0,66  1669840    237 /usr/lib64/libpython2.6.so.1.0<br>
repoquery  637     root  mem    REG               0,66   154664     25 /lib64/ld-2.12.so<br>
repoquery  637     root    0u   CHR              136,0      0t0      3 /dev/pts/0<br>
repoquery  637     root    1w  FIFO                0,8      0t0 261194 pipe<br>
repoquery  637     root    2w  FIFO                0,8      0t0 261195 pipe<br>
repoquery  637     root    3w   REG               0,66     2531    628 /var/log/yum.log<br>
repoquery  637     root    4u  sock                0,6      0t0 261197 can't identify protocol<br>
repoquery  637     root    7u  sock                0,6      0t0 258410 can't identify protocol</p>
<p dir="auto">When I look at what the process is actually doing at that moment, this is what I see<br>
[root@um-apache tmp]# strace -p 637<br>
Process 637 attached<br>
futex(0x12410c0, FUTEX_WAIT_PRIVATE, 0, NULL</p>
<p dir="auto">^CProcess 637 detached<br>
&lt;detached ...&gt;<br>
[root@um-apache tmp]# strace -p 638<br>
Process 638 attached<br>
recvmsg(7, ^CProcess 638 detached<br>
&lt;detached ...&gt;</p>