---
layout: post
title: bbcp
tags: [Network, Linux, Security]
---

In this post, we document how we installed and configured [BBCP](https://www.slac.stanford.edu/~abh/bbcp/) on [pulpo-dtn]({{ site.baseurl }}{% post_url 2017-2-9-pulpos %}).<!-- more -->

* Table of Contents
{:toc}

## Installation
For simplicity, We simply downloaded a precompiled [bbcp binary executable](http://www.slac.stanford.edu/~abh/bbcp/bin/) and placed it in `/usr/local/bin`:
{% highlight shell_session %}
# cd /usr/local/bin/
# wget http://www.slac.stanford.edu/~abh/bbcp/bin/amd64_rhel60/bbcp
# chmod +x bbcp
{% endhighlight %}

We note in passing that although the binary executable was built for 64-bit RHEL 6, it works without issue on RHEL/CentOS 7.

Create account for Jeffrey LeFevre on **pulpo-dtn**:
{% highlight shell_session %}
# ldapsearch -x -H ldap://ldap-blue.ucsc.edu -LLL 'uid=jlefevre'
dn: uid=jlefevre,ou=people,dc=ucsc,dc=edu
gidNumber: 100000
uidNumber: 28981

# groupadd -g 28981 jlefevre
# useradd -u 28981 -g jlefevre -c "Jeffrey LeFevre" -m -d /mnt/pulpos/jlefevre jlefevre
{% endhighlight %}

## Firewall
Append the following 2 lines to `/etc/services`:
{% highlight conf %}
bbcpfirst       60000/tcp               # bbcp
bbcplast        60015/tcp               # bbcp
{% endhighlight %}

Open inbound TCP ports 60000 - 60015 from any using FirewallD:
{% highlight shell_session %}
# firewall-cmd --permanent --zone=public --add-port=60000-60015/tcp
success

# firewall-cmd --reload
success
{% endhighlight %}

## Testing
Similarly install BBCP on on the 4-GPU workstation [Hydra]({{ site.baseurl }}{% post_url 2017-7-28-hydra %}).

Transfer a 4GB file from *hydra* to *pulpo-dtn*:
{% highlight shell_session %}
$ bbcp -P 1 -F 4GB.dat jlefevre@pulpo-dtn.ucsc.edu:/mnt/pulpos/jlefevre/4GB.dat
{% endhighlight %}

**NOTE**:

1). We must use `-F` option, which forces the copy by not checking if there is enough free space on the target host, in order work around a bug in Ceph Luminous; otherwise we'll get the following error:
{% highlight plaintext %}
bbcp: Insufficient space to copy all the files from hydra.soe.ucsc.edu
{% endhighlight %}

2). BBCP doesn't honor *ssh_config*. If I place the following stanza in `~/.ssh/config` on hydra:
{% highlight conf %}
Host pd
        HostName pulpo-dtn.ucsc.edu
        User jlefevre
        IdentityFile ~/.ssh/pulpos
{% endhighlight %}
and attempt a transfer using the following command (`pd` instead of `jlefevre@pulpo-dtn.ucsc.edu`):
{% highlight shell_session %}
$ bbcp -P 1 -F 4GB.dat pd:/mnt/pulpos/jlefevre/4GB.dat
{% endhighlight %}
the command will fail:
{% highlight plaintext %}
bbcp: Unable to connect to  (null) ;  Name or service not known
bbcp: Unable to allocate more than 0 of 4 data streams.
bbcp: Accept timed out on port 60000
bbcp: Unable to allocate more than 0 of 4 data streams.
{% endhighlight %}

3). If we use a private SSH key that is not the default ~/.ssh/identity, ~/.ssh/id_dsa, ~/.ssh/id_ecdsa, ~/.ssh/id_ed25519 or ~/.ssh/id_rsa*id_rsa*, we can use the `-i` option to specify the key. For example:
{% highlight shell_session %}
$ bbcp -i ~/.ssh/pulpos -P 1 -F 4GB.dat jlefevre@pulpo-dtn.ucsc.edu:/mnt/pulpos/jlefevre/4GB.dat
{% endhighlight %}
