corporate LDAP. Inbound traffic is encrypted and controlled by a firewall. Only ports that are related to cluster administration (Ambari), reverse proxy (Knox), JDBC ports (BigSQL and Hive), and SSH are open for users. Outbound traffic is not restricted. After users log in to the management node, they can connect via Secure Shell (SSH) to data nodes that are connected to the private network. All inbound traffic from the corporate network to data nodes is blocked. Data nodes are connected only to the cluster data network or to the private network. 

Management nodes in the cluster have two network interfaces: 

One of the interfaces is connected to the corporate network (also called a public network).  

The other interface is connected to the private network (sometimes referred to as a data network).  

All the data traffic is exchanged between management node and data nodes through the private network only. Thus, there is dedicated bandwidth and higher performance with the added benefit of security. 

## Setting up the data nodes to access external data sources with Apache Sqoop and similar tools 

One of the use cases for big data analytics is to offload the organization's relational database management system (RDBMS) data to Apache Hadoop for archival or running analytics at large scale. Tools such as Apache Sqoop and IBM Fluid Query are used to import data from external sources. These tools launch MapReduce jobs, which read the data from external sources in parallel. 

In this scenario, port forwarding in the firewall must be enabled, so that the data nodes can read external sources by forwarding the traffic to or from management nodes. 

Run the following commands as root on the management nodes to enable port forwarding between data nodes and management nodes. Data nodes can then initiate communication to servers outside of the private network and receive data, but external servers cannot access the internal network. 

echo 1 > /proc/sys/net/ipv4/ip_foward /sbin/iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE /sbin/iptables -A FORWARD -i eth0 -o eth1 -m state --state RELATED,ESTABLISHED -j ACCEPT /sbin/iptables -A FORWARD -i eth1 -o eth0 -j ACCEPT 

These commands assume eth0  is the public interface. The eth1  traffic is routed to outside of the network as though it were coming from the management node. Adjust the interface names ( eth0  and eth1 ) to match your environment. After the data is completely imported, port forwarding can be turned off. 

## Related publications 

For more information, see the following web page: 

IBM BigInsights V4.2 documentation https://ibm.biz/BdrnVB 

