Using Tsunami UDP in AWS- How it helps, Configuration and Comparison

Tsunami UDP is an open source file transfer protocol that helps you with faster files transfers compared to traditional models like FTP and SCP. Tsunami UDP is designed with a fast user-space file transfer protocol that uses TCP control and UDP data for transfer over very high speed long distance networks, it helps you achieve more throughput than possible with TCP over the same networks.

In this article we are going to use Tsunami UDP to transfer files from Amazon EC2 US-East region to other AWS regions.

Step 1:) Installing Tsunami UDP on Amazon EC2:

Tsunami UDP needs to be compiled from the source and we have used Red Hat Enterprise Linux 6.4 (RHEL 6.4) EC2 instance in the source and target regions.  Since we are using RHEL we have to ensure that firewall disabled on RHEL 6.4 EC2 instances or add rules to iptables.

TsunamiUDP needs some dependency packages and it should be installed from the repository cvs, autoconf, gcc, and automake.

Execute the following commands for Installing Tsunami UDP on Amazon EC2. The following command will install both Tsunami UDP Server and Tsunami UDP Client in the Amazon EC2 instance.

$ cvs -z3 -d:pserver:anonymous@tsunami-udp.cvs.sourceforge.net:/cvsroot/tsunami-udp co -P tsunami-udp $ cd tsunami-udp $ ./recompile.sh $ sudo make install

Tsunami UDP Server and Tsunami UDP Client have options to configure lots of parameters based on the file transfer requirements. For simplicity reasons we are going with default settings.

Step 2:) Open relevant ports in Security Groups for Tsunami UDP:

Tsunami UDP uses the port 46224 on TCP and UDP. We have allowed port 22 for SSH. The following ports illustrated in the below image should be allowed in the AWS security group for Tsunami UDP to transfer files.

Step 3) Transfer files from the Server Side
On the server side traverse to the directory/path where the files are available for transfer and start the Tsunami UDP Server daemon.
$ cd /path/to/files
For transferring single file:
$ tsunamid file1 
For transferring multiple files:
$ tsunamid file1 file2 …
For all files inside folder:
$ tsunamid *

When the file transfer is completed, you will get to know transfer details as shown below,

Transmission complete. Server 1 transferred 536870912 bytes in 62.07 seconds (66.0 Mbps)

Step 4) Receive files in the Tsunami UDP Client side

On the client side you can receive/download the files by connecting to Tsunami UDP Server using the Tsunami UDP client.

$ cd /path/to/receive/files $ tsunami tsunami> connect ec2-XX-XX-XX-83.compute-1.amazonaws.com  When the file transfer is completed, you will get to know transfer details like below Transfer complete. Flushing to disk and signaling server to stop… !!!! PC performance figure : 32 packets dropped (if high this indicates receiving PC overload) Transfer duration       : 62.06 seconds Total packet data       : 9440.27 Mbit Goodput data              : 8760.23 Mbit File data                       : 4096.00 Mbit Throughput                  : 152.12 Mbps Goodput w/ restarts  : 141.16 Mbps Final file rate               : 66.00 Mbps Transfer mode             : lossless Now lets see what we observed as Test results in comparison to SCP Tsunami UDP vs SCP Comparison Matrix: The following table illustrates the comparison (time taken) between Tsunami UDP and SCP between Amazon EC2 USA-EAST and AWS Singapore region. Source : US-EAST  | Target: Singapore EC2 Instance Type File Size / Type Tsunami UDP (time) SCP (time) m1.large 5GB Single File Compressed 2m 45s – m1.xlarge 5GB Single File Compressed 1m 28s – m2.4xlarge 5GB Single File Compressed 51.56s 11m 50s m2.4xlarge 50GB Single File Compressed 19m 33s 1hr 50min 50 GB Single compressed file when uncompressed translates to 500 GB file size. Some Best practices while using Tsunami UDP in AWS: Point 1) When you are moving larger files between Amazon EC2 regions, disk IO is very critical , so use RAID 0 EBS Volumes in the Amazon EC2 instances for better IO performance Point 2) Use right EC2 instance size for better performance and lower cost. If you are going to transfer 5 GB compressed file, m1.large will be cheaper whereas if you are going to transfer 500 GB compressed file, m2.4xlarge might be faster and cheaper. Point 3) Atleast 512 MB memory is needed for Tsunami UDP. Do not use micro, small and other EC2 instances with moderate IO and smaller memory for this purpose. Point 4) As the instance size increases the IO capacity also increases in Amazon cloud. For faster file transfers in event of DR etc, use larger instance types. Example: In event of DR when RTO is critical, you can transfer 50GB files using m2.4xlarge than m1.large.