Data transfer

There are multiple way to transfer data to or from Dwarf. Just to list the ones we will show in this tutorial

• SFTP
• rsync
• GridFTP
• FileZilla

SFTP

Let us introduce a command that is used in a terminal in order to connect with another machine, in our case it's Dwarf. 

sftp <USERNAME>@dwarf.if.pw.edu.pl

Here a <USERNAME> is Your account name on a remote server. Executing this command will bring You to the sftp> command line.

Here one can execute all basic linux commands like: ls, cd, pwd, mkdir... If You would like to operate on the local server (the one you are connecting from) You can operate by adding additional l letter that indicates that You operate on LOCAL server. Thus the basis commands looks like: lls, lcs, lpwd, lmkdir....

Executing pwd and lpwd command would print working directory that is on Dwarf (most likely /home/<USERNAME>), and local print working directory that is on Your LOCAL server (most likely /home/<USERNAME>, as well).

The power of sftp> comes from two commands put and get. Commands can be described graphically:

• get command means that you want to get files FROM the remote server
• put command means that you want to put files TO the remote server

The following picture presents the abilities of the sftp> technology:

rsync

rsync is very powerful technology, that is widely used by the NTG Group. At the end of this section we attach the bash script that can be executed on any machine. The main part of this script is the following command:

rsync -vrhe ssh --progress --append-verify <SOURCE> <DESTINATION>

Here <SOURCE> is the path to a folder or files that You want to copy to <DESTINATION>. rsync works in both ways, meaning that You can copy data TO or FROM any remote server being connected to Dwarf. 
Giving it by example. Imagine that we have the following list of files in ~/example_local directory on Your local machine:

and You want to send them to your ~/example_remote/ directory on Dwarf. Then one should execute the following command:

rsync -vrhe ssh --progress --append-verify ~/example_local/* <USERNAME>@dwarf.if.pw.edu.pl:~/example_remote/

 rsync_copydata

GRID FTP

In progress...

FileZilla

In progress...

Tutorial was prepared by Andrzej Makowski.