Recently I have been investigating one of my Linux server's speed using the copy [cp] command. This came about due to 'upgrading' the server from Centos 6 to 7. Once I had migrated a domain, I ran the following command
yes | cp -rf 'stg/core' 'tmpdir'
To my surprise, the first run of this command took over 38 minutes to complete. Crap! Yes a simple cp -a command is a very painful and slow process. Read more about the pain and times under Centos 7 in this article Allegedly a server upgrade to Centos 7. Is this a bit of Centos 7 love or a server configuration gone horribly wrong or time to change commands or a combination of all or some of these options.
After spending quite a bit of time playing around with cp command, I wanted to bench mark against another copy command. Step rsync.
Rsync to the rescue?
As the name indicates, rsync command is used to sync (or copy) files and directories. An important feature of rsync is that it works on “delta transfer algorithm”, meaning it will only sync or copy the changes from source to destination instead of copying the whole file. Ultimately reducing the amount of data sent over network.
| command | description |
|---|---|
| -v, –verbose | verbose output |
| -q, –quiet | suppress message output |
| -a, –archive | archive files and directory while synchronizing ( -a equal to following options -rlptgoD) |
| -r, --recursive | sync files and directories recursively |
| -b, --backup | take the backup during synchronization |
| -u, --update | don’t copy the files from source to destination if destination files are newer |
| -l, --links | copy symlinks as symlinks during the sync |
| -n, --dry-run | perform a trial run without synchronization |
| -e, --rsh=COMMAND | mention the remote shell to use in rsync |
| -z, --compress | compress file data during the transfer |
| -h, --human-readable | display the output numbers in a human-readable format |
| -P, --progress | show the sync progress during transfer |
| -S, --sparse | handle sparse files efficiently |
Example commands
| command | description | example |
|---|---|---|
| rsync -zavh | Copy or Sync directory locally | rsync -zavh /home/accounts /work |
| rsync -zarvh | Copy or sync files and directories from local to remote system | rsync -zarvh /home/accounts user@192.54.189.22/opt/path |
| rsync -zarvh --progress | Same as above, however this show synchronisation progress | rsync -zarvh --progress /home/accounts user@192.54.189.22/opt/path |
What command did I use to copy a directory with 109MB file size
rsync -arv httpdocs/core tmpdir
Post the initial run, subsequent runs will have the -u prompt added
rsync -aurv httpdocs/core tmpdir
For a complete list of rsync commands, run the command
rsync -h
Times
How did rsync compare to cp?
| run | cp | rsync |
|---|---|---|
| 1 | 7:09 | 0:08 |
| 2 | 0:01 | 0:00:10 |