Transfer all backups over a single TCP/IP connection
Currently, our server spend a lot of time backing up to 3 different sources. Knowing that our server previously had a cap of 250 Mbit/s outgoing bandwidth and one of our remote backup servers it was and is 2 Gbit/s.
So I wondered why it took so long.
After a recent upgrade, where we increased the cap from 250 Mbit/s to 1 Gbit/s, and where we got SSD's in RAID 10 instead of SAS 15K RPM in RAID 10, we got backup generation time down from an hour or so to a few minutes.
However backup transportation to remote hosts still take as long time.
Reviewing the cpbackup_transporter.log file, I can see that it take 1 file and first transfer to each backup destination, then the next file, and so on.
This means that there is created a TCP/IP connection to the backup server, 1 file is transferred, and the connection is closed again. Sometimes only a few MB.
If instead 1 TCP/IP connection was made, where all backups to that destination is transferred at once, that would remove the overhead of creating and destroying TCP/IP connections, and at the same time make it possible to transfer the backups faster.
This is because (as far as I know) TCP/IP connections start out slow, then going faster and faster as it realizes it can use more bandwidth.
However it'll never realize that when it only transfer tons of small files.