cPanel & WHM Version 98 has been released, and brings a slew of great updates. Take a look at what is included, and then upgrade today!

Amazon S3 Adjustable Chunk Size

Jeremy shared this idea 5 years ago
Open Discussion

The Amazon S3 backup script hard codes chunck size to 20MB. This causes excessive PUT transactions that can make small daily backups exceed free tier and raise the costs of S3 backups in general.


This combined with the other feature request for retry chunck instead of entire file would make S3 backups more cost efficient.

https://features.cpanel.net/topic/amazon-s3-cpanel-backup-transporter-code-to-retry-failed-uploads-in-chunks-rather-than-entire-file


Please include a WHM Backup advanced option to adjust upload_chunk_size.


  1. /usr/local/cpanel/Cpanel/Transport/Files/AmazonS3.pm
  2. # The size we break the file into for multipart upload
  3. # We hardcode this for now; but we may make this a config
  4. # option if there is demand for it.
  5. $OPTS->{'upload_chunk_size'} = 20 * 1024**2;

Leave a Comment
 
Attach a file