cPanel & WHM Version 98 has been released, and brings a slew of great updates. Take a look at what is included, and then upgrade today!

Optimization: Unused md5 generated for every pkgacct file during backup

Austin Morton shared this idea 6 years ago
Needs Feedback

In the file /usr/local/cpanel/bin/backup when a file is setup to be transported to remote destinations an md5 is generated. If no remote destinations are setup, this md5 is entirely unused from what I can tell. Even with remote destinations enabled I was unable to find a place that made use of the md5.

  1. sub transport_file {
  2. my ($args_ref) = @_;
  3. $logger->info("Queuing transport of file: $args_ref->{'local_path'}");
  4. my $queue_entry = {
  5. 'cmd' => 'copy',
  6. 'user' => $args_ref->{'user'},
  7. 'md5' => Cpanel::MD5::getmd5sum( $args_ref->{'local_path'} ),
  8. 'time' => time,
  9. 'local_path' => $args_ref->{'local_path'},
  10. 'remote_path' => $args_ref->{'remote_path'},
  11. 'keep_local' => $args_ref->{'keep_local'},
  12. 'session_id' => $args_ref->{'session_id'}
  13. };
  14. return queue_backup_transport_item( $queue_entry, $args_ref->{'queue'} );
  15. }

The biggest issue for me is that we are generating backups onto NFS storage, so generating this md5 causes the entire archive to be read back over the network again.

Replies (3)

photo
1

We have the same issue - backups are stored on an NFS volume, and the md5 sum wastes considerable time and bandwidth by reading back all of the archives.

photo
1

This is a big deal for us as well, as we are using SSHFS for backups.

photo
1

I spoke with cPanel support about this and they've provided, in my opinion, an adequate workaround: we can patch out the md5 checksum manually and have it automatically apply to the script using a 'postupcp' hook.

I've implemented this technique as follows:


  1. cat << EOF >> /scripts/postupcp

    run-parts /etc/postupcp.d

    EOF

  2. chmod +x /scripts/postupcp


  3. mkdir /etc/postupcp.d
  4. chmod 700 /etc/postupcp.d
  5. cat << EOF > /etc/postupcp.d/backup-no-md5

    #!/bin/sh

    sed -i'.bak' -e "s/Cpanel::MD5::getmd5sum( \\\$args_ref->{'local_path'} )/''/g" /usr/local/cpanel/bin/backup

    EOF

  6. chmod +x /etc/postupcp.d/backup-no-md5


This will setup a folder /etc/postupcp.d and any scripts in it will be run after running "upcp". A script is established in this folder to patch out the md5 calls from the backup script.

Alternatively, one can also patch the backup script manually and then just mark it as excluded from cpsync: https://documentation.cpanel.net/display/ALD/The+cpanelsync.exclude+File

Leave a Comment
 
Attach a file