Incremental Backups Multiple Retention

Sovjohn shared this idea 3 years ago
Completed

In the updated 11.38+ backup system, which looks quite nice, there is a significant downgrade in terms of functionality compared to the "Legacy" one:

Incremental backups, for some reason, can only be restored through ONE copy (!), or so claims the documentation.

This is absolutely inconsistent with what happened up until 11.36.x, where if Backups were set to incremental, despite certain copies being rolled out to /monthly or /weekly, administrators always had the capability of restoring any backup at all.

I would like incremental backups to be treated equally with other backups (compressed / uncompressed) so that if a customer decides they want to go back in time 7 days, and not 1, we are able to accommodate their request.

Best Answer
photo

This is now in a public build of version 64, 64.0.1, which is in the CURRENT tier. Update to version 64 now to take a look!

https://documentation.cpanel.net/display/64Docs/64+Release+Notes#id-64ReleaseNotes-TheNewBackupSystem

Comments (26)

photo
3

I agree. The new backup system should retain multiple backups for incremental backups, just like on the legacy system, or even better, allow there to be multiple retention ponts like the (un)compressed option...

This is a completely backwards step for the new system!!

photo
2

I agree, I would like to see this plus a number of other functions that we lost in the new backup system, which were in the legacy backup system for over a decade, brought back.

photo
1

Agree with every one here. Definitively a backwards step from cPanel and an absolute necessity.

photo
1

It is true, for hosting rpovider is a disaster... if you want to restore a mothnly backup need to do from ssh.. and them i cannot understand why deleted weekly backup...

photo
3

Yes this limitation makes the new system's incremental option pretty useless for me, I'll have to stick to the legacy system (with the experimental hard links feature on) until this request is implemented. Otherwise, I find it uses too much CPU power and storage space, but it's also important that I still be able to maintain daily/weekly/monthly retention points, making this an ideal combination of options that I don't want to lose.

photo
2

I agree. This is such a backwards step. I can't use the new system without this feature.

We definitely need it !

photo
2

just realize this NOW, WOW, WHY this even happened at first place?

we are not able to use new backup system due to this too!

photo
2

Definitely add this! I have a site of 63GB. If a hacker attacks me 1-2 hours before the Incremental Backup Iam completelly gone!

So make sure to apply this in the next patch update!

photo
2

I also vote for incremental backups with hard links! Currently it creates a daily backup of 136GB of one of our servers which is kind of an overhead. Better to use cpanel backup for system files and rsnapshot for incremental account backups w/ hard links for now.

photo
1

I think it's a must to retain the incremental backups for several days so that if a customer decides they want to go back in time 7 days, we are able to accommodate their request.

photo
1

Why does it offer multiple backup options but then states you are limmited to only one! Something is inconsistent here.

photo
1

This is a must!

photo
2

It is very strange to offer only one backup point. There should be unlimited points of backup possible, so admin could set, just like he can set how often backup is done and how long to keep those backups. If R1Soft can do it - so can cPanel! :)

photo
1

I have a few backup points setup. Only one enabled. Script will not let me do incremental to a mount point. I think i have to delete as well as disable. this interface really needs attention - that looks like a bug.

photo
1

Hey all! I know we're working on incremental backups, but I don't know if this is in the scope of that project. I'm working to find out and will let you know!

photo
2

I just had a customer who wanted to restore a database from the day before yesterday. They made changes yesterday and the database was changed. Of course, this morning's backup contains the changed database, not the database before the changes. I wish I had seven days of independent backups.

photo
1

For now you can cron a script for your own multiple saved sql dump backups. I have a client who is using this:

AutoMySQLBackup

https://sourceforge.net/projects/automysqlbackup/

photo
1

@jimhermann - you can make your own daily incremental rotation script. Here's an example of something we're using:

  1. #!/bin/bash

    #Rotate Backup Script

    #11 days ago:

    DAY31=`date -d "11 days ago" +%Y%m%d`

    #Delete the backup from 11 days ago, if it exists:

    #find /backup2/cpbackup/ -maxdepth 1 -type d -name "*_backup" -mtime +10 -exec rm -rf {} \;

    #Todays date:

    DAY0=`date +%Y%m%d`

    #The source directory:

    SRC="/backup2/cpbackup/daily"

    #The target directory:

    TRG="/backup2/cpbackup/$DAY0"

    #Execute the backup

    cp -al $SRC $TRG

Set WHM to use the legacy incremental backup method without compression. Then make this script run with the /scripts/precpbackup or /scripts/postcpbackup whatever your preference is.

This will create hard links to files instead of creating duplicate copies. It saves a ton of space and you get the benefit of having X days worth of incremental backups.

Good luck!

photo
1

I don't recommend make copies using hardlinks because the cpbackup is strange with the dumps and some other files. It writes inside the file instead overwrite it and this make all hardlinks of that file to be updated and you loose any retention you have made.

To workaround this I use ZFS and make snapshots before (precpbackup) make the updates (and I enable the compression in volumes too so I can have compressed incremental backups /yH5BAEAAAAALAAAAAABAAEAAAIBRAA7). You can also do this with other snapshot prepared systems like BTRFS (but is too buggy, works fine only if you use a 3.16 kernel or greater) ou even with a LVM (but it's harder to do and more slow than other).

photo
2

Good morning requesters! Just got an update from the product owner for the team that will be working on remote incremental backups. The team is hoping to get started on remote incremental backups toward the end of the version 60 development cycle, and that research would include this request. There's a chance that remote incremental backups could be included in v62, but it's not yet sure if multiple retentions will be included in that release. I'll let you know as soon as there's any more information!

photo
3

I'm back! The team working on remote incremental backups has confirmed that, barring anything unforeseen, multiple retention points will land in version 64! Once we hit the EDGE tier with a version that would be usable I'll be back to point everyone to it. If you have questions in the meantime, please do let me know!

photo
5

This feature has been merged into version 64, which is currently available in the EDGE tier. There's a known defect in the UI (the text beneath the "Incremental" option indicates that may only have one backup with incremental backups) but the functionality of the feature is not impaired.

photo
1

Thank you for the update.

Is this new multiple-retention feature in v64 applicable to local incremental backups, remote incremental backups, or both?

photo
2

This is just local incremental backups right now. Remote incremental backups didn't make it in to version 64, but will be in 66. You can watch this feature request for updates on that:

https://features.cpanel.net/topic/as-a-server-administrator-i-want-support-for-remote-incremental-backups-so-that-i-can-make-less-resource-intensive-backups

photo
2

Hey Benny. It's very nice to see how all these features requested so many years ago are finally coming true. Since 2015 you guys are reaching an unparalleled level of improvement in the product. Congrats to all the team! Keep up this awesome work!

photo
2

This is now in a public build of version 64, 64.0.1, which is in the CURRENT tier. Update to version 64 now to take a look!

https://documentation.cpanel.net/display/64Docs/64+Release+Notes#id-64ReleaseNotes-TheNewBackupSystem

Comments have been locked on this page!