Active-Active Redundancy or High-availability

Michael Kruger shared this idea 4 years ago
Open Discussion

I would like to see active-active redundancy or at the very least a high-availability active-inactive feature in WHM/cPanel. Starting out with N+1 would be sufficient.

Several virtual platforms already offer this, and there are several SAN solutions that do as well. Unfortunately those solutions are cost prohibitive for shared hosting. The custom solutions that most hosts are using rely on custom support labor and often use unproven methods.

With today's SAN environments or direct connection it is possible to achieve the necessary throughput on the network with a heartbeat and sync channel to accomplish this feature request. I would think that this can be accomplished from cPanel's programming side as well because cPanel has implemented complex features in the past.

I believe this is a very serious matter and I would be willing to pay higher costs for this feature. I would also purchase more licenses from cPanel if this existed.

I have contacted cPanel support about this several times in the past but the response has always been that cPanel is not interested. cPanel tells me that the hosting community is not interested in this. So, I would like for everyone to vote on this so that we can get the ball rolling. Please show cPanel how important this is to the hosting community! Please comment and vote for this feature!

Comments (22)

photo
2

sure we are interested in this feature, I've been contacting the support since we started using cPanel 6 years ago but i got the same answer, anyhow the only thing preventing me or my company from taking cPanel to the premium hosting level is the lack of HA solution, we are willing to invest in this feature and of course it will increase my number of cPanel licenses. (such solution will not require SAN only a decent network connectivity between the nodes)

for cPanel to implement this they should basically find a way to move all application data to a database engine (MySQL in this case) in order to have a seamless replication for the customers/accounts data between the nodes, as for the configuration files its very easy to write some scripts that will sync conf files between the N nodes.

photo
1

This is how I see that high availability should work :

Customers files on a high availibility NAS, and NAS mounted on two or more servers

Customers databases on MySQL Cluster

Customers DNS on DNSOnly Cluster

IP's on a hartbeat / load balanceing

cPanel installed and licenced on two or more nodes.

I'm not sure how exactly cPanel would cope with this configuration and I belive there would be some work to get everyting working flawlessly with cPanel.

The only part that cPanel should manage is making sure cPanel's are aware of this configuration, and syncing any modification on one node that's not in the users files or users databases or users DNS accross to the other server.

We are working towards such a configuration and would like it very much if cPanel couuld make it become a reality.

The total cost of all this is not prohibitave. A small host would not be interested in this type of solution but for a medium to large host is seems like and important step forwards.

This would also need a way to add a new node. The admin would install cPanel, add NAS's, then clone / sync the onther serveur configurations then adding the new IP to the load balancing scheme.

Medium/large webhosting companies would write their own script for adding NAS's and IP's to load balancing system. They would however need a simple way to sync cPanel configurations and also a central control panel accross all cPanel nodes.

photo
3

To be honest as a hosting provider I can't believe this functionality was not added before now. It seems like something most hosting companies would kill for. The ability to link two cpanel servers for redundancy should be in the next major release!

Please cPanel! There is really no GOOD alternative to cPanel (except LxAdmin) and that barely compares.

photo
1

There are so many tutorials available on web to mirror or take a

backup of your web files with different methods, here I am creating this

article for my future reference and here I’ll be using a very simple

and versatile command of Linux to create a backup of your website. This tutorial will help you to sync data between your two web servers with “Rsync“.

Sync Two Apache Web Server

The purpose of creating a mirror of your Web Server with Rsync

is if your main web server fails, your backup server can take over to

reduce downtime of your website. This way of creating a web server

backup is very good and effective for small and medium size web

businesses.

Advantages of Syncing Web Servers

The main advantages of creating a web server backup with rsync are as follows:

  1. Rsync syncs only those bytes and blocks of data that have changed.
  2. Rsync has the ability to check and delete those files and

    directories at backup server that have been deleted from the main web

    server.

  3. It takes care of permissions, ownerships and special attributes while copying data remotely.
  4. It also supports SSH protocol to transfer data in an encrypted manner so that you will be assured that all data is safe.
  5. Rsync uses compression and decompression method while transferring data which consumes less bandwidth.

How To Sync Two Apache Web Servers

Let’s proceed with setting up rsync to create a mirror of your web server. Here, I’ll be using two servers.

Main Server

  1. IP Address: 192.168.0.100
  2. Hostname: webserver.example.com

Backup Server

  1. IP Address: 192.168.0.101
  2. Hostname: backup.example.com

Step 1: Install Rsync Tool

Here in this case web server data of webserver.example.com will be mirrored on backup.example.com. And to do so first, we need to install Rsync on both the server with the help of following command.

[root@tecmint]# yum install rsync [On Red Hat based systems]

[root@tecmint]# apt-get install rsync [On Debian based systems]

Step 2: Create a User to run Rsync

We can setup rsync with root user, but for security reasons, you can create an unprivileged user on main webserver i.e webserver.example.com to run rsync.

[root@tecmint]# useradd tecmint

[root@tecmint]# passwd tecmint

Here I have created a user “tecmint” and assigned a password to user.

Step 3: Test Rsync Setup

It’s time to test your rsync setup on your backup server (i.e. backup.example.com) and to do so, please type following command.

[root@backup www]# rsync -avzhe ssh tecmint@webserver.example.com:/var/www/ /var/www

Sample Output

tecmint@webserver.example.com's password:

receiving incremental file list

sent 128 bytes received 32.67K bytes 5.96K bytes/sec

total size is 12.78M speedup is 389.70

You can see that your rsync is now working absolutely fine and syncing data. I have used “/var/www” to transfer; you can change the folder location according to your needs.

Step 4: Automate Sync with SSH Passwordless Login

Now, we are done with rsync setups and now its time to setup a cron for rsync. As we are going to use rsync with SSH

protocol, ssh will be asking for authentication and if we won’t provide

a password to cron it will not work. In order to work cron smoothly, we

need to setup passwordless ssh logins for rsync.

Here in this example, I am doing it as root to preserve file ownerships as well, you can do it for alternative users too.

First, we’ll generate a public and private key with following commands on backups server (i.e. backup.example.com).

[root@backup]# ssh-keygen -t rsa -b 2048

When you enter this command, please don’t provide passphrase and click enter for Empty passphrase so that rsync cron will not need any password for syncing data.

Sample Output

Generating public/private rsa key pair.

Enter file in which to save the key (/root/.ssh/id_rsa):

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /root/.ssh/id_rsa.

Your public key has been saved in /root/.ssh/id_rsa.pub.

The key fingerprint is:

9a:33:a9:5d:f4:e1:41:26:57:d0:9a:68:5b:37:9c:23 root@backup.exmple.com

The key's randomart image is:

+--[ RSA 2048]----+

| .o. |

| .. |

| ..++ . |

| o=E * |

| .Sooo o |

| =.o o |

| * . o |

| o + |

| . . |

+-----------------+

Now, our Public and Private key has

been generated and we will have to share it with main server so that

main web server will recognize this backup machine and will allow it to

login without asking any password while syncing data.

[root@backup html]# ssh-copy-id -i /root/.ssh/id_rsa.pub root@webserver.example.com

Now try logging into the machine, with “ssh ‘root@webserver.example.com‘”, and check in .ssh/authorized_keys.

[root@backup html]# root@webserver.example.com

Now, we are done with sharing keys. To know more in-depth about SSH password less login, you can read our article on it.

  1. SSH Passwordless Login in in 5 Easy Steps

Step 5: Schedule Cron To Automate Sync

Let’s setup a cron for this. To setup a cron, please open crontab file with the following command.

[root@backup ~]# crontab –e

It will open up /etc/crontab file to edit with your default editor.

Here In this example, I am writing a cron to run it every 5 minutes to

sync the data.

*/5 * * * * rsync -avzhe ssh root@webserver.example.com:/var/www/ /var/www/

The above cron and rsync command simply syncing “/var/www/” from the main web server to a backup server in every 5 minutes. You can change the time and folder location configuration according to your needs. To be more creative and customize with Rsync and Cron command, you can check out our more detailed articles at:

photo
1

I don't see this feature coming in the current form. Too complex in the current way services are deployed. The Cluster/HA options in other control panels are coming with a lot of strings attached. Every kind of services (Mail, Cpanel, Database, httpd..) is different in terms of HA and scaleability.

Perhaps with CentOS 7.0 and the docker hype a decoupling can be done.

Currently to stick to a Virtual Server (on KVM HA or VMWARE HA) will be an easier option. When it comes to costs for maintaining such a cluster/HA setup, it looks a bit cheaper.

photo
1

Their is already a feature for multiple apache servers currently tagged as planned.

I presume that feature would allow websites to keep up and running.

Maybe we could then hope of an email only custer that would help with email high availability too.

Things like FTP/SSH could come later.

photo
1

there are ways to do it, lot of big players already do it on their own,

but I dont think cPanel will do it until 2020...

by then, CentOS will already natively support it...

photo
1

Cpanel, please... listen to customers.. This isn't just another nice to have feature. It's a critical, vital part that should have been included years ago.

photo
1

I would second that this is a vital feature. The larger hosts have the resources to do something custom, we are a small provider and this is why we rely on cPanel. At a most basic level, just making it easy to administer multiple Apache backend servers via cPanel, and build in support for real-time sync of files is a must, in the same way as we can currently sync DNS. Everything else at this stage is a "nice to have".

photo
1

A lot of the problems with failover solutions have to do with limitations of the DNS system. In retrospect, it should have been designed with the capability of defining primary and failover IPs for hostnames.

There are some third-party DNS providers who sort of do this, however; so if cPanel came up with a solution for the cloning aspect, it could be implemented using a third-party DNS provider that supports failover.

Richard

photo
1

With cpanel DNS cluster and 5s TTLs it could be possible to introduce high availability based on an active - standby scenario where cpanel accounts are provisioned to two servers. Automated SQL replication and rsync data replication at admin specified intervals would be the most basic configuration options with options to automatically and dynamically hold sync'ing off during heavy server load.

This would be a remarkable step forward for cpanel. The plan I outline here is merely plucked from my imagination at this moment, with the talent at your company you could devise architectures that allow us hosting business owners to stay at the cutting edge and remain competitive in a marketplace where high availability is no beginning to be not only a sought after feature, but one that is expected even in basic paid hosting configurations.

photo
1

It would be really appreciate if WHM have this kind of features, in this is scenario we only need to create to DNS enteries and backup server one should be lower priority. when the Main Web Server went down without fail backup server start respond to queries.

Thanks

Abbas

photo
1

Having multiple servers sharing SSL certificates and synchronizing IMAP accounts via dovecot should be feasible, I have plans to implement this between a cPanel and a non-cPanel server, but it would be nice to have upstream support.

photo
1

Indeed, this sounds like a must have for today's hosting requirements. It's actually insane it's not supported yet!

photo
2

i assume it can be also insane amount of work to make this working, but would be nice if they would start

photo
1

Yes i think you can start to improve several things on the way to high-availability and get additional features step by step on the way.

I would start for example in supporting for Softwaredefined-Storage, Objektstorage (for example CEPH) or simply distributed storage like nfs and a positility to sync configurations and accounts for several cPanel accounts on the same distributed Storage. After that there can be added a posibility for dynamical switch accouts from one-server to another, added some config options for mysql/mariadb for replication and after that get a solution for checking availabilty of nodes and automatic switch on failover.

So you get different benefits on the way to HA. (new Storage, MySQL-Replication, better/instant account switching etc.)

photo
photo
2

Indeed, this is one of the most expensive workaround for each hosting company.

Several of my clients are asking similar things every after day often and thus the only option to overcome such issues is to either leave cpanel or alter the environment leading the updates un-installed for years.

I am with you in this regard.

photo
2

No doubt, that this feature is more than a need, is a necessity, and workarounds are rare unless complex, costly or unstable

photo
3

I have customers hounding me about redundancy and I can't promise them anything. I don't have the money, time or resources to setup a complete backup server separate from cPanel/WHM. If they implemented this, it would save a lot of peoples bacon.

photo
3

I would love to see this implemented. Any news from cPanel on this ?

photo
3

Unfortunately there hasn't been any movement yet, but as soon as we see something I'll let everyone know!

photo
3

@Benny do you think there is likely to be any movement on this? I would have thought there might be something in development by now? This is currently important for me as I plan next steps with my business. I have been a cpanel customer for over 10 years and I am sad to think I will have to move away from cpanel for my servers, but seems cpanel just isn't prioritising this so I have to look at other solutions.

photo