cPanel & WHM Version 114 has been released, and brings a slew of great updates. Take a look at what is included, and then upgrade today!

Active-Active Redundancy or High-availability

Michael Kruger shared this idea 11 years ago
In Progress

I would like to see active-active redundancy or at the very least a high-availability active-inactive feature in WHM/cPanel. Starting out with N+1 would be sufficient.


Several virtual platforms already offer this, and there are several SAN solutions that do as well. Unfortunately those solutions are cost prohibitive for shared hosting. The custom solutions that most hosts are using rely on custom support labor and often use unproven methods.


With today's SAN environments or direct connection it is possible to achieve the necessary throughput on the network with a heartbeat and sync channel to accomplish this feature request. I would think that this can be accomplished from cPanel's programming side as well because cPanel has implemented complex features in the past.


I believe this is a very serious matter and I would be willing to pay higher costs for this feature. I would also purchase more licenses from cPanel if this existed.


I have contacted cPanel support about this several times in the past but the response has always been that cPanel is not interested. cPanel tells me that the hosting community is not interested in this. So, I would like for everyone to vote on this so that we can get the ball rolling. Please show cPanel how important this is to the hosting community! Please comment and vote for this feature!

Replies (42)

photo
3

sure we are interested in this feature, I've been contacting the support since we started using cPanel 6 years ago but i got the same answer, anyhow the only thing preventing me or my company from taking cPanel to the premium hosting level is the lack of HA solution, we are willing to invest in this feature and of course it will increase my number of cPanel licenses. (such solution will not require SAN only a decent network connectivity between the nodes)


for cPanel to implement this they should basically find a way to move all application data to a database engine (MySQL in this case) in order to have a seamless replication for the customers/accounts data between the nodes, as for the configuration files its very easy to write some scripts that will sync conf files between the N nodes.

photo
3

At least 1 method of load balancing/high availability will be very nice, please consider to add it..

photo
2

Hi If the project leadares like zen load balancer we can help in the integration with the cpanel.


Regards

Zen load balance developer member

photo
1

This is how I see that high availability should work :


Customers files on a high availibility NAS, and NAS mounted on two or more servers

Customers databases on MySQL Cluster

Customers DNS on DNSOnly Cluster

IP's on a hartbeat / load balanceing


cPanel installed and licenced on two or more nodes.


I'm not sure how exactly cPanel would cope with this configuration and I belive there would be some work to get everyting working flawlessly with cPanel.


The only part that cPanel should manage is making sure cPanel's are aware of this configuration, and syncing any modification on one node that's not in the users files or users databases or users DNS accross to the other server.


We are working towards such a configuration and would like it very much if cPanel couuld make it become a reality.


The total cost of all this is not prohibitave. A small host would not be interested in this type of solution but for a medium to large host is seems like and important step forwards.


This would also need a way to add a new node. The admin would install cPanel, add NAS's, then clone / sync the onther serveur configurations then adding the new IP to the load balancing scheme.


Medium/large webhosting companies would write their own script for adding NAS's and IP's to load balancing system. They would however need a simple way to sync cPanel configurations and also a central control panel accross all cPanel nodes.

photo
3

To be honest as a hosting provider I can't believe this functionality was not added before now. It seems like something most hosting companies would kill for. The ability to link two cpanel servers for redundancy should be in the next major release!


Please cPanel! There is really no GOOD alternative to cPanel (except LxAdmin) and that barely compares.

photo
2

There are so many tutorials available on web to mirror or take a

backup of your web files with different methods, here I am creating this

article for my future reference and here I’ll be using a very simple

and versatile command of Linux to create a backup of your website. This tutorial will help you to sync data between your two web servers with “Rsync“.


Sync Two Apache Web Server

The purpose of creating a mirror of your Web Server with Rsync

is if your main web server fails, your backup server can take over to

reduce downtime of your website. This way of creating a web server

backup is very good and effective for small and medium size web

businesses.

Advantages of Syncing Web Servers

The main advantages of creating a web server backup with rsync are as follows:


  1. Rsync syncs only those bytes and blocks of data that have changed.
  2. Rsync has the ability to check and delete those files and
    directories at backup server that have been deleted from the main web
    server.
  3. It takes care of permissions, ownerships and special attributes while copying data remotely.
  4. It also supports SSH protocol to transfer data in an encrypted manner so that you will be assured that all data is safe.
  5. Rsync uses compression and decompression method while transferring data which consumes less bandwidth.

How To Sync Two Apache Web Servers

Let’s proceed with setting up rsync to create a mirror of your web server. Here, I’ll be using two servers.

Main Server


  1. IP Address: 192.168.0.100
  2. Hostname: webserver.example.com

Backup Server


  1. IP Address: 192.168.0.101
  2. Hostname: backup.example.com


Step 1: Install Rsync Tool

Here in this case web server data of webserver.example.com will be mirrored on backup.example.com. And to do so first, we need to install Rsync on both the server with the help of following command.

[root@tecmint]# yum install rsync [On Red Hat based systems]

[root@tecmint]# apt-get install rsync [On Debian based systems]


Step 2: Create a User to run Rsync

We can setup rsync with root user, but for security reasons, you can create an unprivileged user on main webserver i.e webserver.example.com to run rsync.

[root@tecmint]# useradd tecmint

[root@tecmint]# passwd tecmint

Here I have created a user “tecmint” and assigned a password to user.


Step 3: Test Rsync Setup

It’s time to test your rsync setup on your backup server (i.e. backup.example.com) and to do so, please type following command.

[root@backup www]# rsync -avzhe ssh tecmint@webserver.example.com:/var/www/ /var/www

Sample Output

tecmint@webserver.example.com's password:

receiving incremental file list

sent 128 bytes received 32.67K bytes 5.96K bytes/sec

total size is 12.78M speedup is 389.70

You can see that your rsync is now working absolutely fine and syncing data. I have used “/var/www” to transfer; you can change the folder location according to your needs.


Step 4: Automate Sync with SSH Passwordless Login

Now, we are done with rsync setups and now its time to setup a cron for rsync. As we are going to use rsync with SSH

protocol, ssh will be asking for authentication and if we won’t provide

a password to cron it will not work. In order to work cron smoothly, we

need to setup passwordless ssh logins for rsync.

Here in this example, I am doing it as root to preserve file ownerships as well, you can do it for alternative users too.

First, we’ll generate a public and private key with following commands on backups server (i.e. backup.example.com).

[root@backup]# ssh-keygen -t rsa -b 2048

When you enter this command, please don’t provide passphrase and click enter for Empty passphrase so that rsync cron will not need any password for syncing data.

Sample Output

Generating public/private rsa key pair.

Enter file in which to save the key (/root/.ssh/id_rsa):

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /root/.ssh/id_rsa.

Your public key has been saved in /root/.ssh/id_rsa.pub.

The key fingerprint is:

9a:33:a9:5d:f4:e1:41:26:57:d0:9a:68:5b:37:9c:23 root@backup.exmple.com

The key's randomart image is:

+--[ RSA 2048]----+

| .o. |

| .. |

| ..++ . |

| o=E * |

| .Sooo o |

| =.o o |

| * . o |

| o + |

| . . |

+-----------------+

Now, our Public and Private key has

been generated and we will have to share it with main server so that

main web server will recognize this backup machine and will allow it to

login without asking any password while syncing data.

[root@backup html]# ssh-copy-id -i /root/.ssh/id_rsa.pub root@webserver.example.com

Now try logging into the machine, with “ssh ‘root@webserver.example.com‘”, and check in .ssh/authorized_keys.

[root@backup html]# root@webserver.example.com

Now, we are done with sharing keys. To know more in-depth about SSH password less login, you can read our article on it.


  1. SSH Passwordless Login in in 5 Easy Steps


Step 5: Schedule Cron To Automate Sync

Let’s setup a cron for this. To setup a cron, please open crontab file with the following command.

[root@backup ~]# crontab –e

It will open up /etc/crontab file to edit with your default editor.

Here In this example, I am writing a cron to run it every 5 minutes to

sync the data.

*/5 * * * * rsync -avzhe ssh root@webserver.example.com:/var/www/ /var/www/

The above cron and rsync command simply syncing “/var/www/” from the main web server to a backup server in every 5 minutes. You can change the time and folder location configuration according to your needs. To be more creative and customize with Rsync and Cron command, you can check out our more detailed articles at:

photo
1

We like the idea of having a high availabiliy NAS shared between two servers, however this would mean that cPanel would have to accept to have the same account on two servers both using the same DNS cluster and allowing for all functions to be compatible with this configuration.

photo
1

I would love to be able to have multiple cpanel VMs behind a load balancing device such as Netscaler to build out a farm.

photo
1

I don't see this feature coming in the current form. Too complex in the current way services are deployed. The Cluster/HA options in other control panels are coming with a lot of strings attached. Every kind of services (Mail, Cpanel, Database, httpd..) is different in terms of HA and scaleability.


Perhaps with CentOS 7.0 and the docker hype a decoupling can be done.


Currently to stick to a Virtual Server (on KVM HA or VMWARE HA) will be an easier option. When it comes to costs for maintaining such a cluster/HA setup, it looks a bit cheaper.

photo
3

I Believe this feature would be a gamechanger for cPanel as we know it.

Some things to consider though.

If this feature comes to cPanel, i would love to see an option where cPanel would create a full-scale "webserver-cluster" of the available webservers, where storage and mysql would be remotely accessed.

We're in deep need of this before we can go on and use cPanel for our environment!

photo
2

I've been thinking about this a bit more.


This feature sound's very complicated and I think there are two parts or steps to make this become true.


PART 1


The first is to give sysadmins the ability to set up thier own high availability system :


cPanel already supports DNS clustering and an external mysql server. An e-mail only custer would also be handy, but there is already a feature for this.


We could mount a high availability NAS or SAN storage for homedirs and set up IP loadbalancers. IP loadbalancers can be set to check availibility and to perform sticky round'n robin (for sessions) etc.


The first thing we would need is for cPanel to allow to create an account with a domain that already exists in the DNS cluster without changing the account's IP and without changing the existing files in the homedir as the users files will already exist on the NAS or SAN storage.


We would also need cPanel to make sure that both servers play well together while sharing the same homedir over our NAS/SAN mounted drive space.


PART 2


The second part, which would come after the first part, would be to provide the technology give high availibility out of the box.


There are some systems that already do this, like for instance HybridCluster. To achieve this, they set DNS ttl to the lowest value possible, and automaticaly change an account's IP to point it to the other server when one server goes down. They use ZFS to keep files in sync, but a NAS or SAN based system would be just as good for our usage. Some form of heartbeat would need to be used to automaticaly change the DNS settings for an account to point to the new IP.


Another path instead of NAS or SAN could be to install a proxy on all servers, and sync homedir files with rsync in deamon mode for instance. When a site is moved automaticaly to the other server the proxy DNS would be changed to point to the second server that would proxy back to the first server until the files were in sync then the proxy would be inversed to that people who have a dns cache on their router are proxied to the second server. If a server stops responding, all sites are moved to a working server automaticaly and admins could easily move a site from one server to the other without any downtime.


All of this would need Part 1 to be working so in our view, this feature request could be split into two features, one making cPanel compatible with this type of setup while requiring sysadmins use external storage for homedirs and their own ip balancers, and the second making cPanel able to manage high availabily all on it's own, allowing WHM root users to simply add a new server to a cluster and let cPanel manage the rest.

photo
1

Their is already a feature for multiple apache servers currently tagged as planned.


I presume that feature would allow websites to keep up and running.


Maybe we could then hope of an email only custer that would help with email high availability too.


Things like FTP/SSH could come later.

photo
1

there are ways to do it, lot of big players already do it on their own,

but I dont think cPanel will do it until 2020...


by then, CentOS will already natively support it...

photo
1

Cpanel, please... listen to customers.. This isn't just another nice to have feature. It's a critical, vital part that should have been included years ago.

photo
1

I would second that this is a vital feature. The larger hosts have the resources to do something custom, we are a small provider and this is why we rely on cPanel. At a most basic level, just making it easy to administer multiple Apache backend servers via cPanel, and build in support for real-time sync of files is a must, in the same way as we can currently sync DNS. Everything else at this stage is a "nice to have".

photo
1

A lot of the problems with failover solutions have to do with limitations of the DNS system. In retrospect, it should have been designed with the capability of defining primary and failover IPs for hostnames.


There are some third-party DNS providers who sort of do this, however; so if cPanel came up with a solution for the cloning aspect, it could be implemented using a third-party DNS provider that supports failover.


Richard

photo
1

With cpanel DNS cluster and 5s TTLs it could be possible to introduce high availability based on an active - standby scenario where cpanel accounts are provisioned to two servers. Automated SQL replication and rsync data replication at admin specified intervals would be the most basic configuration options with options to automatically and dynamically hold sync'ing off during heavy server load.


This would be a remarkable step forward for cpanel. The plan I outline here is merely plucked from my imagination at this moment, with the talent at your company you could devise architectures that allow us hosting business owners to stay at the cutting edge and remain competitive in a marketplace where high availability is no beginning to be not only a sought after feature, but one that is expected even in basic paid hosting configurations.

photo
2

yes... please for gods sake add that... we host sites that have wordpress and they are sites that are 30k to 45k posts... we have a heck of a time with this. Constantly changing infrastructure...

photo
1

If a single site gives you a headache that way, I think it's time for that customer to use something different then a shared hosting facility as in a cPanel server.

For me, customers that are that big I force to a VPS system on AWS, at least for their website. If the rest is within certain limits that part can still stay on the cPanel servers.


Also if changing infrastructure is a big issue, build on AWS. You can scale your machines up with just a few clicks. Although be prepared to read up on the somewhat complex cost structure and ways to keep track on the overal costs.

photo
1

I find it difficult to believe that you would have any issues with that. Naturally it all depends on how well your server is configured and how you've optimised it. And of course what the hardware specifications are. Many variables come in play.

I host about 90 WordPress sites, big ones, on one of my cPanel servers with no issues. They are high traffic ones, too.

photo
1

If you need help, post your case and i try to help you. But cPanel it's to help to sell or resell, not for hosting big websites. Do you know an medium-big enterprise, not in hosting bussiness, that use cPanel?

photo
1

we hosts 1000s of them... that are 30k to 45k posts typically in size. Around 250MB to 650MB databases.

photo
1

it should be nice if cpanel support apache cluster with load balancing , and exim cluster like it do with dns cluster .


it is important to give customer high availability also in shared hosting with many account

photo
photo
2

the load balancing feature would be nice from an upgrade perspective. say you want to upgrade wordpress (as an example) on one server and see if that causes any issues with clients. if it does you can easily move them back to the pre-upgraded system.


also - if one server is under a DOS attack - you can move sites to the other server. (yes?)

photo
1

This will not help you in case of a DOS attack, only in case the attack is focused on a specific IP and the attacker is dumb enough not to check if the domain is moved. If you change DNS records to a new server the new server will die. If you do load balancing, they will both die. DOS is whole different ballgame. And should also not be fixed on an application/service level.

photo
photo
1

This could be a useful feature for a number of reasons.


1. Eventually, a successful site will start exhausting server resources, even on a high end server.

2. Sometimes, a site might be featured in news or someplace, and there will be a spike in traffic.


Here is one possible idea to set this up:


1. create the concept of a 'cluster' in cPanel

2. allow users to add/remove servers to/from cluster


3. automatically configure MySQL replication among servers in the cluster

4. automatically configure GFS file system (or something else) for user home folders

5. automatically add multiple A/AAAA records to dns zones (this will load-balance requests between servers); maybe allow load balancer IP to be configured?

6. automatically configure dns zones on all servers

7. automatically configure email servers

8. etc..


The point of this setup is:


1. load balancing - if a site has a huge traffic spike, the system will be able to handle the load

2. high availability - if a server goes down, various daemons' configuration allows everything to keep working as if nothing has happened

photo
1

It would be really appreciate if WHM have this kind of features, in this is scenario we only need to create to DNS enteries and backup server one should be lower priority. when the Main Web Server went down without fail backup server start respond to queries.


Thanks

Abbas

photo
2

This feature is really urgent for cpanel server administrator, and i think it should have high priority.


What can be done manually is to rsync files thru cronjob. But what about databases and ftp accounts and the protect dir. this really needs cpanel developers to be able to do it


Regards

photo
1

We will soon be creating a CEPH custer in order to test KuberDock and it would be nice if we could also use it for user's home dirs.


I'm still not sure that it should be up to cPanel to deal with the file sync. Well at least for the first versions they could maybe give guides and focus on making sure users details outside their homedir and apache configs are in sync.

photo
1

Having multiple servers sharing SSL certificates and synchronizing IMAP accounts via dovecot should be feasible, I have plans to implement this between a cPanel and a non-cPanel server, but it would be nice to have upstream support.

photo
1

Could using Amazon EFS (https://aws.amazon.com/efs/details/) with the upcomming remote MySQL service (Without root access) be able to do the job?


There would be some tweaking at the level of applying change and restarting apache that would need to be configured, plus some alternatives to quota's.


but it may be possible

photo
1

Indeed, this sounds like a must have for today's hosting requirements. It's actually insane it's not supported yet!

photo
2

i assume it can be also insane amount of work to make this working, but would be nice if they would start

photo
2

Yes i think you can start to improve several things on the way to high-availability and get additional features step by step on the way.


I would start for example in supporting for Softwaredefined-Storage, Objektstorage (for example CEPH) or simply distributed storage like nfs and a positility to sync configurations and accounts for several cPanel accounts on the same distributed Storage. After that there can be added a posibility for dynamical switch accouts from one-server to another, added some config options for mysql/mariadb for replication and after that get a solution for checking availabilty of nodes and automatic switch on failover.


So you get different benefits on the way to HA. (new Storage, MySQL-Replication, better/instant account switching etc.)

photo
photo
2

Indeed, this is one of the most expensive workaround for each hosting company.


Several of my clients are asking similar things every after day often and thus the only option to overcome such issues is to either leave cpanel or alter the environment leading the updates un-installed for years.


I am with you in this regard.

photo
2

No doubt, that this feature is more than a need, is a necessity, and workarounds are rare unless complex, costly or unstable

photo
1

Time has passed, and already 3 years we use the only reliable cloud clustered solution home made compatible with cPanel.

At YOORshop, we made it not only is flexible but performant.

photo
photo
2

We use multiple dedicated cPanel servers around the globe. As a server administrator this would be the best option I can hope for in cPanel. With this option available within cPanel we won't need to make complicated settings and look to other software to provide HA, LoadBalancing and Replication to our customers.


@benny could you please update us all about the plan to include this feature in near future soon?


--

Sam

photo
2

Hey all! This isn't yet defined on the roadmap, but we're definitely talking about the best ways to implement this. If we had our druthers we would already be working on it! The good news is that we're slowing removing the giant impediments that are core to the product that have been preventing us from working on this already. As soon as there's an update, I'll make sure to share!

photo
3

I have customers hounding me about redundancy and I can't promise them anything. I don't have the money, time or resources to setup a complete backup server separate from cPanel/WHM. If they implemented this, it would save a lot of peoples bacon.

photo
1

It would be nice not to have to pay a third party for failover, please implement this.

photo
1

Which third party allows this ? Do you mean using VMWare ? Or is there another soluton out there ?

photo
1

Hey guys, let's take the third party discussion to the forums to to a private email conversation. If you'd like me to pass each other your email addresses, just let me know.

photo
photo
3

Clients more and more are starting to require this, especially our larger e-commerce sites.

Would be great if we could do this through cPanel.

photo
2

Any possibility to consider this request? This feature is a must to bring cPanel to 2020. This is more and more frequently required by VPS customers, hence we are losing sales because of this lack of such a modern feature! :-(

photo
1

Translate by Google :


Do like me I created my own synchronization between two cPanel servers, and in two different datacenters with different IPs each time and possibility to choose the country of localization of IPs. So my customers come to my house and if my customers want to have direct synchronization between two of their two servers I make them for 2,999 euros (1 day of work)


----------------------------------

Faites comme moi j'ai créé ma propre synchronisation entre deux serveur cPanel, et dans deux datacenters différents avec des IPs différentes à chaque fois et possibilité de choisir le pays de localisation des IPs. Comme ça mes clients viennent chez moi et si mes clients veulent avoir directement une synchronisation entre deux de leurs deux serveurs je leur fais pour 2 999 euros (1 journée de travail)

photo
1

"

Any possibility to consider this request? This feature is a must to bring cPanel to 2020. This is more and more frequently required by VPS customers, hence we are losing sales because of this lack of such a modern feature! :-(

"


I completely Second this.

photo
1

"

Any possibility to consider this request? This feature is a must to bring cPanel to 2020. This is more and more frequently required by VPS customers, hence we are losing sales because of this lack of such a modern feature! :-(

"


Third!

photo
1

UP, We need this

photo
1

With cPanel's statements garding replication through SAN's and other technologies, it show's more of a willingngess to develop for the big players instead of the entire community at large.

photo
photo
3

We have begun heavy research phases of High Availability for cPanel. We have a forecasted roadmap for HA and we will continue adding additional information to it as we flesh out the project further.

photo
6

We have begun heavy research phases of High Availability for cPanel. We have a forecasted roadmap for HA and we will continue adding additional information to it as we flesh out the project further.

photo
4

Good news folks!

We have updated the roadmap with more information about the direction we're heading. As always, feedback is welcome.

https://docs.cpanel.net/knowledge-base/roadmaps/cpanel-cloud-edition-roadmap/

Dustin Scherer (he/him) | Product Owner | @dustinscherer

photo
2

It should be good now. The link was updated and the post was not.

photo
2

The link is currently not working

photo
1

So the plan is to use MySql/MariaDB inside a container for each user? Why not handle the database replication separately for the entire server using galera cluster?


Isn't the NFS server a single point of failure? When I first saw this feature request years ago I was thinking in something like Ceph.

photo
1

That's great news - thanks for updating us. I'd like to know if there is a forum area where we can discuss this topic? I have some questions and would like to contribute ideas. Thanks

photo
1

I have been looking deeply into how to do this myself but would prefer very much to use an integrated cPanel/WHM solution. I look forward to being an early adopter of this tech.


Is there an upgrade path, or will I need to transfer all of my clients over to a new setup running this?


Cheers,

photo
1

An additional question I have about this setup.

1. How will DNS work? Can I make each Node have a DNS server so that if one node goes down DNS still works for all of my configured sites?

2. Will I be able to have multiple file servers, and will they be seperate to the nodes, or can they be a part of the nodes?


Thank you,

photo
1

Still off (404)

photo
1

The link is wrong: Is it this one ? https://docs.cpanel.net/knowledge-base/roadmaps/cpanel-cloud-edition-roadmap/


That's great news - thanks for updating us. I'd like to know if there is a forum area where we can discuss this topic? I have some questions and would like to contribute ideas. Thanks

photo
1

any updates?

photo
1

Link in cPanelDustin post leads to a 404 page.

photo
1

Links don't work. 404 errors

photo
1

What happened to this roadmap? The link 404s and no related roadmap exists here: https://docs.cpanel.net/knowledge-base/roadmaps/

photo
1

The link is broken again.

photo
1

Links broken!

photo
1

Is this project cancelled? Link isn't working anymore

photo
1

Ditto, I get a 404 page

photo
1

the link it's still 404


something i found was this: https://docs.cpanel.net/knowledge-base/roadmaps/cpanel-linked-nodes-roadmap/

photo
photo
3

Good news folks!

We have updated the roadmap with more information about the direction we're heading. As always, feedback is welcome.

https://docs.cpanel.net/knowledge-base/roadmaps/cpanel-cloud-edition-roadmap/

Dustin Scherer (he/him) | Product Owner | @dustinscherer

photo
1

Let's be honest, when High Availability comes into the mix, it's time companies/customers etc. move away from cPanel and start to invest seriously in their infrastructure. cPanel is not designed for HA, and the complexities to implement mean that it's highly unlikely this will happen in any short time frame.

Unfortunately for many people using cPanel and requiring HA, this is likely driven by an end customer who doesn't have a strong development team to support this. There is no middle ground unfortunately. cPanel is designed for quick and easy use for both end user and supplier. HA is a large leap, it requires the skills and expertise to support. It's not a simple click and config like cPanel is.

cPanel is great. I use regularly for certain projects. I also actively avoid for other projects. Scale is important. You don't build a saw mill with a hand saw moving extremely fast..... Right tools for the right job.


Regards,

Michael Cropper

photo
2

I find it suprising that this is still something not delivered after 10 years of asking. Poor show cPanel.

photo
1

26da9aaf4a15602bca35753662af8298

Sent from my iPhone

photo
1

not work,

404 Not Found



nginx/1.24.0

Leave a Comment
 
Attach a file