cPanel & WHM Version 92 has been released, and brings a slew of great updates. Take a look at what is included, and then upgrade today!

Store Email on a different partition.

Chris shared this idea 4 years ago
Open Discussion

IMAP mail has become the norm, with customers needing more and more space for messages. At the same time, SSD drives are now common, which provide a great performance boost for websites and databases but at a lower capacity and a premium cost.


In the current situation, messages are stored in the home partition at /home/username/mail which means that web hosts using cPanel have to store large volumes of mail on the same drives used for websites.


It would make sense to allow the /home partition to run from SSD's but have a separate /mail partition for Email storage that could utilize higher capacity, slower drives.


The proposal is simply to change the location of mail from /home/username/mail to /mail/username - or to specify the mail partition in the basic WHM settings. This would allow the web host to provide more space for Email and to locate it on separate, more appropriate storage. It would also improve performance.

Comments (20)

photo
1

This can be done with hooks and/or mail service configuration changes. For the most part, this is not needed unless the server is severely overloaded, in which case there are other options to consider.

photo
1

I appreciate your opinion and I can see your point of view.


My opinion on your response is that message storage usage and "severely overloaded" servers don't always go hand in hand. Someone wishing to store thousands of messages will not be causing any greater load on the server, or affecting website performance but they simply have a requirement for more space. That space would be better served from a storage array, or a different drive that is separate to the one used for the home directory. There are quite a few people in this feature request system exclaiming that they want to use SSD's to boost performance, but that type of storage isn't really appropriate for storing huge amounts of mail at this point in time, due to low capacity and high cost. I accept that SSD's will eventually increase in size and drop in price, but I maintain that this is still a good idea.


I also agree that it is possible to do this with some modification, but this feature request is specifically asking for that feature to be included as standard. There are lots of things that can be done with customized configurations and scripting, but I think there is an increasing demand to separate out storage for different purposes and this functionality would begin to meet those demands. If you are comfortable making your own modifications via hooks and configuration changes then that's great. I'd rather this functionality was built in.


In a cloud infrastructure, it would be an advantage to store mail on an entirely separate storage pool that is made up of high capacity 3.5" drives and run the home partition from faster SSD storage. On a single server configuration, using a secondary drive or array would serve the same purpose. With improvements in the software used in this industry It is now possible to get more accounts out of the hardware, but with IMAP now being so popular, it is storage space for messages that takes up the bulk of the server's storage in some cases.


I'm of the opinion that if you are running out of space on a server then you may not have balanced that server correctly and there is no point in overselling services so much that you end up with too many accounts and not enough space. The way you design your hardware and infrastructure very much dictates how it gets used. But you can't ignore the fact that, even if a server isn't overloaded, disk I/O from the processing and retrieval of Email can be significant - and while that might not adversely affect performance, if that I/O was removed from the main /home partition it would provide a performance boost for websites and allow for more accounts to be run from the server, which is nothing to sniff at. At the same time, you can't ignore the fact that in many cases, message storage requirements now outweigh all other capacity needs. It's about getting the right balance, running the right number of accounts on your servers and different types of clients often from different hardware - and I strongly believe that implementing a change like this would go a very long way to making that balance much easier to achieve.


Personally, I'd like to offer my clients super fast, low latency storage for their websites and databases but run their message storage from higher capacity, more affordable drives. I don't think I'm alone in that requirement but I'm also very open to any opinion anyone would like to share.

photo
1

"not needed unless the server is severely overloaded" => not true, as many said we would like to store websites on a fast SSD disks and email on standard hard drives.

photo
photo
2

I would also like to see the ability to store emails, especially the archives on a separate partition. This way, expensive SSD's can be used to service content (Web / SQL / IMAP / etc), while cheaper SATA HDD's can be used as storage.


Usting hooks todo this will probably break things like backups, moving accounts to another server, etc.

photo
2

Yes, that is correct. I've seen a number of hosting providers doing this and is impossible to migrate an entire account: the email accounts are not migrated and the restore process throw errors in regards of non-existent mail folders. Also, backups will not include email accounts contents.

photo
1

Has someone tried with mounting a different partition (on the non-ssd drive) for /home/user/mail (and not using the symlink solution ?

OK, this has to be done by manually and is sysadmins work, and must be done for each account, but in that case that there are not many accounts on the server, should backup and Transfer or Restore of a CPanel Account then not continue to work ?

Probably quota will not work anymore ?

https://serverfault.com/questions/67038/how-do-i-mount-sub-directory-to-a-hard-drive-in-linux

photo
photo
1

I think the larger benefit to something like this is actually preventing something like a wordpress exploit from allowing hackers to siphon a user's email. In addition to SoftDux's note re: storing mail on larger partitions.


Either way, I think this, coupled with some sort of native email clustering would be extremely pertinent.

photo
1

This is a great idea and just what we were looking for. With products like this in the market ( [link omitted] ) we would be able to utilse the SSD for web and HDD for email.

Moderator note: Removed link to external commercial site

photo
1

As soon as this is implemented i will be migrating to cPanel/WHM for all services.

In my experience, 90% of clients cPanel space is dedicated to emails.

This suggestion together with mailbox compression (this feature is in pre-release) would improve storage flexibility, scalability and performance when using HDD for emails and SSD for everything else.

photo
2

This is similar to what I posted here:

https://features.cpanel.net/topic/set-different-mail-storage-path-for-specific-plans


In my case I don't mind hosting email for all users on the regular storage for most users, but there should be an option to use another partition like you said for different plans or accounts (business email). This way, you could have a cPanel hosting plan with bigger email accounts, and normal users upgrade to those plans and now they are using the cheaper storage.


I absolutely 100% agree with you. You can't offer decent email accounts from cPanel today and I suspect cPanel is losing customers based on this. Alot of customers are just migrating to Gmail or Office 365 because they offer huge sizes for email account. Email storage is budget storage, not high performance SSD on RAID setups.


Or at least another solution for now, if this is way too complicated for cPanel to implement, is archiving older emails to another partition. That way, regular emails that need frequent access are still on the regular storage, but very old emails that usually are not accessed frequently go to a cheaper storage automatically.

photo
3

While I like this request a lot ..chances are it will never get implemented. However there is a silver lining ... this is coming in the 11.66 release due in a couple of weeks https://features.cpanel.net/topic/add-option-to-compress-mdbox-in-v58


Basically native Dovecot mail compression which should save up to 90% of the disk space occupied by email storage.

photo
1

This feature is very much needed as more and more users keeps hundreds of GB's worth of email on cpanel hosting. I have a setup where I have a 480GB SSD and 4TB HDD. I would very much like the websites to leverage the speed SSD provides and the storage the HDD provides for emails.

photo
2

I'd like to see some input from cpanel on this one.

photo
2

Wow! I'm so sorry to see that we hadn't contributed at all here. I swear I'd said something, but it certainly appears not!

This is definitely one of those things that we have in mind going over the next few years of strategic planning. While I can't provide any solid insight for you right now, I can tell you that the motivations behind this request (specifically wanting to separate the hardware costs of different services from each other) is being considered in all of our decisions right now.

photo
2

Essential feature, without it cPanel is not fully prepared to compete with its rivals (e.g. they offer option to store mails on different partition/mounted disk etc.).

photo
1

I'm stucked on this, because I need same solution. In my case 85% client storage is for mails, while as little as slow for web because of use of the same HDDs.

Please Benny. try to solve his soon!

Thx

photo
3

100% agree this is a very important must have. We can't compete with other email service providers who offer large email storage. IMAP emails from clients are eating up much of my precious SSD space, while the huge HDD is unutilized.

Can cPanel please fast-track this option pls?

photo
2

This really is a fundamental, requirement now and should be treated with high importance.

photo
1

another way would be mutiple home folders

/home1 is ssd

/home2 is hdd

/home1/user/mail alias to /home2/user/mail


OR

allowing best of both worlds centos Only makes /home/user/... however this is ONLY Alias files

/home/user/mail/ - aimed at /home2/user/mail -- HDD

/home/user/public_html - aimed at /home1/user/public_html -- SSD

this way we set in the user account profile the default file paths for the type of accounts and a default the server will make the home folders with permissions in all folders allowing speed and size allocations


issues like this is holding my company back a lot.

photo
2

The idea of doing a symlink seems like the best way to handle this as we have several clients that like having direct access to their mail folders. As a matter of fact it seems this could be done with a simple hook in the account create script to create a new folder on the HDD and then create the symlink on the SSD.

Might be a new topic, but this gives me the idea of pre-made hook scripts that cPanel could offer for situations like this and others. This way it makes it easy to implement and the scripts are officially supported.

photo
1

If you have an idea for a separate feature request, please do create a second one, even if it's related to this one. We appreciate the feedback.

photo
photo
1

If it really is as simple as a symlink, perhaps the best place for the configuration option would be in the Basic Server Setup page. This variable (being a directory) could then be used by the hook to create the symlink on account creation with the path of your choice. Otherwise, perhaps a Tweak Settings option with a fixed setting - such as "Store email at /home/mail/{user}" and the "mail partition" would need to be always mounted at "/home/mail".

I suppose this would have a knock on effect for backups and for account transfers - and of course, there would need to be a way to convert from the old structure to the new. It doesn't seem too complicated though.

photo
5

To continue and keep customers not moving to office etc this is a feature cPanel should prioritize.

Having a dedicated server for hosting customers with sites/databases on SSD and email on SATA would keep us as a hosting company able to provide quality hosting and still offer alot of space for a reasonable price!

photo
1

This would be useful. We would like to offload mail files from website files. DA already support that, cPanel should follow...

photo
1

I find myself in the same dilemma.

With everything being as optimized as possible, I wonder why storing emails on SSDs. Separating the storage of emails and public_html is necessary to optimize costs.


Looking forward to the progress of this feature