cPanel & WHM Version 82 has been released, and brings a slew of great updates. Take a look at what is included, and then upgrade today!
 

Enhance FPM support

cPanelJacob shared this idea 3 years ago
Completed

Now that EA4 has php-fpm packages I would like to see cPanel & WHM offer support for configuring per-user pools so that I can provide sites that need better performance with dedicated PHP resources.

Best Answer
photo

Hey everyone! Version 60 just went to CURRENT. We're still discussing many of the concerns that were brought up here, but haven't acted on them (as far as I know). If you want to see specific things added to this feature, definitely submit feature requests for them! If you have any questions feel free to email me.

Comments (21)

photo
1

With PHP-FPM users can access each other user's PHP cache and the only way to make it secure is to either disallow caches like apcu or to run a seperate PHP-FPM instance per user.


Some webhosts will want muliple pools, others will need multiple instances.

photo
1

I've been thinking about this feature a bit differently from the other SAPIs that we offer. The primary reason being that php-fpm isn't the same as the other SAPIs we offer.


The experience that I have had in mind is that you have a *global* SAPI (dso, cgi, suphp, etc) and then you have the ability to set certain sites or users as "fpm pools" in the WHM MultiPHP interface. This allows for per-user processing, but not on a global level, if you want it on a global level it shouldn't be difficult to iterate on this plan to allow for a "on account creation, create fpm pool" type option.


The idea of a shared multi-user pool terrifies me for various security reasons.

photo
1

with your php-fpm support implementation in EA 4, you are doomed to failure ... just installed latest version, tested and i see no way it will work properly...


some developers have prototypes for EA3, like this one https://github.com/magenx/cPanel-php-fpm,

you wasting time with unnecessary functions rather than moving forward with todays technology....

photo
1

> with your php-fpm support implementation in EA 4, you are doomed to failure ... just installed latest version, tested and i see no way it will work properly...


I can tell you that it does work properly, you just have to know how to configure it.


I've attached a document that we haven't released yet (still awaiting validation internally), however it covers in detail how to configure EA4 + php-fpm. There are probably some steps missing (like enabling the service).


The implementation we deliver with the next iteration will use these same methods (in an automated fashion, of course). We are actively working towards getting these sorts of features in to EA4 however we have to make some feature parity with EA3 first, and that takes time.

photo
1

no, you are missing a lot more, like creating php-fpm master service per user account, with its own master php-fpm config, and chrooting its pool to home directory... otherwise this account will be able to see/hack entier server....


php-fpm => php-fpm-{cpaneluser}

php-fpm.conf => php-fpm-{cpaneluser}.conf

www.conf => {cpaneluser}.conf


this way you can run completely isolated instance with personal opcode cache,

and stop/start/debug it without breaking other users...

photo
1

The plan is per-user pools, the config specified in that article is enough to get you started, it's certainly not the entire config that we will be delivering when we release this feature.


Now, you mention copying over "php-fpm" which file in specific are you referring to there? I'm not entirely sure why you're breaking out php-fpm.conf and www.conf in this configuration, at that point I'd just provide a single configuration file for the user and manage the pool without the init script.


chroot is certainly an option we will be exploring as we go into this implementation. We've also considered putting each FPM pool inside it's own container so that it has as much isolation as we can reasonably give from the rest of the system.

photo
1

php-fpm => php-fpm-{cpaneluser} is an init script in CentOS6, that will pick up those cloned config files.

in CentOS7 it will be systemd configuration.

but anyway, you can not start one master and fork any child process/pool from the same in shared environment. this will lead to some serious issues. you need to run all isolated.

thats why every cpanel user must have its own master php-fpm process/config and own pool in it.

master process takes few mb and no cpu, but from other side of course, memory consumption will depend on the number of active pools, every pool will use dedicated realpath and opcode caches.


googling around, and searching github, probably i see now the full picture.


thats why i was confused and found your config a bit jerky, after i tested lots of configurations, i was expecting to see in cPanel something really promising, but in fact you just loaded defaults.

photo
1

Yes, PHP-FPM pools don't isolate users, for instance with pools all users share the same APCu cache and can access each other's cache.


The only way to fix this security issue on shared hosting is to have a full php-fpm instance per user. Seperate pools don't help from a security perspective and are only there to provide different settings to different users.

photo
1

That is certainly a workable approach. So we're looking at single fpm service instances per user, with perhaps setting up pools on a per-domain basis (if doing so has any benefit).


At this point, memory management becomes a concern. Being able to do an ondemand setup on a per-user basis has big advantages when you're looking at scaling to a whole lot of users, but I see the security implications here. One work around that occurs to me is using socket activation (or a similar concepts for c6) where the pool isn't started until the first request and shut down after a certain number of minutes being idle. This approach has issues when you consider that:


1. the first request may be a bit slow

2. it still can have scaling issues if too many sites get hit at the same time


Thoughts?


My biggest concern with this feature is how it will scale. I can verifiably state that this will have HUGE benefits for users on single domain / few domain systems. I think we are going to have to develop a formula for determine whether we recommend using FPM to an administrator or not.

photo
1

Unless I'm mistaken, Linux has quite a good memory management system and I don't think that alot of idle processes would be an issue, I also don't think that memory would be an issue.


We currently use Litespeed in Process Group mode and it doesn't use that much ram on a modern server (we have 256GB per server and only use about 100GB excluding file cache.

photo
1

Under a competently administered system, you will be ok. What worries me is the people who buy 1gb VPSes and then put 1000 accounts on it.

photo
1

Litespeed allows to choose between deamon mode (one since process that is forked) and process group mode (one master process per user that is forked only for that user). The advantage with Litspeed is you can set the expire time, we have set ours to 30 minutes so if no further calls are made the owner process is stopped.


I think like litespeed this should be up to the administrator to decide which mode he want's. Either using one pool per user (recommended for low to medium memory) or one instance containing a single pool for each user (recommanded for high memory servers).


I don't think the individual php-fpm instance should be the default recommended option, but an option with a warning that says to only use if you know what you are doing and you have enough memory on the serveur.


A VPS with only 1 GB ram and 1000 accounts would probably crash anyway due to other limits like maxing out the CPU.

photo
1

@Matt Dees, I have looked into the possible implementation of php-fpm in EasyApache 4.


We are currently using an old mod_fastcgi instead of mod_proxy_fcgi which should also not break the MultiPHP selector provided with EA4. Is there an way to manually register additional PHP-Versions which can be

selected by the User in cPanel.


Or will your automated version of php-fpm be implemented with cPanel 54 when EA4 will be default?


Since PHP-FPM is about 2-4x faster on our servers than the other available PHP implementations it would be very helpfull if we can get it to work with the new MultiPHP selector.

photo
1

How is this progressing?

photo
2

Currently we are planning to do this in the 58 cycle (note: delays do occur, planning is the operative word there). Development should begin in the next week.

photo
1

Good to hear Matt. This is a big one, as everyone already knows....


Thanks for the quick reply.

photo
1

Hello Matt,


Any update?

photo
3

Hey all! We've got this framework built, and a UI that allows you to enable/disable per-domain. We're automatically creating pools for each user as enabled as well. We are currently on schedule for this to land in v58.

photo
2

FYI - I totally understand why it is unsecure to allow malicious users to access pools / cache for other users and in a typical shared hosting environment that makes sense. The downside is that you can't share the cache for things like drupal/wordpress/etc.


Our case is a bit different. We are running mod-ruid2 to isolate permissions on the file system by having apache run as the user. This avoids inadvertent interference between user apps and potentially prevents user exploits from affecting other users.


If possible, it would be preferable to have the performance benefit of a shared system mem/cache pool but have file-system level / setuid per directory.


Not sure if that is something you have considered or if there is a good work-around.

photo
1

i gess good old chroot will save you from many problems:


  1. chroot = /home/user
  2. chdir = /public_html
  3. php_admin_value[doc_root] = /public_html
  4. php_admin_value[open_basedir] = /public_html
  5. php_admin_value[upload_tmp_dir] = /public_html/tmp

photo
1

thanks @backblaze but would user and group run as nobody for the process requiring all user file permissions as nobody?

photo
1

Yes I agree, cPanel musn't forget that pools don't isolate users and only allow to have different settings for different users. One php-fpm instance per user is require for things like APCu cache to be only accessible by that user. Otherwise all users can read, modify and empty other user's cache.

photo
1

ready to test it cause need some review for Magento 2 hosting on cPanel :)

photo
2

Got a quick update for you all! This UI is in EDGE, and I like it quite a bit. I've attached a screenshot. If you have an EDGE server you should take a look. Random side note: If you would like to run an EDGE-tiered server, but don't have a spare license, you can apply for a developer license.

photo
1

Hopefully there's a GUI for default and per user / domain settings for each pool somewhere. That GUI is just an on/off setting which of course is still necessary, but is only the smallest tip of the iceberg.

photo
2

Hey everyone! This feature has been delayed to v60 for usability and security concerns. It was removed from EDGE today and will make a return in a future EDGE build.


This removal includes both the UI that was included in WHM and the relevant API calls. In the event that you have configured your server to use PHP-FPM on EDGE, your current implementation will not be altered, but you will not be able to manage PHP-FPM though the WHM UI.


As always, let me know if you have questions!

photo
1

When is the planned release for v60?

photo
2

The first EDGE build of cPanel & WHM version 60 (which will be 59.9999.??) will hit the EDGE tier in the next week, we think. From there it's typically another 6-8 weeks before a new version starts its move through the release tiers.

photo
1

Okey. Will the PHP-FPM UI be available in that EDGE build?

photo
1

It is already available in Edge inside of the MultiPHP manager when using EA4.

photo
1

This feature would create php-fpm configs for new accounts created? So that you don't need to make a configuration for each new account created.

photo
1

No, this feature requires being enabled on a per-domain basis. In the future we may iterate towards being able to enable FPM pools on every new account/domain however there are memory concerns with this approach for FPM.

photo
1

Okey, what PHP handler will other account's use? FCGI?


How much memory would you say it would take per account if you use PHP-FPM?

photo
1

Whatever the SAPI is set to for the server. I believe that fcgid will be available.


The amount of memory depends on a LOT of factors, if they are idle accounts, not very much at all. If they are semi-active, it can be quite a memory hit.

photo
2

It's not yet certain if it will or won't make it into early EDGE versions, but I do want to mention that we don't recommend you run EDGE on production servers at all. The feature is intended to replace the need to manually configure FPM pools manually, yes, and to allow you to manage many other aspects of the service as well. As soon as it hits EDGE I'll update here, for sure.

photo
2

I noticed that this feature was not mentioned in the august development update on the blog, was there any particular reason for that? Is this feature available in the current EDGE-tier?

photo
3

Nope, no particular reason. It's definitely in EDGE, I just completely failed to mention it. I'll address it more in the next update!

photo
5

We've made a ton of improvements to the PHP-FPM mangement in the Multi-PHP interface in WHM. We'd love to see you take a look!


dad6a448d4e86b74cc6da344f7319914

photo
1

Very nice.

Some questions from what I see in the image:

- FPM will be able to be turned on/off independently from what PHP version you select as default? - FPM will be able to be turned on/off on a per account basis?

photo
1

That's starting to look pretty good ! will it have options for other FPM settings or would we just change them in the correspinding php.ini ?

photo
1

@Kent: Yup! Enabling FPM will be independent from the PHP version, and is enabled on a per-domain basis.


@Greg: For now it's just those that are adjustable in WHM, and the rest will need to be adjusted in the php.ini.

photo
2

Thanks for responding, thats cool, if possible would be great to have the additional / slightly more advanced options there in the future would be great and save lots of time when making the same changes across all pools.

photo
2

Greg, what sort of options would you like to see? the idea was to put in the options we knew everyone would use then recieve feedback on what else to add. We very purposefully made this UI to not have every single PHP-FPM options available as that would be rather overwhelming to users.

photo
2

Couple of question I haven't been able to get an answer on in the forum. (see here for more details - https://forums.cpanel.net/threads/confused-about-easyapache4-and-fpm.568841/#post-2305351).


If we had used the previous manual method to configure FPM, it was mentioned on the forum that "the changes you must make are not compatible with the actual FPM support in cPanel version 60."


So, if one has been running FPM over the last few versions using the previous method, will that completely break when updating to 60? ...or will it continue to work but just require some manual removals of the user.conf and userdata fpm.conf files before activating the newer method on a user by user domain by domain basis?


Also, the current instructions use the following in the fpm.conf files:


  1. <IfModule proxy_fcgi_module>
  2. ProxyPassMatch "^/(.*\.php(/.*)?)$" "unix:/home/<user>/run/php70fpm.sock|fcgi://localhost/home/<user>/public_html/
  3. DirectoryIndex index.php
  4. </IfModule>



but I used the following that was recommended offsite (http://blog.remirepo.net/post/2014/03/28/PHP-FPM-and-HTTPD-2.4-improvement for the later Apache 2.4 versions:


  1. <IfModule proxy_fcgi_module>
  2. <FilesMatch \.php$>
  3. SetHandler "proxy:unix:/home/<user>/run/php70-fpm.sock|fcgi://localhost"
  4. </FilesMatch>
  5. </IfModule>


... and it did prove to work a lot better and more compatible.


Do you know what method will be used in 60? ...because the outdated ProxyPassMatch method is pretty useless for me since it broke sites such as wordpress multi-site installs, messed with directory indexes disallowing previously working methods (like problems with site with both .php and .html indexes and FancyIndexing file listings). The SetHandler method doesn't have those problems. (see more here - https://forums.cpanel.net/threads/php-fpm-directory-index-index-html-file-not-found-problem.529701/).

photo
2

This is a great question, I personally used this file from cPanel to manually create my FPM pools - https://features.cpanel.net/attachments/1210


like kdean asked will this be read in v60 or is it likely to break the sites and require the removal of the manually created FPM pools, and then recreate using the GUI?


(obviously if the GUI could just scan and import the predefined settings from the manual FPM pools that would be much easier)

photo
2

Hey @kdean and @greg. In version 60 we're implementing using FileMatch, rather than ProxyPass. Additionally, our implementation in version 60 will not work with the old manual directions. The old version should still work after you upgrade, but if you want to use the Multi-PHP manager in WHM you will need to remove those customizations and then implement again in 60.

photo
1

Would you provide instruction on how to deactivate userpool FPM we did with build 58 and allow it to work with Multi-PHP FPM?

photo
1

Thanks Benny, it's not as much of an issue knowing this ahead of time ! I can make a few preparations for this when v60 comes out :)

photo
2

@JayPK: Sure, I can work to get something put together for users that need to convert over, but it should just be reverting your changes and then implementing through WHM. I'll take a look and see what we can do!

photo
1

will this future only available from WHM? what about with cPanel front end for users?

i think it will be really good, if user can decide to use FPM or not.

For php-fpm configuration like benny image

dad6a448d4e86b74cc6da344f7319914


I think that should keep on WHM.

photo
2

There's currently not a plan to allow users to enable this on the cPanel side, only through WHM.

photo
1

hmmm okay, so we can enable users to use FPM by default right?

photo
2

@hrace009 that unfortunately won't be possible in the initial release of our PHP-FPM feature. It may be possible with a post account creation hook, but I'll know more about that next week after we hit code freeze. I'll let you know once this is live!

photo
2

finally there is something,


but please dont forget that you have to use php-fpm in chroot mode

otherwise every account can be hacked easily.

im sure cpanel has users chroot by default too.


also make sure in this case document root can be set properly, and php.ini per pool aswell

cause im not sure how multiple subfolders in the same home dir will work

like

/home/user/public_html/

/home/user/public_html/project_1/

/home/user/public_html/other_user_test_project/


waiting for release to test it.


thank you

photo
1

chroot'ing each user's FPM pool into their /home/virtfs/%user% directory would be ideal. Except, /home/virtfs/%user% doesn't exist for every user and I'm not sure by what logic cPanel determines if a user gets a /home/virtfs/%user% directory.

photo
2

Benny should have some more information on this shortly but the approach we are going with which is off by default is that if a user is in jailshell or noshell then they will be jailed automatically providing that /var/cpanel/feature_toggles/apachefpmjail exists.

The one thing to note is that this works based off a patch to fpm itself which performs a getpwnam whenever a new pool is forked off. This is done to avoid extra restarts to PHP-FPM which can be problematic for a variety of reasons.


You may view the patch at https://raw.githubusercontent.com/CpanelInc/scl-php70/master/SOURCES/php-7.0.x-fpm-jailshell.patch

photo
1

I haven't followed all of this and I'm not entirely sure how you are going to implement all of this, but couldn't most (all?) of this be done in the php-fpm init script?


For example, modify /etc/init.d/ea-php56-php-fpm so that it checks to see if the virtfs directory exists for the user, and create it if it needs it. Something like


  1. for u in `/bin/ls -A /var/cpanel/users`
  2. do
  3. if [ ! -d "/home/virtfs/${u}" ]
  4. then
  5. /usr/local/cpanel/bin/jailmount ${u}
  6. fi
  7. done

And automatically set each /opt/cpanel/ea-php56/root/etc/php-fpm.d/%user%.conf to chroot to


/home/virtfs/%user%


I'm sure it's not completely as simple as that, but this would avoid the need to patch the PHP code with custom code for FPM.

photo
1

sparek: The problem with that approach is that FPM restarts would have to wait on thousands of executions of jailmount before coming back up, meaning that PHP will be unavailable for an undetermined amount of time while it loops through every user on the system.

photo
1

Yea, but how many times does the master php-fpm process have to be restarted? And how long does it take to mount a user's virtfs?


Doesn't patching the fpm source to do this on each instantiation of an FPM pool create startup cost, per process? Seems to me, it would be more cost effective, process-wise, to do this at the beginning rather than during each individual process's start.


Plus, this gives you the advantage of unsetting an FPM pool's chroot - through the user's /opt/cpanel/ea-php56/root/etc/php-fpm.d/%user%.conf file - if you so desire. Which gives you a bit more configuration flexibility.


An addition to this would be to create a process, possible something in tailwatchd, that checks and insures that all user's have a virtfs jailmount all the time. Add a hook after account creation for creating the jailmount virtfs. Just something that insures that /home/virtfs/%user% always exists. Or is there a reason why every user shouldn't have their own virtfs mount? This kind of goes beyond the scope of this feature request, but if every user always has a virtfs mount, then patching the PHP FPM source is moot and really adding my tidbit of code to /etc/init.d/ea-php56-php-fpm is also moot.


The bit of code I posted only runs the jailmount command if a user's virtfs doesn't exist. If every user's virtfs mount always exists, the the process cost of checking this in the init script would seem to be negligible. Certainly less than doing it on a per pool startup check.


I'm not saying my way is the best, but I think it's a lot more simple. I think patching the PHP source with this code is kind of over complicating the issue.

photo
1

I actually tested the timing on this.


An unmodified /etc/init.d/ea-php56-php-fpm - restarting it on one server took on average 0.4526 seconds


Then I modified /etc/init.d/ea-php56-php-fpm to jailmount users if the /home/virtfs/%user% directory didn't exist. Restarting it on the same server took on average 0.715 seconds.


That's a difference of about 0.26233 seconds.


I conducted this test by restarting /etc/init.d/ea-php56-php-fpm, sleeping for 60 seconds, restarting /etc/init.d/ea-php56-php-fpm, sleeping for 60 seconds, and restarting /etc/init.d/ea-php56-php-fpm. Then I took that total time needed to complete this, subtracted the 120 seconds that I slept and divided by 3


The modified /etc/init.d/ea-php56-php-fpm mounted 81 users. But it really only mounted those the first time. Each subsequent /etc/init.d/ea-php56-php-fpm restart would have found that /home/virtfs/%user% already existed and would have skipped over them.


Granted, this is a small test. But I think it shows that this is feasible. Especially if you consider doing a jailmount of each user after account creations and/or have a separate daemon on check that works to keep jailmounts mounted.


I suppose all of this basically boils down to: Why isn't every user automatically given a /home/virtfs/%user% jailmount? Why doesn't this stay consistently mounted? And is there any reason why every user shouldn't have a /home/virtfs/%user% jailmount? If every user consistently has a /home/virtfs/%user% jailmount then all of this is moot and chrooting can be done in each user's FPM pool.


I posed this question on the forums about a month ago, but didn't get a lot of traction.

photo
1

@sparek-3 to answer your questions about virtfs: virtfs is tied to Jailshell. Unless a user's assigned jailshell (or noshell), then we don’t create virtfs.

photo
1

But why?


That's the question I've never gotten an answer to.


I understand that virtfs was initially used for jailshell, but you can expand on this so much more and offer a jailed PHP environment. cPanel even created the /usr/local/cpanel/bin/jailmount binary at some point to create this virtfs for a user.


This is essentially what the php-7.0.x-fpm-jailshell.patch file is doing.


  1. if ( 0 != system( jail_command ) ) {

is running the /usr/local/cpanel/bin/jailmount command for that user.


If every user already has a virtfs jailmount, this isn't needed. And I think it makes more sense to do this in the /etc/init.d/ea-php56-php-fpm init script anyway.


Unless there is a reason why every user shouldn't have their own /home/virtfs/%user% mount point.

photo
1

while its only betabut think this way you have it now is not good approach.


- no chroot?? even in beta, really?

you will have a lot of work to make it right, this is why its not available now...


- only one php-fpm config file? it grabs all the fpm pool configs?? this is a huge security flaw.


php-fpm: master process (/opt/cpanel/ea-php70/root/etc/php-fpm.conf)


- creating master systemd service file? probably file must be in /etc/systemd/system/?

/usr/lib/systemd/system/ea-php70-php-fpm.service


- nameless socket and stored in root folder? it must be created in folder with its user name. and it must have user name so it will be easy to debug many issues.


/opt/cpanel/ea-php70/root/usr/var/run/php-fpm/31af0b71da8ad5e80e86ed883b85b2d83aadb2e5.sock

/opt/cpanel/ea-php70/root/usr/var/run/php-fpm/8fae611e8e6fc1fe96639813de15fd64621324e6.sock

/opt/cpanel/ea-php70/root/usr/var/run/php-fpm/c288a9f521f815ca6697f73faef0799951fcb9b7.sock


- apache log and php-fpm log in different folders??

/etc/apache2/domlogs/

/home/demo/logs/


and lots of other issues...

anyway lets talk about what we have now:


chroot - this is must have, and this is the only way to run php-fpm in multiuser shared environment. php-fpm must be crooted to user home folder. there will be some issues when you have another addon domain, projects in subfolders, etc


in nginx you can easily have master chroot folder but using "map" to dynamically map a document_root into variable and pass it to php-fpm the way you need it.

without hardcoding it in php-fpm pool.


so i guess in cpanel maybe you must have config in:


  1. # To customize this VirtualHost use an include file at the following location
  2. # Include "/etc/apache2/conf.d/userdata/std/2_4/example/example.com/*.conf"


then every cpanel account must have its own php-fpm.conf file, to run its own pool config from it.

first of all - opcache sharing is very dangerous and it brings lots of errors.

then imagine a situation when server is running with 1000 e-commerce accounts and someone made a typo in one of the pool configs. php-fpm wont be able to restart and all accounts are also down. this is a big NO.

i do not want losing clients because of this.


so php-fpm.conf file must be created as:

  1. php-fpm-{cpaneluser}.conf

for every user account there must be its own service file, for example:

  1. /etc/systemd/system/ea-php70-php-fpm-{cpaneluser}.service

and it will call its own php-fpm-{cpaneluser}.conf with its own pid, logs, settings and paths, this will call its own pool config file {cpaneluser}.conf


when i call opcache from one account i can see this:


  1. /home/demo/public_html/index.php
  2. hits: 10, memory: 26.62KB, last used: Mon Sep 26 09:17:36 2016
  3. /home/myshop/public_html/index.php
  4. hits: 5, memory: 26.62KB, last used: Mon Sep 26 09:17:57 2016
  5. /home/example/public_html/index.php
  6. hits: 1, memory: 26.62KB, last used: Mon Sep 26 09:17:47 2016


all logs must be in one location per user account/domain.


i guess you have some idea.


im working with one company that manages our cpanel servers, they have modified it to run with php-fpm for EA3, im pushing them to do some modifications for EA4.

cause you guys have some much resources and time to make it right, but you always fail, and miss every new technology, always step behind...

photo
1

Some interesting suggestions but I'm not agree with them all.


The chroot is really interesting and important, but it should be optional.

Most os cPanel shared servers (almost all mines) uses CageFS or similar solutions that do the same in better way.

The FPM chroot obviously changes the base dir, this will be a big problem in some ugly PHP systems that use absolute path in theirs scripts (/home/{user}/public_html), so in most of cases the sysadms will prefer a CageFS/Jailshell like solution instead a chroot.


About put all config in one or multi files, to be honest, I don't care. In the execution time the effect will be the same.


About the systemd, remember that this won't be only for RHEL 7. The 6 versions will need to run it too.


I'm also don't understand the socket locations and names. Thinking like a hacker I don't care about the sockets name, if I have access to some one the only think that I need to know is that is a FPM socket, I will try exploit it no matter the name they are.


To be honest, as the cPanel has becoming a RPM based system, it will be more interesting if it respect the POSIX and put the sockets in /var/run using the user names in socket name to help the identify. The ACLs of the socket make the security part.


I don't understand what do you trying to say about php-fpm.confs. The normal user don't need to had access to such file, I don't understand how "someone made a typo in one of the pool configs" without be a root?

These files only have the FPM limits and locations, I don't want that any users had access to it, so I don't see any these problems that you like.


And please forget about that crazy idea of make a service for each user. This will be terrible hard to keep, imagine you make some php.ini change and you have to make 1000 restarts?

This strange approach will also waste memory/cpu with excessive overhead for nothing.

Each php-fpm.conf for each user already their own configs to make logs, limits and locations without the need to make a separate service for them or even if all they are in the same conf file, and if you (because your users, even the resellers don't have access to these files) miss a typo, the service log or the stderr will say where the tipo is when you try to restart it.


Sorry for any bad english

photo
1

For chroot support, touch /var/cpanel/feature_toggles/apachefpmjail and make sure the user has noshell or jailshell set as their shell.


As for the master process issue - the development team was adamant on a single master process per version with pools per domain. What are the security issues you are concerned with? We were not able to identify any major issues with the shared master process that we were unable to patch or fix with chrooting.

photo
1

touched apachefpmjail, what exactly i should see after?


one php-fpm master config and process leads to security issues

if you enable opcache one account from one pool will be able to see and extract data from another account pool using opcode cache.

in case of keys collision, exploited wp-config.php from one account will be loaded into another account.


also again , all accounts are hostages of one process.


@Yuri we are talking about default setup, i think CageFS is from CloudLinux.

and there is no problem to restart multiple services in one command, you should know that.


looks like you never had experience hacking one account from another on shared hosting, and believe me CageFS wont save you if i see your files and have code access...

photo
1

I agree with the opcache issue, sharing files is very bad, I'm currently using FPM on a few sites they have the individual pools configured as per the cpanel docs (https://confluence2.cpanel.net/display/CKB/Configure+PHP-FPM+with+User+Pools+for+EasyApache+4) and they share the same cache, if I load an opcache (domain.com/opcache.php) script onto one of the domains and look at the cached files I Can see all the other sites cached files and their location. Meaning a single user could view everything in the pool from other users.

I'm not sure if I'm allowed to post this (If not I'm sorry !) but I have created a request already asking for a WHM opcache integration with separate opcache pools for each user perhaps this could be of use... https://features.cpanel.net/topic/zend-opcache-user-interface-for-whm-cpanel-with-separate-caches-for-each-cpanel-account

photo
1

As far as the opcache issue goes, would using Xcache instead of PHP's opcache be a solution? At least for PHP 5.6. You could set xcache.var_namespace_mode = 2 and xcache.var_namespace="uid" would that work to separate virtualhosts? Or at least separate users on a shared server. Granted, xcache hasn't been updated in 2 years and it doesn't appear to support PHP7 (is it a dead project?)

photo
1

From what I understand its basically recommended for everyone using PHP5.5+ to move to opcache from the other ones such as xcache, APC etc and then in terms of PHP7 I'm not sure the other ones are even fully compatible but could be wrong there

photo
1

Perhaps I'm not the best one to comments on this. As I understand it Xcache is an opcode cacher, just like Zend OpCache that is included with PHP. It is true, that the general consensus is that Zend OpCache works better than Xcache. And, Zend OpCache seems to work with PHP 7, something XCache doesn't. BUT, if Zend OpCache is going to reveal information about other VirtualHosts, then it really doesn't make sense in a shared hosting environment, correct?


So the ability to use Zend OpCache - at least in terms of with PHP-FPM - becomes moot if the security implications of it are that great. So while Zend OpCache may perform best, it's useless if it doesn't provide security.


That's where I'm suggesting (or inquiring about) Xcache. XCache is an opcode cacher, like Zend OpCache. It does not work as well as Zend OpCache and it does not work for PHP 7. But with the namespace feature in XCache, can you isolate the caching per VirtualHost (or per user)? That's what I'm asking.


Ideally, Zend OpCache would include the namespace feature in XCache allowing the opcode cache to be isolated per VirtualHost (or per user). But it doesn't.


XCache, while it doesn't perform as well as Zend OpCache, it's better than nothing, right? (Assuming you are using PHP 5.6 or PHP 5.5). And if XCache can take care of the security implications of isolating opcode cache per VirtualHost (or per user), then isn't it a better choice?


Granted, this does nothing for users that want to use PHP 7 or PHP 7.1.


Zend OpCache might work with FCGI, but then don't you lose the ability to chroot your PHP environment as you do in FPM?


I'm not completely advocating XCache, I'm just asking if it can be an alternative that might provide solution to some of the issues being addressed here.

photo
1

I've been testing the FPM jail chroot (/var/cpanel/feature_toggles/apachefpmjail). I seem to have run into a snag.


It would appear that there is a limit to the number of jailmounts you can have, or at least something that stops further jailmounts after a certain period. At least in an OpenVZ environment. This may be a limitation or conflict with something in OpenVZ.


It would seem that after mounting 157 virtfs jailmounts, further jailmount attempts are not successful. /home/virtfs/%user%/home/%user%/public_html doesn't populate. I doubt that 157 is an arbitrary limit, in fact I have as many as 219 succesful jailmounts in a KVM environment. I suspect that the limit has to do with something else and I think it may have something to with some restrictions in OpenVZ.


This is not an issue specifically with /var/cpanel/feature_toggles/apachefpmjail but instead is an issue with the ability to create jailmounts. I am not able to successfully create jailmounts using the /usr/local/cpanel/bin/jailmount command either.


I will see if I can do some further testing on this when I get caught up on some other work. But in the mean time, I thought I would mention this and see if anybody else has been able to replicate this issue. Specifically within an OpenVZ environment and with more than 157 users (or at some point, do the users stop being successfully jailmounted?). Of note, the /usr/local/cpanel/bin/jailmount command may create the /home/virtfs/%user%/home/%user% structure, but fails to create the /home/virtfs/%user%/home/%user%/public_html structure.

photo
1

After more investigation on this, the mount failures are caused by a restriction on the number of mountpoints inside OpenVZ.


By default (I assume), OpenVZ containers are limited to 4096 mount points (of which, I can only get 4091 mounts, not sure what happened to the other 5). This is controlled by the variable - /proc/sys/fs/ve-mount-nr


You can check the total number of mounts by:


cat /proc/mounts | wc -l


And if you are in an OpenVZ environment, you can check your max mount points by:


cat /proc/sys/fs/ve-mount-nr


This value can only be increased by the host node, either by directly increasing the limit:


echo 10000 >/proc/sys/fs/ve-mount-nr


Or, probably more appropriately through the /etc/sysctl.conf file on the host node, by adding or modifying the variable:


fs.ve-mount-nr = 10000


And saving the file and running sysctl -p


(10000 is just an example in this case, I'm not sure if there is an upper limit that this can be set to)


But this will be dependent on how willing your VPS provider is to make this change on their host node. So bottom line, some OpenVZ servers may experience problems with this PHP-FPM chroot if they are unable to get fs.ve-mount-nr variable increased. It's hard to say exactly what the user limit would be for this, as it depends on the total number of mountpoints you have and how many (if any) additional jailshell mountpoints you have. But I'd say the limit is going to commonly be some where between 150 and 200 users.


Nothing really cPanel can do about this as it's not a cPanel issue, but is an OpenVZ issue. But it might be worth putting an advisory of some sort attached to this PHP-FPM chroot if the environment is OpenVZ.


Does this limit affect any other server environments? I'm not seeing the issue on standard, KVM, or Xen based environments (at least not yet) but might be worth further testing.

photo
1

i just noticed that MPM Event is not used when switching to php-fpm.

will it be fixed somehow?

you have to understand that this is not only opcache issue there.

i still could not test any chroots or jails, even after touching this file /var/cpanel/feature_toggles/apachefpmjail


what exactly should happen?

photo
1

The FPM pool should be automatically chrooted after touching this file, ignoring configuration file settings.

photo
1

Thank you, thank you, thank you to everyone (especially sparek-3 and backblaze) that's been providing feedback here. There's been some discussion around it, and both Matt and Ken have been taking a look. I know we haven't been as vocal as I'd like, but we're definitely taking your feedback into account. As soon as there's anything firm to share here, we'll let you know!

photo
2

Hey everyone! Version 60 just went to CURRENT. We're still discussing many of the concerns that were brought up here, but haven't acted on them (as far as I know). If you want to see specific things added to this feature, definitely submit feature requests for them! If you have any questions feel free to email me.

Comments have been locked on this page!