Ability to prevent autossl editing .htaccess files

Monarobase shared this idea 6 months ago
Pre-Release

This request was superseded by https://features.cpanel.net/topic/autossl-act-differently-based-on-apache-version

cPanel currently plans to edit .htaccess rewrite rules in v60 to make sure rewrite rules don't block AutoSSL verification.

We have added appropriate rules to vhosts includes (that require Apache 2.4 or LiteSpeed to work) rendering editing .htaccess files useless in our case.

cPanel's idea is to not just add lines at the beginning or the end of htaccess files but to edit rules that customer's have added to their .htaccess files. We do not want to take the risk of cPanel's automatic edits confusing customers who don't understand why their rules have changed, or maybe even breaking existing .htaccess files causing issues on websites.

We want an option to prevent autossl from editing .htaccess files. This option would be for advanced users so could be in the form of a touch file (something like '/usr/local/apache/autossl_disable_htaccess_edit'). If the file exists then autossl would skip editing .htaccess files.

Best Answer
photo

Hey everyone, thanks for the feedback and sorry for my delay (I was out all last week). I'm going to pass it on to the team that would be doing the work on this, and will let you know what feedback I get from them. In the meantime, I'd like to ask you to take a look at another, related to request.

AutoSSL should adjust behavior based on Apache version.

If you think that might be a better solution, or at least one that would solve this problem for you, I'd like to ask you to remove your vote from this request, and add it to that one.

I'm going to close comments for now, but please know: even if I didn't approve your comment for discussion here I'm passing on the feedback to our development team. If you'd like to add anything further, feel free to send it my way!

Comments (36)

photo
1

One point of clarification here: we are *only* adjusting rules when DCV fails with a redirect error. Additionally, we are adding exceptions to .htaccess files that include redirects only.

For example, if we start here:

  1. root@host # cat .htaccess
  2. RewriteOptions inherit
  3. RewriteEngine on
  4. RewriteCond %{HTTP_HOST} ^.*$
  5. RewriteRule ^duck$ "https\:\/\/duckduckgo\.com\/" [R=301,L]
  6. # BEGIN HTTPS Redirect
  7. RewriteEngine On
  8. RewriteCond %{HTTPS} !=on
  9. RewriteRule ^/?(.*) https://%{SERVER_NAME}/$1 [R,L]
  10. # END HTTPS Redirect

After the adjustments the .htaccess would look like this:

  1. root@host # cat .htaccess
  2. RewriteOptions inherit
  3. RewriteEngine on
  4. RewriteCond %{HTTP_HOST} ^.*$
  5. RewriteCond %{REQUEST_URI} !^/[0-9]+\..+\.cpaneldcv$
  6. RewriteCond %{REQUEST_URI} !^/[A-F0-9]{32}\.txt(?:\ Comodo\ DCV)?$
  7. RewriteCond %{REQUEST_URI} !^/\.well-known/acme-challenge/[0-9a-zA-Z_-]+$
  8. RewriteRule ^duck$ "https\:\/\/duckduckgo\.com\/" [R=301,L]
  9. # BEGIN HTTPS Redirect
  10. RewriteEngine On
  11. RewriteCond %{HTTPS} !=on
  12. RewriteCond %{REQUEST_URI} !^/[0-9]+\..+\.cpaneldcv$
  13. RewriteCond %{REQUEST_URI} !^/[A-F0-9]{32}\.txt(?:\ Comodo\ DCV)?$
  14. RewriteCond %{REQUEST_URI} !^/\.well-known/acme-challenge/[0-9a-zA-Z_-]+$
  15. RewriteRule ^/?(.*) https://%{SERVER_NAME}/$1 [R,L]
  16. # END HTTPS Redirect

Currently we are forced to do this in order to allow us to support Apache 2.2 with AutoSSL. If you are using Apache 2.4 you can avoid this completely by adding code to a pre-virtualhost include to prevent your DCV checks from failing due to redirects.I also want to mention that editing .htaccess files is something cPanel & WHM have done for a very long time, for a multitude of reasons.

photo
2

For Apache 2.4, I don't think it's a previrtualhost include (noone at cPanel has been able to give us anything like this). What we use is a virtualhost include that occurs inside the virtualhost.

This same include can be adapted to work for Apache 2.2 if placed at the beginning of the .htacess file instead of in the virtualhost.

What I'm failing to find is the reason for adding rewrite cond's to all rules instead of adding 5 lines at the beginning of .htaccess files. This would be less likley to break everything and would be more obvious to users that there site hasn't been hacked.

photo
4

How is cPanel considering millions of websites that the CMS controls the htaccess and more specifically, the sites are version controlled in GIT. With the htaccess on the server side being updated, where a server has only read perms from the repo, it creates issues for the git status and even if the solution is simply to download the new htaccess and send it through GIT from the dev side, that would be a significant job where you own/manage a large number of sites, controlled by GIT.

photo
1

That's definitely a perspective we hadn't considered, and makes a whole lot of sense. Practically, it should only trigger these adjustments once, but I completely understand how that's still an incredible inconvenience. I'll definitely be bringing this up internally. Thank you for reaching out!

photo
2

It definitely only should do these changes once, during initial setup. Currently it seems to be doing these RewriteCond additions periodically (since updating to v60).

photo
1

One more thing that should be considered is that a lot of people use nginx as a reverse proxy in front of Apache. In those cases all redirects happens at the nginx level such as redirects from http to https and from www to non www. This is the right thing to do for performance reasons. However this also means that all those lines in the .htaccess from CPanel do nothing at all.

I guess once the certificate is issued for the first time and we have enabled all those redirects in nginx there will be no way to renew the certificate without removing the redirects from nginx. Are there any rules published anywhere that we should add at the nginx level?

Moreover Benny said above that if somebody uses Apache 2.4 a pre-virtualhost include is possible. What should be added at there for those who do use Apache 2.4?

photo
1

You're correct, in this instance, that compatibility with nginx wasn't considered as far as I know. Especially as a proxy. Once we add Nginx support to EA4, support will definitely be considered. This isn't a great place to discuss integration like that, but a forum thread would likely lead to a lot of good feedback.

The pre-virtualhost redirect text is in an above comment, but definitely test it on your server to make sure everything still works as expected.

photo
1

Thank you for your answer. Since I wrote this comment I tried to write my own rules for nginx. So this is what I have now and it seems to be working just fine for every nginx vhost. So this combined with the fact that cPanel writes its own rules at the htaccess files only when needed it fixes my own problem. I thought to write it here just in case somebody else may need it. I agree that further discussion about nginx is not needed at this thread and of course I'm looking forward to the full integration of nginx with EA4.

So if you make any redirect from http to https at nginx as a reverse proxy to apache you might want to place something like this at the beginning of your server block.

  1. location ~*.*\.(txt|cpaneldcv)$ {root /home/yourusername/public_html;try_files $uri @backend;}location / {return 301 https://yourdomain$request_uri;}
  2. #you have to do this in two blocks
  3. #yourodomain.com and http://www.yourdomain.com and place them isnide the server block

This is my own way and it is not guaranteed that it may work

photo
1

Definitely need a solution for this - all mod_rewrite stuff in our git-managed .htaccess files is getting systematically mutilated.. :(

photo
3

We have been running for a bit now with our fix to disable rewite rules, in an apache include. This works for all sites that resolve and don't have any non rewrite based rules that deny access to the files but some customer's .htaccess files are still being modified.

Because of this will still need a way (a touch file, a tweak setting or something else…) to forbid cPanel from editing rewrite rules our customers .htaccess files.

Glad to see that we're not the only ones who need this.

photo
1

Can you share such a patch? Because while this request become truth, a lot of sites all around are being messed up with this issue.

It's a allegedly a feature, but since it is badly implemented, it became a bug for a lot of us.

photo
3

Here are few situations in which we don't want autossl editing the .htaccess file :

If the .htaccess file contains a global Deny from all

If the .htaccess file contains <FilesMatch> for <Files>

cPanel editing customers .htaccess rules is causing confusion for customers, who contact our support with questions like : has my account been hacked ? why are you editing my rewrite rules ? how can we keep default wordpress rules etc.

In our oppinion this feature request should be treated as a bug report and not a feature request.

It's very urgent that cPanel stops doing this on our servers. We don't need it because we have a global include that does this. We don't want it because it creates confusion and customer support requests.

How complicated would it be to add either a touch file or a tweak setting to prevent this from happening ?

photo
1

To clarify for everyone: any edits that are actually causing websites to break need to be reported to our support team, so that we can accomadate and account for them. The edits to the .htaccess files should *never* take a website offline, and if they are then it absolutely IS a bug.

photo
2

Hello Benny. The problem here is that these edits not always break sites. The collateral problem is that lots of customers don't want/like to see their files being edited without permission. They take it badly, like messing with their privacy. They argue that today "you" edited our .htaccess, who will assure no other files are being edited?

photo
2

Ours are not edits causing sites to break, but edits causing customers to worry, loose trust with us and to contact us and complain.

For the moment all we can do is say sorry this is cPanels choice and we can't do anything about it. They answer, if cPanel does things like this why don't you use a different control panel.

We don't like this situation at all.

photo
2

My defining of this as very much a BUG is because the system is making changes to files which wouldn't/shouldn't normally be changed in this way, AND there's no current way to prevent these changes being made (short of disabling autossl), so to my mind that's definitely a BUG.

There's also a huge assumption being made here by the system - that *any* RewriteRule is (potentially) going to prevent the URL to the file autossl needs being read. But that isn't always the case...

For example, a typical RewriteRule scenario for a CMS/framework front-controller might look like:

  1. RewriteCond %{REQUEST_FILENAME} !-f
  2. RewriteCond %{REQUEST_FILENAME} !-d
  3. RewriteRule ^.*$ index.php [L,QSA]

..the RewriteCond's here specifically exclude requests to *real files* or directories, so the rule isn't going to cause a problem. BUT, currently the system will still stuff another redundant RewriteCond in here, and continue to repeatedly do so (if it's manually removed, e.g. by 'git checkout .htaccess).

In short: there absolutely *must* be an option within the autossl settings to prevent .htaccess files being altered in this way .. both globally, and per-site, preferably.

photo
2

I definitely understand the frustration. From our side, we have always classified .htaccess files as part of Apache configuration files, which means we can/should manage them. This conversation has definitely helped us to understand how customers view the .htaccess files: as part of their website content, rather than as a server configuration file.

As always, if there's any change for us, I will definitely be back to let everyone know.

photo
1

You're right Benny, customers (both cPanel's direct customers, plus their respective end-customers) consider .htaccess files to be part of their website; their project. That's because almost anything they'd have in place to run a website these days, be that Zend Framework, Laravel, Symfony, WordPress, etc (the list goes on) .. ALL include a boilerplate .htaccess file out-of-the-box, normally in the web root directory itself. That's why it's always going to be an issue if something external to that (i.e. cPanel) is changing that file..

photo
2

We don't mind cPanel adding code to the top or to the bottom of .htacces files but do mind when lines are added amungst customer edits.

If you choose the touch file solution couldn't you just add an if statement around the code that runs the final autossl search/replace in htaccess files :

  1. if ( ! file_exists(/usr/local/cpanel/deny.autossl.htacces.modifcations))
  2. {
  3. // Your code here
  4. }

This mustn't be that long or complicated to do ?

photo
2

Benny, let me add a suggestion for a quick fix in the meantime: just add a comment after the modified .htaccess entries so customers know it's been automatically modified. "# added by cPanel AutoSSL" or something to that effect.

photo
1

Maybe i'm missing something here but there is no need to edit .htaccess files at all:

We've added the following in the pre_main_2.conf in apache 2.4:

  1. <LocationMatch ".*\/\.">
  2. Require all denied
  3. Redirect 404
  4. </LocationMatch>
  5. <LocationMatch "^\.well-known|\/\.well-known">
  6. Require all granted
  7. </LocationMatch>
  8. <FilesMatch "^\.">
  9. Require all denied
  10. Redirect 404
  11. </FilesMatch>
  12. <DirectoryMatch "^\.|\/\.">
  13. Require all denied
  14. Redirect 404
  15. </DirectoryMatch>
  16. <DirectoryMatch "^\.well-known|\/\.well-known">
  17. Require all granted
  18. </DirectoryMatch>

it seems to do the job. (we also block anything else that start with a dot).

photo
1

Everybody understand that cPANEL should use httconfig includes and catch all calls for AUTOSSL directly there.

WITHOUT ANY NEEDS to create .txt files and update .htaccess of any customers.

THIS REALLY REALLY SIMPLE AND COMPLETLY TRANSPARENTS.

I have already open a ticket about this.

photo
2

Hey everyone. I talked to the Product Owner whose team worked on AutoSSL, and he's considering adding a feature to AutoSSL to help in these situations, but he is incredibly concerned about the correlating increase in support load.

His idea right now is to add an option to "Advanced Options" that would read something like "Prevent cPanel from automatically managing .htaccess files." and would come with a caveat. If the option were enabled, cPanel's support team would not be able to assist you with AutoSSL problems, and enabling that option would add a red banner on the AutoSSL page that we can't help you with AutoSSL unless you turn that off.

How do you folks feel about this? Would this be a good middle ground for you?

photo
1

This is certainly better than nothing. Is this a long term idea or something to resolve the problem for the time being? Medium-long term I would much prefer for cPanel to provide a supported solution, e.g. the DNS verification method or opting for global includes on compatible systems.

photo
1

Yes that sounds fine. We would also need a way of applying this via SSH, either API or just editing a file.

photo
1

Sorry if I don't understand, but can you ellaborate and share why exactly cPanel would be unable to provide support in case this feature is added and enabled?

As I understood after lots of replies, these edits are intended to fix a bug when the server is running Apache 2.2. Based on that, we only want AutoSSL to not mess with .htaccess files if the server is running on Apache 2.4. So... Is there any real reason why the edits are absolutelly required under Apache 2.4?

photo
1

What about adding the no autossl support remark as a quick fix and then detect if the web server is Apache 2.4+ then add the rewrite rules outside the .htaccess.

I guess cPanel will want to do alot of testing before adding the global rewite rules on Apache 2.4+ and that this could take quite a bit of time so allowing us to disable it without support until this happens would be nice.

photo
1

To answer some of your questions:

  • This would likely be a long-term resolution, with allowing the editing of .htaccess files being the primary goal.
  • At this point I and nearly 100% positive that all of our new features come with API calls, so it shouldn't be a problem to make that happen here.
  • The edits are to ensure functionality with Apache 2.2, not to fix a bug in Apache 2.2. Apache 2.2 functions slightly differently than 2.4, which is why the edits were necessary.

I'll bring up the idea of adjusting behavior based on Apache version again, to see if this is worth investigating as well. It sounds like, overall, this is an acceptable middle ground. If that continues to be true I'll let the PO know and he can take all of this to the feature team to see what they think collectively. If you have additional feedback feel free to add it here. I'm about to step away for the weekend, so you likely won't hear from me until next week. :)

photo
3

That's exactly the point Benny. If the edits are to ensure functionality with Apache 2.2, why in the world are the .htaccess being edited under Apache 2.4?

photo
1

I mean... if the server is running Apache 2.4, downgrading to 2.2 is largely unlikely. So, edits are not needed at all.

In the other hand, feel free to edit the files if the server is running on Apache 2.2. All in all, take into account that edits to end users files are not really a nice touch.

photo
1

I can't be 100% certain, but I assume it is a combination of the number of cPanel & WHM servers still running Apache 2.2, and the amount of effort that it would take to develop two ways to accomplish the same thing.

photo
2

Ok, ask the team for this: May it be feasible to evaluate the results of a single IF routine to evaluate the currently installed version of Apache before starting to glob and edit thousands of .htaccess files?

photo
2

I don't understand why cPanel aren't able to use other server-wide Apache configuration ensure that the requests to the request needed by autossl are working (i.e. unaffected by .htaccess RewriteRules) .. seems like having to doctor all user-added RewriteRules with a RewriteCond could if avoided if some condition higher up the 'stack' of Apache configuration levels mitigated it.

Benny, could you ask the team by exactly it is that the current behaviour is apparently the *only* way to mitigate the problem?

photo
2

Nobody understand why Apache 2.2 should work without .htaccess (which is the sake way).

and instead Apache 2.4 should still run messing around .htaccess

Did you see this?

Everybody are asking the SAME: you have to port your script in the .httpconfig and leave .httaccess alone.

This will work both for Apache 2.2 and 2.4!

photo
1

We're OK with no autossl support when .htaccess is turned off so long as we can simply turn it back on if we do need support.

If we need support now for autossl, it won't be because of .htaccess issues. Please do this as soon as possible ! :)

photo
1

Hey everyone, thanks for the feedback and sorry for my delay (I was out all last week). I'm going to pass it on to the team that would be doing the work on this, and will let you know what feedback I get from them. In the meantime, I'd like to ask you to take a look at another, related to request.

AutoSSL should adjust behavior based on Apache version.

If you think that might be a better solution, or at least one that would solve this problem for you, I'd like to ask you to remove your vote from this request, and add it to that one.

I'm going to close comments for now, but please know: even if I didn't approve your comment for discussion here I'm passing on the feedback to our development team. If you'd like to add anything further, feel free to send it my way!