Allow users to install websites from version control

Jacob Brown shared this idea 1 year ago
Planned

Just a small feature request we think would be useful for clients utilizing web files that get version controlled through svn, git or mercurial.

Our proposed implementation would be to have a VC control tools (ie a GIT button) in the File Manager that allows users to pull/push/checkout/checkin repositories to the directory they choose, and keep it updated.

The current method is to download the zip from git and upload it via the file manager, over-writing files as it goes however we propose the VC method to allow users to roll back after a failed deployment.

Best Answer
photo

This is now officially planned for v68, baring unforeseen circumstances!

Comments (35)

photo
2

Very interesting. How would you expect database migrations (upgrades to the schema, etc) or cache flushes to be handled?

photo
1

Very good point. In terms of CI integration and the like, I'm not totally sure however I imagine that if you're utilizing something some sort of automation, you wouldn't be working with cPanel. Especially now that people are moving towards Docker and it's friends.

Majority of scripts do release an "update" script which handles database alterations. Most scripts I've worked with also don't VC config files, and handle updates to the config file through a script designed to non-destructively change it. (ie wp-config.php is wp-config.php.sample)

photo
1

We could look for a Makefile and run make if we find it. Developers could lump in whatever their needs are into the default make task and dispatch the required tasks from there.

photo
1

Symfony/doctrine and Laravel use the so called migrations that are updated or rolled back through a console command.

Another command also rebuilds the composer autoload cache.

Perhaps along with the remote URL in each site the user could define a couple of commands to be executed after the synchronization.

Alternatively, something like the travis yml could be loaded in the remote rep, to define the commands that need to be executed.

photo
photo
3

This is a feature that I personally feel is extremely valuable to our more technical users, as this is how modern developers interact with code and push code around. When looking the ability to deploy from version control I can see quite a few advantages and disadvantages. When discussing this feature I will use 'git' however that does not mean we are unwilling to look into mercurial or svn support.

I do feel that just building this into the file manager, while possibly a great minimum viable product for this feature, is not harnessing the entire power of git for this feature.

Below are some of the various use cases that I have come up with while grokking this feature:

  • Ability to clone a 3rd party software repository and deploy/update a web app via git
  • Ability to maintain one's own web content from a 3rd party hosted git repository
  • Ability for the host to maintain a list of allowed git repositories that users can deploy software from.

The usual feedback I hear on this is that this is too technical of a feature for use by your average cPanel user.

photo
2

It is definitely how most devs push code around. It's how we've done it at all my jobs, it's how I do it for my personal projects and it's how my friends prefer to share code with each other.

Ideally, you would be able to:

  • Clone
  • Commit/add
  • Push
  • Pull
  • Revert
  • Switch branch

I like the idea of whitelisting/blacklisting repositories. Whether this should be a WHM or cPanel managed list, I'm not sure of. That might be getting a bit too crazy, but it would be good for companies (like the one I work for) who utilize cPanel for "microsites or blogs", but want to maintain that extra security where people can only download from our repos.

It is definitely too technical for the lower level cPanel users (guys who pay $30/month for 2GB of webspace because the hosting company will do most changes to their account, like creating emails, databases, subdomains, etc), but for the mid-level and higher-level cPanel users (casual devs, guys who aren't blase about computers, and professional devs), it would be a welcomed featured.

Most low level guys will just ignore it anyway :P

photo
photo
2

I would better encourage some version controlling system (both remote and on server) so any changes under specific repository over a specific branch can be used for this.

I personally don't feel the cPanel will be interested as this feature involves moderator controls and understanding however for cPanel, I would love to see this as simple as it can i.e. some options where user can change his mode to developer mode and than all such settings may start appearing.

Anyways, I am strongly supporting this feature as in many cases with me, I have to alter system configurations and have to provide shell access to my clients requiring this feature with cpanel which makes the system unstable farmost time.

photo
1

Version control from the shell can already be accomplished. After reading this I decided to check and can confirm that I can checkout files from my SVN repo using a "normal shell" and "jailed shell" using a cPanel accounts login.

What I would really like to see is it implemented into the file manager so that SSH isn't required.

Going by Adams response, I feel that they might be interested and I also feel that it would be a nice additional feature for their hosts to be able to boast about.

photo
photo
1

Honestly, this feature is not necessary. There are more important features that cPanel needs to focus their time on than something the "more technical users" as one puts it, can simply setup themselves. SNV is just about obsolete, and who uses Mercurial? Git is what most everyone uses, and the ones that don't, should.

BUT... What would be nice is if cPanel installed Git by default with itself, like it does everything else, that way developers working on cPanel servers that don't have root access can setup Git repos per project/user.

IF... you do have root access, it is fairly simple to setup Git on cPanel servers. I currently am and have been using Git for my projects for some time now on cPanel servers with no issues. The setup isn't much different, just requires a few extra steps to get it working the "cPanel" way.

As Root:

1. Verify that the /etc/yum.conf file does not contain perl* in the exclude line.

2. yum install git

3. nano /opt/suphp/etc/suphp.conf

change: allow_file_group_writeable=true

change: allow_directory_group_writeable=true

As User: (not root, or you will run into all kinds of permission errors)

1. Enable shell access for user account - Jailed Shell or Normal Shell (Doesn't matter, they both work)

2. Login As User, and setup your repo...

I personally like to initialize my repo before the public_html directory, and follow the cPanel structure in my repo for better compatibility with cPanel severs. But, everyone has their own weird way of doing things, so do whatever makes the most sense to you.

My Way example:

SSH to /home/some-user-account

Type: git init

Type: git config --global user.name "FirstName LastName"

Type: git config --global user.email "youremail.com"

Type: git config --global core.excludesfile ~/.gitignore

Type: git remote add origin https://somerepo.git

/home/some-user-account/.gitignore: (ignore everything but public_html directory)

  1. /*
  2. !/public_html

This .gitignore file is basically for cpanel stuff, you can add another to the public_html directory to ignore project files. Git supports multiple .gitignore files and can be placed just about anywhere. They work the same exact way robots.txt files do. So if you want Git to ignore for example, your cache folders and all xml files, just create another .gitignore file and place it inside your public_html directory.

  1. *.xml
  2. cache/

Anytime you want to deploy, just login as user and type: git pull origin master (or whatever branch u want to pull).

photo
1

As I stated earlier (but after you posted this), you can do it from a shell provided that the web host sets it up on their server. But are webhosts going to invest the time and money into securely setting it up on all their nodes for their clients? Are they going to want to invest in that risk?

Larger ones; sure. Mid-tier and smaller companies; less likely.

Adding this feature would just be yet a nicer thing for cPanel to provide to their websites. At no point have I stated it's necessary, I just said that me - and some other devs I work with - would think it would be nice.

photo
1

I agree that it would be nice. Why I voted thumbs up, even though I highly doubt cPanel will invest the time to implement it.

photo
photo
2

While highly technical in nature, this idea has potential for end-users if it's executed properly. The idea of designers providing a GIT repo URL to customers...and all the customer has to do is register it within cPanel and hit deploy...that has sexy written all over it.

photo
1

It's something cPanel could roll out in multiple stages over time. A good start would be to just have Git installed at server level by default, so its available at the user level without having to have the server admin install/setup Git. This quick-fix solution would still require the admin to enable shell access for users, but at least Git would already be available to use once a shell is enabled.

Ideally and interface of some sort would be preferred, especially for the non-tech users or in cases where a server admin refuses to enable shell access. Again, this part of it could be rolled out over time. At least the quick-fix solution I suggested would solve the problem for devs that currently have shell access, but not root level access.

I know someone suggested building it into File Manager, but I personally think it would make more sense to build a separate Interface for this, e.g. like Cron Jobs, etc. More simple to maintain and scale since you would essentially be building a stand-alone Git client out of JavaScript.

photo
1

@bruce - your thinking is right in line with how we'd approach this project. As of right now git is already available to end-user cPanel accounts if they have shell access. Beyond that, though, I'm definitely thinking a dedicated interface for configuring git repositories at a file location. (Links to this interface could totally exist inside File Manager so that you could select a folder and start from there...but the feature itself would be outside of File Manager).

photo
photo
1

I'm really curious about this feature. Some of my customers have been asking for this as they are mostly designers and do not have a lot of experience using SSH. Any comments on the status?

photo
1

We haven't considered adding this to the roadmap since I started in this position in March, but I can definitely bring it up again. With only 17 votes over nearly a year, it doesn't seem like this has much interest in the community.

photo
2

I have to use cPanel for clients on a regular basis and I find this extremely frustrating, so much so that I ended up using Plesk instead which has GIT control baked in and can be set up in about 3 clicks. Now I just use subdomains with protected directories that are all set up to automatically pull any changes from GitHub and everything works seamlessly - clients can see me update things in real time.

To confirm cPanelAdamF's statement: It is absolutely sexy.

photo
1

Staging / Development / Production sites with GIT through cPanel. That'd be awesome :)

photo
1

My team and I just had a serious discussion about this with intent to start planning it out. I'll have more to say after we explore the idea more...including a call for focus group members...when we get done with that exploration.

We're excited!

photo
1

Good news!!

A nice feature I like in other services is the ability to configure a webhook (for example at github) so that a pull is triggered whenever a new version is pushed on the master branch.

photo
2

yup, we're looking at that as the main way to be notified of new changes that need to be pulled in. We've also considered a scheduled pull daily (probably during the day rather than overnight).

As far as development goes, first thing is to make the first clones work. Then we'd tackle an 'update now' button...then start talking about triggered updates or regularly scheduled updates or both.

photo
1

Cool :)

Selecting which branch to probe will also be very useful for test/staging environments.The "overnight" thing is a bit dangerous.

What is overnight for one site might be working hours for another.

I would make the execution time configurable per case.

BTW, is this going to be a WHM or Cpanel feature?

Is composer included in installations?

photo
2

Right now our thinking is that this should be a cPanel feature at first. Keeping a directory up to date could be useful at the WHM layer, but we'd like to keep it simple at first.

re working hours - yeah, definitely. We'd rather updates to the site happen automatically & conveniently...and not cause panic at 1am during upcp.

photo
2

Something for your checklist if you are still gathering specs.

It would be nice to be able to configure pre and post sync scripts. That way things like cache clear, composer installations or updates and the like can be automated too.

Also, it would be nice to have a way to define environment variables and .env files (used by laravel and symfony), although this can be probably handled manually from the file manager.

photo
2

On it. We were already thinking about pre-sync/post-sync scripts. Hadn't thought of ENV variables, though, glad you said something about that!

photo
1

@dimkasta - Something to keep in mind is that the work for https://features.cpanel.net/topic/ruby-app-support-through-phusion-passenger will already have the ability to specify ENV variables for the launch of processes. Are you wanting additional set of ENV vars for the pre-flight and post-flight scripts too?

photo
1

Thanks for the tip

I will check it, although I do not use ruby

In any case, the file manager works ok for those files

photo
photo
2

This is now officially planned for v68, baring unforeseen circumstances!

photo
1

Nice!

photo
photo
1

A lot of this is already do-able in v64 and newer. We're prepping some documentation for advanced users comfortable at the command line that will describe the way to achieve it right now.

Don't worry. Still building out a nice new feature in the cpanel interface for it too ;-)

photo
1

New documentation for how to do it right now (v64 or newer) is now available.

https://documentation.cpanel.net/display/CKB/How+to+Host+Git+Repositories+on+a+cPanel+Account

As a result of working up this doc, the team and I identified three major scenarios that we need to plan for in the development of this feature:

  1. "Single Remote without automatic deployment" - basically just repository hosting. For people who don't want to use Github or one of the other special purpose hosts.
  2. "Single Remote with automatic deployment" - hosting a repo on your cpanel account with the expectation that new changes are automatically checked-out upon push
  3. "Multiple Remote with automatic deployment" - hosting a clone of a repository hosted elsewhere (eg Github) and automatically monitoring that remote for new updates.

I've got those roughly ordered in terms of complexity. 1 and 2 are already possible right now, and the new document explains how to do it via command-line. 3 is where the fun is ;-)

photo
1

Question for the room: How do you plan on marketing this feature? What would advertisements on your hosting company's website look like for this?

I ask because this will help guide us in how best to chop up this feature's capabilities and how best to schedule out which parts of the project we do first.

photo
4

"Dev like it's 2017, not '97" — perhaps? :p

photo
1

Laravel forge and envoyer might give you nice ideas

photo
1

What would your hosting Packages (like, WHM Packages) look like? Do we need to slice-up the feature into more than one 'feature manager'-entry so that you can fine-tune your offerings?

photo