Create staging environment using GitLab

Gitlab Webhooks

Prequisite: Please go through the steps to set up GitLab on the Production/live server before going through this tutorial.

Steps to create a staging environment using GitLab branch

1. Create a new branch for your GitLab project as shown below

1 A. Go to your project page in GitLab and click the option to add New branch

Create a Gitlab branch
Add new Gitlab branch

1 B. Create a new branch named as dev

Gitlab new branch

2. Create your staging site by copying the files and the necessary databases

3. Follow all the steps mentioned on the article Autodeployment using GitLab Webhooks on your staging sever with only the exception of Step 7 which needs to be modified slightly on the staging server. For the staging server that command would change to

Please note we are checking out only the dev branch on the staging server. So only your changes on the dev branch will be seen on this server.

4. Add the staging webhook.php file also to the list of webhooks on GitLab for Push events as shown below

GitLab Webhook

This completes setting up your staging server to preview your changes on the dev branch before they get merged to the master branch.

Now the question is how do we commit our changes?

Steps to commit your changes

    1. On your local (working) git repository first of all go on to the dev branch before you make any change
    2. Now make changes to the required files and commit your changes. Since you are on dev branch the changes will automatically get committed to the dev branch.
    3. Now push your changes to the dev server
    4. Observe the changes on the dev server
    5. Assuming the changes are ok, move on to the master branch
    6. Merge the dev branch with master
    7. Push the changes to the production/live server

Autodeployment using GitLab Webhooks

Gitlab webhooks

GitLab Webhooks

GitLab provides various Webhooks to perform automated tasks after a certain event. For e.g. Push events Gitlab webhooks will work exactly similar to post-receive hook in GIT. Only thing is it needs a webpage (mostly on your server) which gets executed after a certain commit is made to the GitLab repository. This webpage can perform tasks like updating the local mirror repository and making the corresponding change in the files in your public_html folder. Read more about webhooks in GitLab

Assuming you have a GitLab repository at: git@gitlab.com:username/some_project.git

Steps to follow on your production/live server

  1. SSH to your production/live server
  2. Generate an SSH key for auto deployment to be used by the GitLab server

    Name of the key: gitlab_rsa_deploy, keep the passphrase blank
  3. Add the contents of the gitlab_rsa_deploy.pub key to Deyloy Keys section of your Project Settings and Enable it
  4. Create a config file in .ssh folder and add below contents to the file. Add your user key added to GitLab user key settings.
  5. Create a folder name gitlab preferably outside your public_html folder
  6. Create a mirror repository by cloning your gitlab project
  7. Set Work Directory
  8. Perform git fetch once on the mirror repository so that the SSH keys get added to the server
  9. Create the webhook file on your server as webhook.php. Add below contents to the file and save this file inside your public_html folder.
  10. Add the URL of the webhook.php file to the list of webhooks on GitLab for Push events as shown below
    GitLab Webhook
  11. Now just commit some change either through GitLab or through your local repository and check if autodeployment is working

Types of GIT repositories

GIT Repositories
GIT Repositories

Three types of GIT repositories

  1. GIT working repository
  2. GIT bare repository
  3. GIT mirror repository

GIT working repository

This repository is the one which is on your local machine. This is the respositoy which you work on i.e. add, edit or delete your application files and commit your changes.

This repository can be created using below 2 ways

  1. git init: Initialises an empty working repository. Necessary files are then added or created later. This command is mostly used when you are starting with a new application in which case the repository is empty when you begin.
  2. git clone: This command is used if you already have a working repository on your server, GitHub, GitLab, etc. This will create a Git repository on your local maching with working/application files already inside the repository.

This repository has a .git folder with all the version related files inside this folder while your application files reside outside this folder.

GIT bare repository

This repository is generally created on your server mostly for sharing or collaboration purposes. This repository does not include your application files. It only includes the version control related files. All the commits from the local repositories are made to this repository

This repository is created using below command

GIT mirror repsitory

This repository is the exact replica of the source repository. It is created for mainly 2 purposes

  1. Autodeployment
  2. Back up your GIT repository to a remote server

As shown in the above diagram, a mirror repository is created on your production server to autodeploy the commits to the working directory. In the above diagram, local commits are pushed to GitHub or GitLab repository. A webhook on these servers then updates the mirror repository on your production server and updates the files in your working directory.

How to connect to a remote GIT from Windows PC

  1. Download and Install GIT Tool from below URL: https://msysgit.github.io/
  2. While installing the tool keep all the default settings
  3. Create a folder named as projects for your site/application (maybe in your Documents folder)
  4. Open GIT GUI tool and generate a key from the Help menu
  5. The keys get created in your Users/USERNAME/.ssh folder
  6. Import both (public and private) keys to your server using CPanel
  7. Open GIT command line tool and navigate to the Documents folder (i.e. one folder above projects)
  8. Run the command: git clone ssh://USERNAME@IP_ADDRESS/home/USER/repositories/REPO_NAME

 

Now just make some changes to your code and commit the change using

git commit

The changes can be pushed to the remote server using below command

git push origin master

Set remote GIT on VPS / Dedicated server

It is possible to set Remote GIT on your own server provided your hosting provider allows to install GIT on the server.

If your site/application is hosted on VPS or a dedicated server then it much easier to do so.

Prerequisite to set remote GIT

GIT should be installed on there server

If you are on VPS or dedicated server then it is likely that GIT is already installed on the server.

To check if GIT is installed or not SSH to your server and try below command.

If GIT is installed then it will show the version of GIT.

If not it will give an error message.

If GIT is not installed then you first need to install GIT on your server. For this you may need root access to your server. If you do not have root access ask your hosting provider to install GIT for you.

Once git is installed follow below steps on the server to set remote GIT

Create a Directory

SSH to your server (using Putty on Windows or using terminal on MAC)

Initialise Repository

Hook (Push to deploy)

post-receive hook in GIT is used to perform action after receiving or committing a file to GIT. This hook can be used to deploy our changes automatically to the live server using below commands.

Add below content to the post-receive hook/file

Press ctrl+d to save the file

File permissions

Through these simple steps your GIT is ready to accept commits. On the client side or on the localhost you can set up Remote GIT to push your changes to the server. This is easily possible if you are using Netbeans or similar software.

Importing SSH key into keychain on Mac

Importing an SSH key to your keychain on Mac allows you to login without entering your passphrase each time.

This helps if you are a developer and require to commit your changes to the server quite frequently.

Here are the steps

  1. If you have a Cpanel on your server, login to your CPanel
  2. Generate an SSH Key (choose a suitable passphrase while generating the key and make a note of it)
  3. Once the SSH Key is generated, Authorise the key
  4. Download Private and Public keys and copy both the keys to your ~/.ssh directory
  5. Change permissions on both keys, Remove staff, everyone no access, Admin read write
  6. Open terminal and run: ssh-add -K ~/.ssh/KEY_NAME
  7. When prompted enter passphrase used while creating the key

How to remove a file/folder from GIT but keep it in the file system

Using the below commands will remove the file/folder from GIT but will also remove them from the local file system

So how can I remove this file/folder from the repo without deleting them in my local file system

So the conclusion is git rm removes a file from the working directory and the index and when you commit, the file is removed from the tree as well while git rm –cached removes the file from only the index and keeps it in your working copy.