Using GitLab CI

Date: 2016-08-23 - Categories: gitlab

I use GitLab for managing all my git repositories and recently started using GitLab CI to run tests when I push from my local machine. Fast forward to today, this website is generated using GitLab CI job and rsync’d onto this server. The setup is fairly simple, I have two branches, master (the production site) and staging, any push the staging branch is deployed to a staging version of the site, and when staging is merged into master it deploys to the production site.

My .gitlab-ci.yml file

stages:
  - build
  - deploy

build:
  image: python:2.7-alpine
  stage: build
  script:
  - pip install -r requirements.txt
  - echo "${CI_BUILD_REF_NAME}" > sphinx_bootstrap_theme/bootstrap/build.html
  - sphinx-build -b html site/source site/html
  artifacts:
    paths:
    - site/html

deploy-live:
  image: pfarmer1978/deployer
  stage: deploy
  only:
    - master
  script:
  - eval $(ssh-agent -s)
  - ssh-add <(echo "$SSH_PRIVATE_KEY")
  - mkdir -p ~/.ssh
  - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
  - echo "$CI_BUILD_REF_NAME / $CI_BUILD_REF / $CI_PIPELINE_ID" > site/html/build.info
  - rsync -av site/html/ web@projectchilli.com:sites/projectchilli.com/html
  environment: Production

deploy-staging:
  image: pfarmer1978/deployer
  stage: deploy
  only:
    - staging
  script:
  - eval $(ssh-agent -s)
  - ssh-add <(echo "$SSH_PRIVATE_KEY")
  - mkdir -p ~/.ssh
  - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
  - echo "$CI_BUILD_REF_NAME / $CI_BUILD_REF / $CI_PIPELINE_ID" > site/html/build.info
  - rsync -av site/html/ web@projectchilli.com:sites/staging.projectchilli.com/html
  environment: Staging

Breakdown of the .gitlab-ci.yml file

stages:
  - build
  - deploy

Here we have two stages for the CI to run through, build and deploy. Basically each job is assigned to a stage, so all the jobs for certain stage are run during that stage. Below is the build job.

build:
  image: python:2.7-alpine
  stage: build
  script:
  - pip install -r requirements.txt
  - echo "${CI_BUILD_REF_NAME}" > sphinx_bootstrap_theme/bootstrap/build.html
  - sphinx-build -b html site/source site/html
  artifacts:
    paths:
    - site/html

The build job uses a python docker image (python:2.7-alpine) as the base, and run the following commands inside the image:

- pip install -r requirements.txt
- echo "${CI_BUILD_REF_NAME}" > sphinx_bootstrap_theme/bootstrap/build.html
- sphinx-build -b html site/source site/html

This builds the sphinx based website into the site/html directory. The CI engine then collects the site/html directory as an artifact for use in later stages:

artifacts:
  paths:
  - site/html

That is the build stage complete. We then move onto the deploy stage. There are two deploy jobs, deploy-live and deploy-staging, we’ll walk through the deploy-live job, but with deploy-staging in mind.

deploy-live:
  image: pfarmer1978/deployer
  stage: deploy
  only:
    - master
  script:
  - eval $(ssh-agent -s)
  - ssh-add <(echo "$SSH_PRIVATE_KEY")
  - mkdir -p ~/.ssh
  - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
  - echo "$CI_BUILD_REF_NAME / $CI_BUILD_REF / $CI_PIPELINE_ID" > site/html/build.info
  - rsync -av site/html/ web@projectchilli.com:sites/projectchilli.com/html
  environment: Production

This job uses a custom docker image (pfarmer1978/deployer) as the base, and runs the following commands inside the image:

- eval $(ssh-agent -s)
- ssh-add <(echo "$SSH_PRIVATE_KEY")
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
- echo "$CI_BUILD_REF_NAME / $CI_BUILD_REF / $CI_PIPELINE_ID" > site/html/build.info
- rsync -av site/html/ web@projectchilli.com:sites/projectchilli.com/html

These commands require a little bit of explanation. Firstly, using eval we pull an ssh-agenti into the enviroment variables of the shell, and then use ssh-add to add an ssh private key into the agent, the content of which are in the $SSH_PRIVATE_KEY, the $SSH_PRIVATE_KEY variable is configured in Variables in the project settings (in the GitLab UI). Then we create the ~/.ssh/ directory and create a small ssh config file. The ssh config file allows rsync to run over ssh without needing to worry about accepting host keys etc. The ssh config file looks like this:

Host *
    StrictHostKeyChecking no

We then echo some of the CI’s built in variables into the site/html/build.info file, and then rsync all the files onto the web server.

There are two differences between deploy-live and deploy-staging, the only and environment settings. The only setting specifies that the job should only run if the git refs match its parameter, environment allows the tracking of deployments via the “Environment” page in the GitLab pipelines UI. In deploy-live only is set to master, this means, only pushes to the master branch will run the deploy-live job. Pushes to the staging branch would run the deploy-staging job.