Manually running builds, executing tests and deploying can become a nightmare and an error-prone process:
What if:
That's the benefit you get of configuring a CI/CD server (in this post we will use Github Actions) and mixing it up with Docker container technology.
This is the third post of the Hello Docker series, the first and second posts.
In this post we will use Github Actions to automatically trigger the following processes on every merge to master or pull request:
We will configure this for both a Front End project and a Back End project.
If you want to dig into the details, keep on reading :)
Steps that we are going to follow:
A Chat app will be taken as an example. The application is split into two parts: client (Front End) and server (Back End), which will be containerized using Docker and deployed using Docker Containers.
We already have a couple of repositories that will create a chat application together.
Following our actual deployment approach (check first post in this series), a third part will be included as well: a load balancer - its responsibility will be to route the traffic to the front or back depending on the requesting url. The load balancer will also be containerized and deployed using a Docker Container.
In our last post we took an Ubuntu + Nodejs Docker image as our starting point; it was great to retrieve it from Docker Image and to have control of which version you're downloading.
Wouldn't it be cool to be able to push our own images to that Docker Hub Registry including versioning? That's what Docker Hub offers you: you can create your own account and upload your Docker Images there.
Advantages of using Docker Hub:
Docker Hub is great to get started: you can create an account for free and upload your docker images (free version has a restriction: you get unlimited public repositories and one private repository).
If later on you need to use it for your business purposes and keep and restrict the access, you can use a private Docker registry, some providers:
Although Docker helps us standardize the creation of a given environment and configuration, building new releases manually can become a tedious and error-prone process:
Imagine doing that manually on every merge to master; you will get sick of this deployment hell... Is there any automated way to do that? Github Actions to the rescue!
Just by spending some time creating an initial configuration, Github Actions will automatically:
One of the advantages of Github Actions is that it's quite easy to setup:
If you want to follow this tutorial you can start by forking the Front End and Back End repos:
By forking these repos, a copy will be created in your github account and you will be able to link them to your Github Actions and setup the CI process.
In our previous post from this series we consumed an image container from Docker Hub. If we want to upload our own image containers to the hub (free for public images), we need to create an account, which you can do in the following link.
In this tutorial, we will be applying automation to our forked chat application's repositories using a Github action workflow. Github Actions will launch a task after every commit where the following tasks will be executed:
Before we start automating stuff, let's give the manual process a try.
Once you have your Docker Hub account, you can interact with it from your shell (open your bash bash terminal, or windows cmd).
You can log into to Docker Hub.
$ docker login
In order to push your images, they have to be tagged according to the following pattern:
<Docker Hub user name>/<name of the image>:<version>
The version is optional. If none is specified, latest
will be assigned.
$ docker tag front <Docker Hub user name>/front
and finally push it
$ docker push <Docker Hub user name>/front
From now on, the image will be available for everyone's use.
$ docker pull <Docker Hub user name>/front
To create the rest of the images that we need, we have to follow the exact same steps.
This can become a tedious and error-prone process. In the following steps we will learn how to automate this using Github Actions CI/CD.
Once they are forked, you need to enter in each of the project settings (Back and Front) your Docker Hub user and password as environment variables. This action has to be done in both repositories.
Front
Back
These variables will be used later to log into Docker Hub (note down: the first time you enter the data in these environment variables, they are shown as clear text. Once you set them up, they are shown as a password field).
Now we can finally begin to automate our tasks.
To configure Github Actions, we need to create a file named main.yml
in the .github/workflows
folder of your project. This is where we'll describe the actions that will be executed by Github Actions. We could have as many yml
files as we want.
The steps to create the yml
file for the back end application are the following:
A summary of this build process:
Let's create our main.yml file at the .github/workflows
folder of our backend repository:
We will start by indicating the workflow name.
./.github/workflows/main.yml
+ name: Backend Chat CI/CD
We can trigger this workflow on several events like push
, pull_request
, etc.
In this case we are going to launch the CI/CD process whenever there is a push to master, or Pull Request pointing to master.
./.github/workflows/main.yml
name: Backend Chat CI/CD
+ on:
+ push:
+ branches:
+ - master
+ pull_request:
+ branches:
+ - master
After that, we can create different jobs for build, deploy, etc. We will start with the ci
job (build and tests) and select OS (Operating System) that all tests will run under. Here you have the list of available OS.
In this case we will choose a linux instance (Ubuntu).
./.github/workflows/main.yml
name: Backend Chat CI/CD
on:
push:
branches:
- master
pull_request:
branches:
- master
+ jobs:
+ ci:
+ runs-on: ubuntu-latest
We need access to all files in repository so we need clone the repository in this environment. Instead of manually defining all steps to clone it from scratch we can use an action
already created, that is, we have available official actions
from Github teams or other companies in the Github Marketplace. In this case, we will use action checkout to checkout our repository from Github team (download repository to a given folder):
./.github/workflows/main.yml
...
jobs:
ci:
runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v1
We will make use of another action in order to setup-node. this action developed by the Github team will allow us to get up and running the nodejs environment (even specifying a version):
In this case we are going to request nodejs version 12.x.
./.github/workflows/main.yml
...
steps:
- uses: actions/checkout@v1
+ - uses: actions/setup-node@v1
+ with:
+ node-version: '12.x'
So we've got our ubuntu + nodejs machine up and running. With the checkout action we have already downloaded our project source code from the repository, so now it's time to execute an npm install before we start running the tests.
To execute it, we create a new step with name Install
and we execute the command npm install in the run
section.
./.github/workflows/main.yml
...
steps:
- uses: actions/checkout@v1
- uses: actions/setup-node@v1
with:
node-version: '12.x'
+ - name: Install
+ run: npm install
All the plumbing is ready, so now we can add a npm test command; this will just run all the tests from our test battery.
We are including install and test scripts in a common step (we have to add a pipe character to indicate that we are going to run several scripts on the same step), you can as well split it into separate steps.
./.github/workflows/main.yml
...
steps:
- uses: actions/checkout@v1
- uses: actions/setup-node@v1
with:
node-version: '12.x'
- - name: Install
+ - name: Install & Tests
- run: npm install
+ run: |
+ npm install
+ npm test
Right after all the scripts have been executed, we want to upload the Docker image that we will generate to the Docker Hub Registry.
Github Actions runs over the OS that we had defined in runs_on
section. These virtual-environments include commonly-used preinstalled software, this allows us to access docker
without the need of running an install docker step.
The first step is to login into the docker hub (we will make use of the environment variables we added into our repository secrets
section.
See section Linking docker hub credentials in this post).
We will create a new job, the cd
(continous delivery) job that:
ci
(continous integration) job to be completed successfully../.github/workflows/main.yml
...
jobs:
ci:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- uses: actions/setup-node@v1
with:
node-version: '12.x'
- name: Install & Tests
run: |
npm install
npm test
+ cd:
+ runs-on: ubuntu-latest
+ needs: ci
+ steps:
+ - uses: actions/checkout@v1
+ - name: Docker login
+ run: docker login -u ${{ secrets.DOCKER_USER }} -p ${{ secrets.DOCKER_PASSWORD }}
In the next step we will continue working on the cd job (this time building the Docker Image).
In the previous post we created a Dockerfile configuring the build steps. Let's copy the content of that file and place it at the root of your repository (filename: Dockerfile).
./.Dockerfile
FROM node
WORKDIR /opt/back
COPY . .
RUN npm install
EXPOSE 3000
ENTRYPOINT ["npm", "start"]
Just as a reminder about this Dockerfile configuration:
Let's jump back into the yml file: inside the cd job, right after the docker login, we add the command to build the Docker Container image.
./.github/workflows/main.yml
...
steps:
- uses: actions/checkout@v1
- name: Docker login
run: docker login -u ${{ secrets.DOCKER_USER }} -p ${{ secrets.DOCKER_PASSWORD }}
+ - name: Build
+ run: docker build -t back .
This command will search for the Dockerfile file that we have just created at the root of your backend repository, and follow the steps to build it.
Hey! I've just realized something strange is going on: you are using different containers to get started, Github action runs the test on a given linux instance and the Dockerfile uses another linux / node configuration pulled from the Docker Hub Registry. That's a bad smell, isn't it? You are totally right! Both setup node and the Dockerfile configuration should start from the same image container. We need to make sure that the test runs in the same configuration as we would have in production. The best solution is use the container section inside a job to run an action from docker image like:
jobs:
my_job:
container:
image: node:10.16-jessie
env:
NODE_ENV: development
ports:
- 80
We could update our configuration and it would be something like:
./.github/workflows/main.yml
...
jobs:
ci:
runs-on: ubuntu-latest
+ container:
+ image: node
steps:
- uses: actions/checkout@v1
- - uses: actions/setup-node@v1
- with:
- node-version: '12.x'
- name: Install & Tests
run: |
npm install
npm test
cd:
runs-on: ubuntu-latest
needs: ci
steps:
- uses: actions/checkout@v1
- name: Docker login
run: docker login -u ${{ secrets.DOCKER_USER }} -p ${{ secrets.DOCKER_PASSWORD }}
- name: Build
run: docker build -t back .
We will apply this image for the ci step, but for the cd we won't need to set it up since we are just building the Docker image defined in the Dockerfile.
The current Docker image that we have generated has the following name: back. In order to upload it to Docker Hub registry, we need to add a more elaborated and unique name:
On the other hand, we will indicate that the current image that we have generated is the latest docker image available.
In a real project, this could vary depending on your needs.
./.github/workflows/main.yml
...
steps:
- uses: actions/checkout@v1
- name: Docker login
run: docker login -u ${{ secrets.DOCKER_USER }} -p ${{ secrets.DOCKER_PASSWORD }}
- name: Build
run: docker build -t back .
+ - name: Tags
+ run: |
+ docker tag back ${{ secrets.DOCKER_USER }}/back:${{ github.sha }}
+ docker tag back ${{ secrets.DOCKER_USER }}/back:latest
Now that we've got unique names, we need to push the Docker Images into the Docker Registry. We will use the docker push command for this.
Note down that first of all we are pushing the ${{ secrets.DOCKER_USER }}/back:${{ github.sha }}
image, and then the ${{ secrets.DOCKER_USER }}/back:latest
. Doesn't this mean that the image will be uploaded twice? The answer is no. Docker is smart enough to identify that the image is the same, so it will assign two different "names" to the same image in the Docker Repository.
./.github/workflows/main.yml
...
steps:
- uses: actions/checkout@v1
- name: Docker login
run: docker login -u ${{ secrets.DOCKER_USER }} -p ${{ secrets.DOCKER_PASSWORD }}
- name: Build
run: docker build -t back .
- name: Tags
run: |
docker tag back ${{ secrets.DOCKER_USER }}/back:${{ github.sha }}
docker tag back ${{ secrets.DOCKER_USER }}/back:latest
+ - name: Push
+ run: |
+ docker push ${{ secrets.DOCKER_USER }}/back:${{ github.sha }}
+ docker push ${{ secrets.DOCKER_USER }}/back:latest
The final main.yml
should look like this:
./.github/workflows/main.yml
name: Backend Chat CI/CD
on:
push:
branches:
- master
pull_request:
branches:
- master
jobs:
ci:
runs-on: ubuntu-latest
container:
image: node
steps:
- uses: actions/checkout@v1
- name: Install & Tests
run: |
npm install
npm test
cd:
runs-on: ubuntu-latest
needs: ci
steps:
- uses: actions/checkout@v1
- name: Docker login
run: docker login -u ${{ secrets.DOCKER_USER }} -p ${{ secrets.DOCKER_PASSWORD }}
- name: Build
run: docker build -t back .
- name: Tags
run: |
docker tag back ${{ secrets.DOCKER_USER }}/back:${{ github.sha }}
docker tag back ${{ secrets.DOCKER_USER }}/back:latest
- name: Push
run: |
docker push ${{ secrets.DOCKER_USER }}/back:${{ github.sha }}
docker push ${{ secrets.DOCKER_USER }}/back:latest
Now if you push all this configuration to Github it will automatically trigger the build .
Once finished, you can check if the docker image has been generated successfully:
And we can check if the image is available in our Docker Hub Registry account:
The steps for creating main.yml
are quite similar to the previous one (backend).
Let's define the workflow name:
./.github/workflows/main.yml
+ name: Frontend Chat CI/CD
We will follow a similar approach as in the Back End workflow: We can trigger this workflow on several events like push
, pull_request
, etc.
In this case we are going to launch the CI/CD process whenever there is a push to master, or Pull Request pointing to master.
./.github/workflows/main.yml
name: Frontend Chat CI/CD
+ on:
+ push:
+ branches:
+ - master
+ pull_request:
+ branches:
+ - master
Define the ci
job:
npm install
and npm test
commands:./.github/workflows/main.yml
...
+ jobs:
+ ci:
+ runs-on: ubuntu-latest
+ container:
+ image: node
+ steps:
+ - uses: actions/checkout@v1
+ - name: Install & Tests
+ run: |
+ npm install
+ npm test
Create the cd
job that will start right after the ci
job has been completed succesfully and checkout the repository to get all files inside the running instance.
./.github/workflows/main.yml
...
jobs:
...
+ cd:
+ runs-on: ubuntu-latest
+ needs: ci
+ steps:
+ - uses: actions/checkout@v1
The next step is to log into Docker Hub.
./.github/workflows/main.yml
...
jobs:
...
cd:
runs-on: ubuntu-latest
needs: ci
steps:
- uses: actions/checkout@v1
+ - name: Docker login
+ run: docker login -u ${{ secrets.DOCKER_USER }} -p ${{ secrets.DOCKER_PASSWORD }}
Let's build the Docker Image.
./.github/workflows/main.yml
...
jobs:
...
cd:
runs-on: ubuntu-latest
needs: ci
steps:
- uses: actions/checkout@v1
- name: Docker login
run: docker login -u ${{ secrets.DOCKER_USER }} -p ${{ secrets.DOCKER_PASSWORD }}
+ - name: Build
+ run: docker build -t front .
As we did with the backend application, we are going to tag the current version using the commit SHA and define it as latest
.
./.github/workflows/main.yml
...
jobs:
...
cd:
runs-on: ubuntu-latest
needs: ci
steps:
- uses: actions/checkout@v1
- name: Docker login
run: docker login -u ${{ secrets.DOCKER_USER }} -p ${{ secrets.DOCKER_PASSWORD }}
- name: Build
run: docker build -t front .
+ - name: Tags
+ run: |
+ docker tag front ${{ secrets.DOCKER_USER }}/front:${{ github.sha }}
+ docker tag front ${{ secrets.DOCKER_USER }}/front:latest
Now we only need to push the images.
./.github/workflows/main.yml
...
jobs:
...
cd:
runs-on: ubuntu-latest
needs: ci
steps:
- uses: actions/checkout@v1
- name: Docker login
run: docker login -u ${{ secrets.DOCKER_USER }} -p ${{ secrets.DOCKER_PASSWORD }}
- name: Build
run: docker build -t front .
- name: Tags
run: |
docker tag front ${{ secrets.DOCKER_USER }}/front:${{ github.sha }}
docker tag front ${{ secrets.DOCKER_USER }}/front:latest
+ - name: Push
+ run: |
+ docker push ${{ secrets.DOCKER_USER }}/front:${{ github.sha }}
+ docker push ${{ secrets.DOCKER_USER }}/front:latest
And the final main.yml
should be like the following:
./.github/workflows/main.yml
name: Frontend Chat CI/CD
on:
push:
branches:
- master
pull_request:
branches:
- master
jobs:
ci:
runs-on: ubuntu-latest
container:
image: node
steps:
- uses: actions/checkout@v1
- name: Install & Tests
run: |
npm install
npm test
cd:
runs-on: ubuntu-latest
needs: ci
steps:
- uses: actions/checkout@v1
- name: Docker login
run: docker login -u ${{ secrets.DOCKER_USER }} -p ${{ secrets.DOCKER_PASSWORD }}
- name: Build
run: docker build -t front .
- name: Tags
run: |
docker tag front ${{ secrets.DOCKER_USER }}/front:${{ github.sha }}
docker tag front ${{ secrets.DOCKER_USER }}/front:latest
- name: Push
run: |
docker push ${{ secrets.DOCKER_USER }}/front:${{ github.sha }}
docker push ${{ secrets.DOCKER_USER }}/front:latest
Finally we will include the Dockerfile to the frontend
project as we did in previous post:
./.Dockerfile
FROM node AS builder
WORKDIR /opt/front
COPY . .
RUN npm install
RUN npm run build:prod
FROM nginx
WORKDIR /var/www/front
COPY --from=builder /opt/front/dist/ .
COPY nginx.conf /etc/nginx/
And include the nginx.conf
file:
./nginx.conf
worker_processes 2;
user www-data;
events {
use epoll;
worker_connections 128;
}
http {
include mime.types;
charset utf-8;
server {
listen 80;
location / {
root /var/www/front;
}
}
}
Just as a reminder about this nginx.conf file:
events set of directives for connection management.
http defines the HTTP server directives.
Let's check if our CI configuration is working as expected. First let's make sure that Github Action has run at least one successful build.
You should see in Github Action build that it has been launched for Front End and Back End repos (login into each github repository):
You should see the images available in the docker registry (login into docker hub):
As we did in our previous post, we can launch our whole system using Docker Compose. However, in this case for the Front End and Back End we are going to consume the image containers that we have uploaded to the Docker Hub Registry.
The changes that we are going to introduce to that docker-compose.yml are:
./docker-compose.yml
version: '3.7'
services:
front:
- build: ./container-chat-front-example
+ image: <Docker Hub user name>/front:<version>
back:
- build: ./container-chat-back-example
+ image: <Docker Hub user name>/back:<version>
lb:
build: ./container-chat-lb-example
depends_on:
- front
- back
ports:
- '80:80'
So the docker-compose.yml
will be like:
version: "3.7"
services:
front:
image: <Docker Hub user name>/front:<version>
back:
image: <Docker Hub user name>/back:<version>
lb:
build: ./container-chat-lb-example
depends_on:
- front
- back
ports:
- "80:80"
We can launch it:
$ docker-compose up
It will download the back and front images from the Docker Hub Registry (latest available). You can check out how it works by opening your web browser and typing http://localhost/ (more information about how this works in our previous post Hello Docker)
By introducing this CI/CD step (CI stands for Continous Integration, CD stands for Continuos Delivery), we've got several benefits:
How about deployment? In the next post of this series we will learn how to create automated deploys using Kubernetes, so stay tuned :).
We are a team of Front End developers. If you need training, coaching or consultancy services, don't hesitate to contact us.
C/ Pintor Martínez Cubells 5 Málaga (Spain)
info@lemoncode.net
+34 693 84 24 54
Copyright 2018 Basefactor. All Rights Reserved.