Building Automated CI server with Drone and Docker

Introduction

Docker is great tool to management linux containers. It brings DevOps to next level, from development to production environment. And of course, before deploy anything to production, software should be tested carefully and automatically.

That’s why Drone, a new lightweight CI server built-on top Go lang and Docker, will help us to resolve the testing problems in simple and fast way.

Setup

This guide will assume you already have Docker and Docker Compose tool. And of course, root permission ;)

Step 1 : Clone my example docker-compose here : https://github.com/khanhicetea/drone-ci

1
2
3
$ git clone https://github.com/khanhicetea/drone-ci
$ cd drone-ci
$ cp .env.example .env

Step 2 : Update your setting in .env file

Step 3 : Run drone via docker-compose

1
2
$ source .env
$ sudo docker-compose up -d

Step 4 : Go to your Drone url (remember use https url), then authorize with Github provider.

Usage

In example repo, I created a sample .drone.sample.yml file so you can follow the structure to create own file.

I will explain some basics here

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
clone:
git:
thumbnail: plugins/git
depth: 5

pipeline:
phpunit:
thumbnail: php:7
commands:
- /bin/sh conflict_detector.sh
- /bin/sh phplinter.sh app lib test
- composer install -q --prefer-dist
- test/db/import testdatabase root passwd testdb test.sql
- php -d memory_limit=256M vendor/bin/phpunit --no-coverage --colors=never

notify:
thumbnail: plugins/slack
webhook: [your_slack_webhook_url]
channel: deployment
username: DroneCI
when:
status: success

notify-bug:
thumbnail: plugins/slack
webhook: [your_slack_webhook_url]
channel: bugs
username: DroneCI
when:
status: failure
branch: production

services:
testdatabase:
thumbnail: mysql:5.7
detach: true
environment:
- MYSQL_DATABASE=testdb
- MYSQL_ROOT_PASSWORD=passwd

This file consists 3 sections :

  • clone : To clone the source code and prepare for pipeline step. This section will be run first
  • services : Declare your docker services (databases, ip server) which source code connect to. This section will be run at sametime with pipeline (after clone)
  • pipeline : Testing pipe, where you put testing logic here.

In this pipeline, I made a example PHP testing through these steps :

  1. Check conflicts in code (grep for >>>> HEAD string)
  2. Run PHP linter in application codes
  3. Run Composer to install all dependencies
  4. Import testing database to mysql services (using testdatabase hostname to connect service)
  5. Run testing script via phpunit tool

Then, notify testing result via Slack channel ! ;)

A picture is worth a thousand words

Drone CI screenshot

Lets automate all the things !

#TIL : Checking forced push conflicts on source code in auto testing

Using automated CI solution likes Travis, Jenkins, DroneCI, … is good solution to ensure quality of software and no breaks in deployment.

Sometimes, developers force push conflicts part to production branch of source code. If the CI tests only backend (python, ruby, php, go, ..) and forget about frontend code, then your application will be exploded !

So checking the conflicts code is required step before testing backend and deployment.

I used grep tool to checking conflicts code in current dir

Create a file name conflict_detector.sh in root dir of source code

1
2
3
#!/bin/bash

grep -rli --exclude=conflict_detector.sh --exclude-dir={.git,vendor,venv,node_modules} "<<<<<<< HEAD" .

Then mini tool print list of conflicted files. If exit code not equal 0 then testing will be failed !

#TIL : Using VarDumper in PHPUnit

The trick is writing the output to STDERR stream, I wrote a helper function below

1
2
3
4
5
6
7
function phpunit_dump() {
$cloner = new \Symfony\Component\VarDumper\Cloner\VarCloner();
$dumper = new \Symfony\Component\VarDumper\Dumper\CliDumper(STDERR);
foreach (func_get_args() as $var) {
$dumper->dump($cloner->cloneVar($var));
}
}

How to use it ?

1
2
3
4
5
// Something magic here :D

phpunit_dump($magic_var1, $magic_var2, $magic_of_magic);

// So much magic below, can't understand anymore

Magic