As default, docker runs your container as root user (uid=0). Although docker isolates your filesystem to protect docker host, but running processes as root is redundant and increasing attacking surface. Even it can make its written files having root owner, which can mess your docker-host filesystem permission.

This is example that show docker runs as root

1

$ docker run --rm alpine sleep 30

and open another terminal to check this

1
2
3
4

$ ps xau | grep 'sleep'
khanhic+ 15552  0.5  0.4 1264452 49280 pts/1   Sl+  17:37   0:00 docker run --rm alpine:3.9 sleep 30
root     15610  0.6  0.0   1520     4 ?        Ss   17:37   0:00 sleep 30
khanhic+ 15876  0.0  0.0  23076  1024 pts/2    S+   17:37   0:00 grep --color=auto sleep

You can see that the process sleep 30 is running as root with pid = 15610

To control which user docker container runs as, you can use the --user [userid]:[groupid] argument

Example

1

$ docker run --rm --user 1000:1000 alpine sleep 30

Then you will get this result

1
2
3
4

$ ps xau | grep 'sleep'
khanhic+ 16275  2.0  0.4 1411916 50124 pts/1   Sl+  17:41   0:00 docker run --rm --user 1000:1000 alpine:3.9 sleep 30
khanhic+ 16336  1.5  0.0   1520     4 ?        Ss   17:41   0:00 sleep 30
khanhic+ 16403  0.0  0.0  23076   984 pts/2    S+   17:41   0:00 grep --color=auto sleep

TIP : you can set a environment variable by add this line to ~/.bash_profile or ~/.bashrc

1

export DOCKER_UID="$(id -u ${USER}):$(id -g ${USER})"`

then use docker command like docker run --user $DOCKER_UID ....

Alpine linux becomes the most base image for docker images because it’s lightweight and handful package manager apk. Sometimes, you create an image that downloads the binary file but can not execute it. It shows something like this:

1

/entrypoint.sh: line ***: [your binary]: not found

The problem is your binary built within shared libraries, so it can’t run without shared libraries dependencies. To findout which libraries is missing, use this

1

$ ldd [your binary path]

This is sample result

1
2
3
4
5
6
7
8

/usr/local/bin # ldd hugo
        /lib64/ld-linux-x86-64.so.2 (0x7fa852f2a000)
        libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7fa852f2a000)
Error loading shared library libstdc++.so.6: No such file or directory (needed by hugo)
        libdl.so.2 => /lib64/ld-linux-x86-64.so.2 (0x7fa852f2a000)
        libm.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7fa852f2a000)
Error loading shared library libgcc_s.so.1: No such file or directory (needed by hugo)
        libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7fa852f2a000)

So we need to install libstdc++ and libc6-compat before run the binary

1

RUN apk add --no-cache libstdc++ libc6-compat

As an software engineering developer, you know that automated CI testing is one of keys to improve software release life-cycle.

But sometimes reality is not as good as you think, CI testing speed is slow (3-10 minutes / build) and it slows the release cycle speed down. And you try to look into your build logs to find out what causes the problem. Then you got it, it’s mostly the DATABASE service (MySQL, Postgres, MongoDB, …)

I will summarize some stages of your database in a testing build:

• First, it initializes the data, loads config and listens to the connections (takes around 10-45 seconds)
• Second, that you import your testing database into the server (including schemas and initialized data) takes around 20-60 seconds
• Then, on each test case, it needs to clear all data then re-imports fixture data (takes around 30-120 seconds)

So how to make these servers run as fast as possible like some Key-Value databases do? (Redis, Memcached). The main different point is the MEMORY! What if we put all data inside memory??

All of we know that RAM speed with 150 times lower latency is technically better than SSD and HDD speed. And as a matter of fact, Linux is a good OS that supports a lot of filesystems, specially tmpfs, which you can mount files into your RAM memory.

However, nothing is perfect and this is not an exception. Actually, it is not a good option for persistent data which is not necessary for testing database. What it really needs is speed only, so it fits in.

That’s my idea, now I will try to test it on my CI environment (I use DroneCI using Docker). In new version 0.8+ of DroneCI, they support us to run docker containers within tmpfs mount.

So I just add this line into my drone config

1
2
3
4
5
6
7
8
9

services:
  testdatabase:
    thumbnail: mysql:5.7
    # Add this 2 lines below to boost your database container
    tmpfs:
      - /var/lib/mysql
    environment:
      - MYSQL_DATABASE=testdb
      - MYSQL_ROOT_PASSWORD=passwd

Result:

• MySQL service initializes in 3 seconds instead of 25 seconds
• Import testing database using mysql client takes below 1 second instead of 17 seconds
• My test cases run 20-30% faster (I have few testcases using database)

So, worth a shot !!

Ref:

• Cover photo from dotnetvibes.com

When you need to debug the docker host of your docker server inside macOS. You can connect to its tty screen by

1

$ screen ~/Library/Containers/com.docker.docker/Data/vms/0/tty

Then type ENTER to enter the screen and start debugging docker host.

To disconnect the screen, press “Ctrl + A > Ctrl + " and confirm “yes”.

Docker is great tool to build a pull-n-run application. But sometimes, your image will be large if you build image from a big base image which has heavy compliling toolbox.

Ex:

One-stage build

1
2
3
4
5

FROM golang:1.9.2
WORKDIR /go/src/github.com/khanhicetea/test/
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build .
ENTRYPOINT ["/go/src/github.com/khanhicetea/test/test"]

Multi-stage builds

1
2
3
4
5
6
7
8

FROM golang:1.9.2
WORKDIR /go/src/github.com/khanhicetea/test/
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build .

FROM scratch
COPY --from=0 /go/src/github.com/khanhicetea/test/test .
ENTRYPOINT ["/test"]

So final image will only contains /test excutable file. COPY --from=0 means COPY from build has index 0 in Dockerfile.

Original MySQL docker image uses a script to generate ssl certificates for service. Sometime we don’t really need it (connect via a docker network link or need a fast enough database service to build a automated test).

We can reduce init time by removing the script from original Docker image

1
2
3
4

FROM mysql:5.7

# Remove mysql_ssl_rsa_setup to ignore setup SSL certs
RUN rm -f /usr/bin/mysql_ssl_rsa_setup

FAST as a FEATURE !!! 🚀

Introduction

Docker is great tool to management linux containers. It brings DevOps to next level, from development to production environment. And of course, before deploy anything to production, software should be tested carefully and automatically.

That’s why Drone, a new lightweight CI server built-on top Go lang and Docker, will help us to resolve the testing problems in simple and fast way.

Setup

This guide will assume you already have Docker and Docker Compose tool. And of course, root permission ;)

Step 1 : Clone my example docker-compose here : https://github.com/khanhicetea/drone-ci

1
2
3

$ git clone https://github.com/khanhicetea/drone-ci
$ cd drone-ci
$ cp .env.example .env

Step 2 : Update your setting in .env file

Step 3 : Run drone via docker-compose

1
2

$ source .env
$ sudo docker-compose up -d

Step 4 : Go to your Drone url (remember use https url), then authorize with Github provider.

Usage

In example repo, I created a sample .drone.sample.yml file so you can follow the structure to create own file.

I will explain some basics here

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

clone:
  git:
    thumbnail: plugins/git
    depth: 5

pipeline:
  phpunit:
    thumbnail: php:7
    commands:
      - /bin/sh conflict_detector.sh
      - /bin/sh phplinter.sh app lib test
      - composer install -q --prefer-dist
      - test/db/import testdatabase root passwd testdb test.sql
      - php -d memory_limit=256M vendor/bin/phpunit --no-coverage --colors=never
  
  notify:
    thumbnail: plugins/slack
    webhook: [your_slack_webhook_url]
    channel: deployment
    username: DroneCI
    when:
      status: success
  
  notify-bug:
    thumbnail: plugins/slack
    webhook: [your_slack_webhook_url]
    channel: bugs
    username: DroneCI
    when:
      status: failure
      branch: production

services:
  testdatabase:
    thumbnail: mysql:5.7
    detach: true
    environment:
      - MYSQL_DATABASE=testdb
      - MYSQL_ROOT_PASSWORD=passwd

This file consists 3 sections :

• clone : To clone the source code and prepare for pipeline step. This section will be run first
• services : Declare your docker services (databases, ip server) which source code connect to. This section will be run at sametime with pipeline (after clone)
• pipeline : Testing pipe, where you put testing logic here.

In this pipeline, I made a example PHP testing through these steps :

1. Check conflicts in code (grep for >>>> HEAD string)
2. Run PHP linter in application codes
3. Run Composer to install all dependencies
4. Import testing database to mysql services (using testdatabase hostname to connect service)
5. Run testing script via phpunit tool

Then, notify testing result via Slack channel ! ;)

A picture is worth a thousand words

Lets automate all the things !