How to reload gracefully supervisor program

Problem

We have some managed worker-programs by supervisor. While deploying new codebase of worker, we have to restart them to update new logic.

But it’s creepy when supervisor kills all processes of programs and starts again.

WHAT IF IT KILLS A WORKING WORKER ?

Of course, the working task will be fucked up. Database transaction, database migration, sending email and other scheduled tasks will be crashed in middle. Some tasks can’t not be reserved, rollbacked or re-runned.

And your customer can’t trust in you twice.

Generic Solution

In movie, people want to do last thing (say last words, have last kiss or even enjoy last song) then kill themself. So computer programs should have that favour, let them finishes their tasks and kill themself instead of being shooted by a UNIX signal.

Keep calm

Mutex lock file mechanism

Get back to think, the main roles of supervisor are monitoring and controlling a number of processes on UNIX-like OS. Easier explaination : supervisor tracks processes and tries to re-init them if they were crashed in somehow.

So if you want to reload a program, just let it kills itself. Then supervisor do the remain job (re-init processes).

Pseudo implementation

Worker code

1
2
3
4
5
6
7
8
9
10
11
12
function doSomeJob() {
// Create a lock file to notify it's doing task so don't shoot it
touch_file('/tmp/somejob.lock')
// Doing task
do_it_here()
// Checking lockfile
if (is_file_exists('/tmp/somejob.lock')) {
remove_file('/tmp/somejob.lock')
} else {
kill_myself()
}
}

Deploying code

1
[[ -e /tmp/somejob.lock ]] && rm /tmp/somejob.lock || supervisor restart program1

PHP worker and Ansible

Worker code

1
2
3
4
5
6
7
8
9
10
11
function doSomeTask() {
touch('/tmp/somejob.lock');
// Doing task
do_it_here();
// Checking lockfile
if (file_exists('/tmp/somejob.lock')) {
unlink('/tmp/somejob.lock');
} else {
exit();
}
}

Ansible tasks

1
2
3
4
5
6
7
8
9
- name: Check if woker lock file exists
stat: path=/tmp/somejob.lock
register: check_worker_lock

- name: Remove if worker lock file exists
file:
path: /tmp/somejob.lock
state: absent
when: check_worker_lock.stat.exists

Ansible handlers

1
2
3
4
5
- name: reload supervisor
supervisorctl:
name: worker
state: restarted
when: check_worker_lock.stat.exists == False

Short explanation by image

bazooka fail kid


Ref:

  • Meme from Google photo search

#TIL : Persistent connection to MySQL

When a PHP process connects to MySQL server, the connection can be persistent if your PHP config has mysql.allow_persistent or mysqli.allow_persistent. (PDO has the attribute ATTR_PERSISTENT)

1
$dbh = new PDO('DSN', 'KhanhDepZai', 'QuenMatKhauCMNR', [PDO::ATTR_PERSISTENT => TRUE]);

Object destruction

PHP destruct an object automatically when an object lost all its references.

Example code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
<?php

$x = null;

function klog($x) {
echo $x . ' => ';
}

class A {
private $k;
function __construct($k) {
$this->k = $k;
}

function b() {
klog('[b]');
}

function __destruct() {
klog("[{$this->k} has been killed]");
}
}

function c($k) {
return new A($k);
}

function d() {
c('d')->b();
}

function e() {
global $x;
$x = c('e');
$x->b();
klog('[e]');
}

function f() {
klog('[f]');
}

d();
e();
f();

Result:

1
[b] => [d has been killed] => [b] => [e] => [f] => [e has been killed] =>

Reducing PDO persistent connections in PHP long-run process (connect to multiples databases)

Instead of using a service object, we should use a factory design pattern for each job (each connection). PHP will close MySQL connection because it destructs object PDO. Then we can reduce the number of connections to MySQL at a same time.

I learned this case when implement a web-consumer (long-run process) to run database migration for multiples databases.

Before fixing this, our MySQL server had been crashed because of a huge opened connections.

Now, everything works like a charm !

Bring it on

#TIL : Using VarDumper in PHPUnit

The trick is writing the output to STDERR stream, I wrote a helper function below

1
2
3
4
5
6
7
function phpunit_dump() {
$cloner = new \Symfony\Component\VarDumper\Cloner\VarCloner();
$dumper = new \Symfony\Component\VarDumper\Dumper\CliDumper(STDERR);
foreach (func_get_args() as $var) {
$dumper->dump($cloner->cloneVar($var));
}
}

How to use it ?

1
2
3
4
5
// Something magic here :D

phpunit_dump($magic_var1, $magic_var2, $magic_of_magic);

// So much magic below, can't understand anymore

Magic

Sử dụng Frontend Boilerplate cho việc cắt HTML Layout

Giới thiệu

Khi cắt HTML Layout từ PSD, bạn sẽ đối mặt với việc bên Design hoặc khách hàng thường xuyên yêu cầu thay đổi giao diện, thiết kế. Điều này gây ra không ít phiền nhiễu và tốn thời gian nếu cứ cập nhật hàng tá giao diện nếu sự thay đổi ảnh hưởng đến những phần layout chính (header, footer, sidebar).

Sử dụng bộ khung sườn (boiler-plate) này, sẽ giúp bạn hạn chế thao tác lặp đi, lặp lại. Lẫn hỗ trợ cho bên backend đổ dữ liệu vào sau này (hoặc chính bạn là người đổ dữ liệu).

Yêu cầu

  • PHP 5.4+
  • Composer

Cài đặt

  1. Clone hoặc download repo frontend-boilerplate tại đây : https://github.com/khanhicetea/frontend-boilerplate
1
git clone https://github.com/khanhicetea/frontend-boilerplate.git
  1. Cài đặt các packages cần thiết (PHP Twig)
1
2
cd frontend-boilerplate
composer install

Khởi động

Chạy câu lệnh sau để chạy một server localhost (port default là 8080)

1
php -S 127.0.0.1:8080 index.php

Quy ước phát triển

  • Cú pháp phát triển template : http://twig.sensiolabs.org/doc/templates.html
  • Mỗi project sẽ có 1 folder riêng, trong đó có 2 folder để bạn phát triển giao diện :
    • folder templates : template html .
    • folder assets : các file css, js & hình ảnh

Khi cần duyệt layout nào bạn vào trình duyệt gõ http://localhost:8080/[tên-file].html

Ví dụ

  • Tạo folder cho project mới test
  • Chạy lệnh sau để chạy server : php -S localhost:8080 -t test index.php, với test là tên folder của project mới tạo.

File test/templates/layout.html

1
2
3
4
5
6
7
8
9
10
11
12
13
14
<!DOCTYPE html>
<html>
<head>
<title>Hello World</title>
{% include 'blocks/header.html' %}

{% block header_scripts %}
{% endblock %}
</head>
<body>
{% block main_content %}
{% endblock %}
</body>
</html>

File test/templates/blocks/header.html

1
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.2.4/jquery.min.js"></script>

File test/assets/css/style.css

1
2
3
4
5
6
7
body {
text-align: center
}

h1 {
font-size: 100px
}

File test/test.html

1
2
3
4
5
6
7
8
9
10
11
12
13
14
{% extends 'layout.html' %}

{% block header_scripts %}
<link rel="stylesheet" type="text/css" href="/assets/css/style.css">
<script type="text/javascript">
$(document).ready(function() {
$('h1').css('color', 'blue');
});
</script>
{% endblock %}

{% block main_content %}
<h1>This is test !</h1>
{% endblock %}

Vào trình duyệt và gõ http://localhost:8080/test.html

Xuất ra HTML để gửi demo

Khi bạn cần xuất ra giao diện để gửi cho khách hàng hoặc bên duyệt layout, chỉ cần chạy câu lệnh sau :

1
php build.php test r
  • test là tên project cần export
  • r là chế độ thay đổi link của các assets từ absolute sang relative

Tất cả layout + assets sẽ được xuất ra folder dist trong folder project. Nén folder này là có thể gửi đi bất ký đâu.

Happy Coding !!! ¯_(ツ)_/¯

Set up PHP 7 development environment

Setup requirements

Install VirtualBox

Install Vagrant

Verify all requirements

Open the terminal and try this command to verify Virtualbox is ready.

1
$ Virtualbox -h

Open the terminal and try this command to verify Vagrant is ready.

1
$ vagrant --version

Pull PHP 7 box source code

Clone this repository from Github :

1
$ git clone https://github.com/khanhicetea/phpbox

Provision the box

1
2
$ cd phpbox
$ vagrant up

Note : First provision will take a long time to build the box (about 30-45 minutes). But next time you run the box, it only takes 2 mins.

Development guideline

SSH to the box

Run this command in phpbox folder

1
$ vagrant ssh

Point your virtualhosts to Box IP

Your PHPBox IP address is 192.168.67.101, you can change in Vagrantfile

Open /etc/hosts file in your PC to map virtualhosts IP to PHPBox

1
$ sudo vi /etc/hosts

Each line in this file will have format (IP HOSTNAME). Eg:

1
2
192.168.67.101 test.dev
192.168.67.101 php7iscool.dev

Folder structure

PHPBox Structure

  • conf : Configuration
    • modules.ini : PHP module configuration
    • nginx_default : Nginx default host
    • php-fpm.conf : PHP7-FPM configuration
    • vhosts_apache.conf : Apache virtualhosts (if you use Apache)
    • vhosts_nginx : Nginx virtualhosts
    • www.conf : PHP7-FPM worker configuration
  • scripts : provision scripts
  • www : Web root
    • default : document root of default host 192.168.67.101
    • test : document root of test host test.dev
  • .gitignore : gitignore file
  • Vagrantfile : Vagrantfile of PHPBox

Create a new virtualhost

  • Duplicate a server section in vhosts_nginx or VirtualHost in vhosts_apache.conf
  • Edit the server name and document root
  • Pointing servername to PHPBox IP via hosts file
  • Restart web server sudo service nginx restart or sudo service apache2 restart

Utils

Composer, Git, Docker is ready

This box included helpful tools like composer, git, docker, redis-server, htop, vim, screen and zsh.

PHPMyAdmin or Adminer

  • Adminer (default) : http://192.168.67.101/adminer
  • PHPMyAdmin : http://192.168.67.101/phpmyadmin

Services

  • MySQL user : root / passwd
  • Redis : localhost:6379

Installation config

You can open the scripts/provision/setup.sh to modify some variables to modify some packages.

Ref

#TIL : F-cking stupid limit of input vars

Today, I tried to debug many hours to find out why my POST request missing some data (specify _token hidden field). :disappointed:

I tried to config NGINX and PHPFPM max_post_size, client_max_body_size but they still gone. After 2-3 hours searching on Google, I found the link from PHP.net,
it has a config value about limiting max input vars (default = 1000), so it causes the problem about missing data.

So I changed max_input_vars = 9999 in my php.ini and everything works like a charm. :smiley:

At least, I had a luck cos it doesn’t run my POST request when missing the CSRF token :grin: My data is save !!!

PHP 5.6 vs PHP 7RC8 - Benchmarking using Docker

New version of PHP, PHP7 has been released yesterday after 11 years of PHP5. I am very excited with it, so I made a benchmark to compare the performance of 2 versions.

The fastest way to test out PHP multi-versions is using Docker image. You can get it from : https://hub.docker.com/_/php/

Let’s start pull PHP images

1
2
docker pull php:5.6
docker pull php:7

Change directory to the web root folder of PHP project (I used my micro-framework, Sifoni to test). And add this file server.php to web folder with content

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
<?php

error_reporting(0);

$filename = __DIR__.preg_replace('#(\?.*)$#', '', $_SERVER['REQUEST_URI']);
if (php_sapi_name() === 'cli-server' && is_file($filename)) {
return false;
}

define('ROOT_PATH', dirname(dirname(__FILE__)));

$autoloader = require_once (ROOT_PATH . DIRECTORY_SEPARATOR . 'vendor' . DIRECTORY_SEPARATOR . 'autoload.php');

\Sifoni\Engine::getInstance()->init()->bootstrap(array(
'path.root' => ROOT_PATH,
'autoloader' => $autoloader,
))->start()->run();

Next, run this command to run with PHP 5.6

1
docker run -it -p 8080:8080 -v `pwd`:/code php:5.6 php -S 0.0.0.0:8080 -t /code/web /code/web/server.php

Open http://localhost:8080/ to see the memory and processing time of PHP5.6.

Continue with PHP7, exit the PHP 5.6 session and run this command

1
docker run -it -p 8080:8080 -v `pwd`:/code php:7 php -S 0.0.0.0:8080 -t /code/web /code/web/server.php

Open http://localhost:8080/ to see the memory and processing time PHP7.

This is my result :

PHP 5.6 benchmarking result

PHP PHP 7 RC8 benchmarking result

Amazing… PHP 7 is winner in Saving Memory and Processesing Time (about 1.4x times than PHP 5.6).

At the moment I tested, official docker image of PHP is PHP7RC8. Hope image maintainer updates it soon :)