To export data from MySQL by specified query to CSV file, you can use this command

$ mysql -B -u username -p database_name -h dbhost -e "SELECT * FROM table_name;" | sed "s/'/\'/;s/\t/\",\"/g;s/^/\"/;s/$/\"/;s/\n//g"

Tip from : https://stackoverflow.com/a/25427665

To run a shell command in all hosts, you can use the module name raw and provide shell command to module args.

Example:

• To list all CPU model name of hosts

$ ansible all -m raw -a "cat /proc/cpuinfo | grep 'model name'"

Alpine linux becomes the most base image for docker images because it's lightweight and handful package manager apk. Sometimes, you create an image that downloads the binary file but can not execute it. It shows something like this:

/entrypoint.sh: line ***: [your binary]: not found

The problem is your binary built within shared libraries, so it can't run without shared libraries dependencies. To findout which libraries is missing, use this

$ ldd [your binary path]

This is sample result

/usr/local/bin # ldd hugo
        /lib64/ld-linux-x86-64.so.2 (0x7fa852f2a000)
        libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7fa852f2a000)
Error loading shared library libstdc++.so.6: No such file or directory (needed by hugo)
        libdl.so.2 => /lib64/ld-linux-x86-64.so.2 (0x7fa852f2a000)
        libm.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7fa852f2a000)
Error loading shared library libgcc_s.so.1: No such file or directory (needed by hugo)
        libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7fa852f2a000)

So we need to install libstdc++ and libc6-compat before run the binary

RUN apk add --no-cache libstdc++ libc6-compat

Sometimes you want to manipulate the HTML DOM elements inside as a string, then you found a lot of HTML parser or DOM library from the Internet (but it supports only NodeJS). How to do it in a browser ?

The answer is "jQuery is your best friend in browser environment" :)

Then you try this :

const content = jQuery('<p><strong>Hello</strong></p><p>from</p><p><strong>KhanhIceTea</strong></p>');
content.find('p > strong').each(function(i, ele) {
    $(ele).css('color', 'red');
});
console.log(content.html());

What you expected

<p><strong style="color: red;">Hello</strong></p><p>from</p><p><strong style="color: red;">KhanhIceTea</strong></p>

But the console print

<strong>Hello</strong>

SURPRISE !?? JQUERY SUCKS ?

Nope ! The reason is simple, DOM data structure is a tree. And, any tree has a root, right ??? Now you understand the problem ? Then we fix it below

const html_content = '<p><strong>Hello</strong></p><p>from</p><p><strong>KhanhIceTea</strong></p>';
// Wrap all elements into one root element (div)
const content = jQuery('<div />').append(html_content);
// or const content = jQuery('<div>' + html_content + '</div>');
content.find('p > strong').each(function(i, ele) {
    $(ele).css('color', 'red');
});
console.log(content.html());

When you need to debug the docker host of your docker server inside macOS. You can connect to its tty screen by

$ screen ~/Library/Containers/com.docker.docker/Data/vms/0/tty

Then type ENTER to enter the screen and start debugging docker host.

To disconnect the screen, press "Ctrl + A > Ctrl + " and confirm "yes".

If you are in console of developer tool and you want to know what the function does, you can view its source by:

1. call .toSource() of variable or function name in Firefox

Example :

>> function hello(name) { return "Hello " + name; }

>> hello.toSource();
<- "function hello(name) { return \"Hello \" + name; }"

2. click 'show function definition' in Chrome

Cron is a UNIX service that helps user run command periodically. And crontab is simple tool to setup cron job as user, just type in the command crontab -e then setup your cron schedule.

Btw, sometimes you want to run the cronjob every seconds (5, 10 or 20 seconds), but crontab only support every minute. How to achieve your goal without using another tool ?

I got an idea that we can use the sleep command to make it done. So this is solution.

This below is crontab rule that run command every 10 seconds.

* * * * * [command]
* * * * * sleep 10 && [command]
* * * * * sleep 20 && [command]
* * * * * sleep 30 && [command]
* * * * * sleep 40 && [command]
* * * * * sleep 50 && [command]

It's simple, right ? ;)

I often use telnet and netcat to debug my TCP server and client. But these tool only support plain connection, mean every data transfer between server and client is unencrypted and unsafe.

So if you want to achieve the same result through secure connection (SSL or TLS), use this command

$ openssl s_client -host example.com -port 443
$ # or short syntax
$ openssl s_client -connect example.com:443

I made a function named telnets in .bash_profile to make it easier to use

function telnets() {
	openssl s_client -host "$1" -port "$2"
}

then I just type this on bash shell within same syntax of telnet

$ telnets github.com 443

TIP: To hide detail of certificates, add -quiet flag into command

More info, check openssl s_client -h

Today my team met a weird bug on Safari browsers (on all Apple devices), that input losts its focus after you type first character.

First, we thought it's javascript bug so we wasted a lot of time for debugging the behaviour of input. But nothing works !

So we continued searching on Google, then we found this wonderful answer

The root issues caused by one CSS overrided -webkit-user-select to none, so we have to prevent it by add this line to end of CSS file

input, input:before, input:after {
	-webkit-user-select: initial !important;
	-khtml-user-select: initial !important;
	-moz-user-select: initial !important;
	-ms-user-select: initial !important;
	user-select: initial !important;
} 

Hope it helpful for you next time ! :D

Time before, I often meet the situation that I forgot the parameters pass to a function (so I have to searching the API docs to read via Google). This progress can take you huge time if it repeats many times.

So I think one way to debug the parameters without reading API docs, that is pass console.log as a callback function parameter.

Examples :

[3,9,27].forEach(console.log);
// So you will get
// 3 0
// 9 1
// 27 2
// Then you know , first parameter is item, second parameter is indexed key

jQuery.ajax('/data.json').done(console.log).fail(console.error);

Sometimes, you have a interval timer to retrieve new data updates from AJAX requests. But even the response status code of response is 304 (no body), the browser will treat it as 200 response and copy the cached body from browser cache to response body. So it will trigger the re-rederning UI components.

The way we can detect it is via its response headers list.

This example is using fetch API (which is supported in major browsers nowaday)

fetch('https://some.thing/has/data')
.then(function (res) {
	if (res.headers.get('status') == '304 Not Modified') {
		return null;
	}
	return res.json();
})
.then(function (data) {
	if (data == null) return;

	// Render your data below
	renderUI(data);
});

When you get stucked in this error message "Access denied for user 'root@localhost' ...", you search the way to reset the root password on the Internet, but life is Hard ! (No answer makes you feel it's right way, even some do not work)

So to solve this problem, we need to understand MySQL Authentication

Step 1 : Disable MySQL Authentication by skip loading grant-tables on loading MySQL server

Open MySQL server config file, it might be in /etc/mysql/mysql.conf.d/mysqld.cnf. Add this line to section mysql

[mysqld]
skip-grant-tables

DANGER : BE CAREFULL ! AFTER RESETTING SERVER, YOUR MYSQL SERVER ALLOWS ANY CONNECTION FROM ANY USER FROM ANY HOST BY ANY PASSWORD

So safe way is to make sure that you are the only one connect MySQL, by

• change to listening port of the server

[mysqld]
skip-grant-tables
port=6033
bind-address = 127.0.0.1

• disable access through MySQL socket

$ sudo chmod 400 /var/run/mysqld/mysqld.sock

Step 2 : Restart the MySQL server

$ sudo systemctl restart mysql

Step 3 : Connect to mysql server by mysql cli, now you can connect free

$ mysql -h 127.0.0.1 -P 6033

Step 4 : Analyze mysql.user table

mysql> use mysql              
Database changed              
mysql> select Host, User, plugin, password_expired, account_locked from user where User = 'root';                                           
+-----------+------------------+-----------------------+------------------+----------------+
| Host      | User             | plugin                | password_expired | account_locked |
+-----------+------------------+-----------------------+------------------+----------------+
| %         | root             | mysql_native_password | N                | N              |
+-----------+------------------+-----------------------+------------------+----------------+
4 rows in set (0.00 sec)

These fields meaning :

• Host : allowed client host name or IP address
  • 127.0.0.1 : allow local clients connect via TCP
  • localhost : allow local clients connect via local UNIX socket file /var/run/mysqld/mysqld.sock
  • % : any wildcard, allow from all hosts
• User : allowed user name
  • root : allow root user
• plugin :
  • mysql_native_password : use hashing function of MySQL PASSWORD('YOURPASSWORD'), stored in authentication_string field (MySQL 5.7+) or password field (MySQL 5.6 or older)
  • auth_socket : use socket
• password_expired :
  • Y : password is expired
  • N : password is not expired (still working)
• account_locked :
  • Y : account is locked
  • N : account is not locked (still working)

Step 5 : Reset your password

Rewrite your sql command by replacing NEWPASSWORD and WHERE statement to match account we analyze in Step 4

MySQL 5.7+

mysql> update user set plugin = 'mysql_native_password', authentication_string = PASSWORD('NEWPASSWORD'), password_expired = 'N', account_locked = 'N' where Host = '%' and User = 'root';

MySQL 5.6 or older

mysql> update user set plugin = 'mysql_native_password', password = PASSWORD('NEWPASSWORD'), password_expired = 'N', account_locked = 'N' where Host = '%' and User = 'root';

Make sure that we changed 1 row by checking the result log : Query OK, 1 rows affected (0.00 sec)

Step 6 : Flushing privileges

mysql> flush privileges;
mysql> quit;

Step 7 : Rollback all config changes

Update your mysql server config file, make sure to comment out skip-grant-tables

[mysqld]
# skip-grant-tables
port=3306
bind-address = 127.0.0.1

$ sudo systemctl restart mysql

Trying to connect to MySQL server with your new password

$ mysql -u root -h 127.0.0.1 -p

If anything works perfectly, last step is enabling access to socket file

$ sudo chmod 777 /var/run/mysqld/mysqld.sock

HOPE IT HELP ! WE SOLVE PROBLEMS BY UNDERSTANDING IT !

You can overrride the Name Resolution with specific IP address without adding the hostname to /etc/hosts file by using --resolve option.

Syntax :

--resolve <host:port:address>

It will resolve IP address when connect to host on port

Example :

This will connect 127.0.0.1

$ curl --resolve google.com:80:127.0.0.1 "http://google.com/"

But this won't connect 127.0.0.1, because we use 443 port for https

$ curl --resolve google.com:80:127.0.0.1 "https://google.com/"

For cover all ports, use * wildcard

$ curl --resolve google.com:*:127.0.0.1 "https://google.com/"

Curl is great tool to do thing with HTTP in command line interface. Sometimes your want to get extra info from HTTP response and put in a variable. Here is the way :

Using --write-out is magical option help you to write out all info you want, or put it in a variable.

Example :

$ code=$(curl --write-out %{response_code} --silent --output /dev/null https://khanhicetea.com)
$ echo $code # get http response status code
200

$ tracetime=$(curl --write-out "%{time_namelookup} %{time_connect} %{time_appconnect} %{time_pretransfer} %{time_redirect} %{time_starttransfer} %{time_total}" --silent --output /dev/null https://khanhicetea.com)
$ echo $tracetime # Trace all timing of http connection (in seconds)
0.068 0.097 0.370 0.370 0.000 0.720 0.721

Use case :

Below code is cron bash script that checks if http response code equals :

• 502 (Bad Gateway), then restart the backend server (nginx -> apache2)
• not 200 then restart the frontend server (nginx)

#!/bin/bash

code=$(curl --write-out %{response_code} --silent --output /dev/null http://example.com)
[ $code -eq 502 ] && sudo systemctl restart httpd || echo "everything works fine"
[ ! $code -eq 200 ] && sudo systemctl restart nginx || echo "everything works fine"

More info variables, you can check it here

$PWD

Your current working directory, so you don't have to use CWD=$(pwd)

$OLDPWD

Your previous working directory

Note : You can jump directly to it by the command cd -

$SECONDS

The number of seconds the script has been running

You can use it for profiling or limiting timeout

TIME_LIMIT=60
START=$SECONDS
while [ $(($SECONDS - START)) -le "$TIME_LIMIT" ]
do
	## Your work here
done

echo "It takes $SECONDS seconds to get here !"

$RANDOM

Get random integer number, for getting random name or just roll a dice ;)

In previous TIL, I learned about the way to define getter and setter in Javascript

Today, I learned it in Pythonic way ;) (in Python3)

So here is example (the easy way to learn from a code) :

class A:
    def __init__(self,x):
        print ('init', x)
        self.x = x

    @property
    def x(self):
        print ('getter')
        return self.__x

    @x.setter
    def x(self, x):
        print ('setter', x)
        if x < 0:
            self.__x = 0
        elif x % 2 == 0:
            self.__x = x
        else:
            self.__x = x * 2

a = A(10)
print(a.x)
a.x = -1
print(a.x)
a.x = 2
print(a.x)
a.x = 7
print(a.x)

Output :

init 10
setter 10
getter
10
setter -1
getter
0
setter 2
getter
2
setter 7
getter
14

This is so cool feature, and even it has deleter property method, which triggers when we run del object.property :D

In reactive JS framework that they use getter and setter like a core of Reactive, let find out more next articles ;)

SQL databases support 2 main wildcards :

• % : represents zero, one, or multiple characters
• _ : represents a single character

I learned this from learn-python3

Example :

def append_if_multiple_of_five(number, magical_list=[]):
    if number % 5 == 0:
        magical_list.append(number)
    return magical_list

print(append_if_multiple_of_five(100))
print(append_if_multiple_of_five(105))
print(append_if_multiple_of_five(123))
print(append_if_multiple_of_five(123, []))
print(append_if_multiple_of_five(123))

Result :

[100]
[100, 105]
[100, 105]
[]
[100, 105]

So default parameters in Python are shared between function calls if it isn't passed from caller. So be careful because shared mutable object can affect your logic between function calls, where MAGIC was born !

One safe way to achieve the goal, use None as replacement like this

def append_if_multiple_of_five(number, magical_list=None):
    if not magical_list:
        magical_list = []
    if number % 5 == 0:
        magical_list.append(number)
    return magical_list

print(append_if_multiple_of_five(100))
print(append_if_multiple_of_five(105))
print(append_if_multiple_of_five(123))
print(append_if_multiple_of_five(123, []))
print(append_if_multiple_of_five(123))

Result :

[100]
[105]
[]
[]
[]

In Python, sometimes you will see this syntax

with something:
    do_something_else(something)

Then you ask yourself, why I have to use this with syntax ? What runs inside that statement ?

Here is how it works, it's called Context Managers in object

You can define context managers for a class of object to make sure some logic runs correctly without forgeting

Example :

class Animal:
    def __init__(self, name):
        self.name = name
    
    def __enter__(self):
        print("Enter the room !")
    
    def __exit__(self, exc_type, exc_value, traceback):
        print("Exit the room !")

kitty = Animal("Kitty Kat")
with kitty:
    print(kitty.name)

Result

Enter the room !
Kitty Kat
Exit the room !

So, when you start using with keyword on a object, it runs __enter__ method, when everything inside with block is runned, __exit__ will be called !

It's cool feature of Python !

Sometimes you want to use another private key to authorize to remote repository.

Just add an environment variable before the command you wanna run : GIT_SSH_COMMAND='ssh -i [your-private-key]

Example :

$ GIT_SSH_COMMAND='ssh -i ~/keys/key1' git pull

Sometimes you want to put .git data into outside directory (to use another disk partition or to protect your git data). Use --separate-git-dir= option to get that.

Example :

$ git init --separate-git-dir=/var/gitstorage/myproject

Bonus : to protect .git data from other users, use this option --shared within octal value (same to chmod)

Example : this will protect git file from writing by group and reading/writing by others

$ git init --separate-git-dir=/var/gitstorage/myproject --shared=0640

Curl is good lib and tool to simulate HTTP requests. One common usecase is reusing the cookies between 2 or more requests. So you don't have to copied last "Set-Cookie" of previous response then paste it to "Cookie" of next request.

To achieve that, you have to use a cookie jar (sounds fun) to store cookies then use that cookie jar in next request. We have two parameters :

-c [cookie_jar_file] : store response cookies in a file

-b [cookie_jar_file] : getting cookies from cookie jar file then send it in request

Combine them together we can simulate a real browser HTTP requests easily.

Example :

$ curl -c cookies.txt -b cookies.txt -XPOST -d "user=admin&password=hahahehe" http://example.com/login
......
$ curl -c cookies.txt -b cookies.txt -XGET http://example.com/dashboard
Hello admin !

You can encrypt and decrypt the file using openssl command line. Somehow you will need to encrypt your important file with a secret key.

Encrypt

openssl enc -aes-256-cbc -in [input_file] -out [output_file]

Then Enter 2 times your secret key (this should be hard to guess and don't loose it)

Decrypt

openssl enc -aes-256-cbc -d -in [input_file] > [output_file]

Then enter your secret key to decrypt the content !

Notice : should use a 10-char length secret with alpha nums and special characters

Today, I read my junior team-mate code and find this line

$number = rand(2,1000)*rand(2,1000);

This made me remember that's same idea of my own in many years ago. Then I ask myself, is it good to generate a random number from 2 random numbers ?

So the main reason that he wrote this was making probability of same number at same time must be low. We can know that this will return a random number from 4 to 1000000. So why don't we just do this ?

$number = rand(4,1000000);

This function will make probability lower than above function, because above function is commutative (AB = BA). So stay away from commutative function (+, *) when generate a random number.

Need more powerful ? Try Pseudorandom Number Generator (PRNG) !!

You can check the history of a domain (A Record). It's useful in case you forgot the old IP of domain.

Check it here : http://viewdns.info/iphistory/

Example : this is Github A record history http://viewdns.info/iphistory/?domain=github.com

TLDR;

Never ever enable xdebug.remote_autostart in cli

Xdebug is handy extension helps you debug your PHP code. But it slows the performance in cli mode, especially run PHP cli tool like composer or phpunit.

So please disable Xdebug in cli mode or set xdebug.remote_autostart=0 in INI file.

We have to deal with exceptions every moment we touch PHP web development, and so please be carefully with running order of exception catching.

Here is an example

<?php

function a() {
	try {
		throw new Exception('dsads');
	} catch (Exception $e) {
		return 'b';
	} finally {
		echo 'c';
	}
}

echo a();

Then the output is

cb

Than mean even return 'b'; runs, the finally code must be runned before function result passes out.

Sometimes, we need to use temporarily data from another database file. There is simple and fast way to achieve that without transfering data from file X to file Y.

We connect to main database

$ sqlite3 main.sqlite

Then using the ATTACH command to attach another database as an alias in main database

> ATTACH another_db.sqlite as TEMP;

Let listing the tables

> .tables
TEMP.users
TEMP.posts
users
admins
categories

As you can see, we got a list of tables, which are prefixed by defined alias TEMP we used in ATTACH command. So you only need to use these tables as normal tables.

Example of copying data between 2 tables (same structure) :

> INSERT INTO users SELECT * FROM TEMP.users;

You can put common environment variables to the file venv/bin/activate. So everytime we active the virtual env, everything is on the way

# venv/bin/active content

# export your env vars here
export FLASK_APP=hello
export FLASK_ENV=development
export DATABASE=hello.sqlite3
export SECRET_KEY=secret_key_here

Too tired of setting your local domain each time you create new virtual development domain, etc helloworld.test, unit.test point to 127.0.0.1

There is a better way to achieve that by using dnsmasq, then set up a wildcard domains for development. In this case I use .test because .dev has been owned by Google and they strictly use HTTPS in mainly browsers.

Install dnsmasq

$ brew install dnsmasq

Adding .test wildcard to config file

$ echo 'address=/.test/127.0.0.1' > $(brew --prefix)/etc/dnsmasq.conf

Setup dnsmasq as a startup service

$ sudo brew services start dnsmasq

Then add 127.0.0.1 (dnsmasq IP) as first DNS resolver

System Preferences > Network > Wi-Fi > Advanced... > DNS > add 127.0.0.1 > move it to top of the list.

Checking everything is worked by listing all resolvers

$ scutil --dns

Try it out

$ nslookup -type=a something.test
$ ping helloworld.test