Yo! Back to my #TIL

Today, I find out a way to insert fastly current date or current time to a cell. It makes you faster 3-5 seconds and reduce confusing about what the time is?

• To insert current date : press Ctrl + :
• To insert current time : press Ctrl + Shift + :

Excel (Spreadsheets) never dies !

TLDR;

Use JSON.parse("[your data in json string]") if your data is big (>10KB for instance)

Short Explaination :

JSON.parse parses a string to object, so it has many strict requirements than Javascript parses the source code (more syntax, more context)

Long Explaination :

https://www.youtube.com/watch?v=ff4fgQxPaO0

NGINX is well known as a simple and good web server right now, but not everyone knows that NGINX can act like a TCP-UDP loadbalancer. So you won't need to install HAProxy when you need a LoadBalancer.

This feature is released on NGINX 1.9+. So you can setup it by this rule

stream {
	upstream backend1 {
		server s1.backend1.com:12345;
		server s2.backend1.com:12345;
	}
	server {
		listen 54321;
		proxy_pass backend1;
	}
	upstream backend2 {
		server s1.backend2.com:7777;
		server s2.backend2.com:7777;
		server s3.backend2.com:7777;
	}
	server {
		listen 8888 udp; # add udp keyword if you want UDP server
		proxy_pass backend2;
	}
}

To learn more, click here : https://docs.nginx.com/nginx/admin-guide/load-balancer/tcp-udp-load-balancer/

Sometimes, you ran into issues that apt update package list so slow or even can not connect to the destination server.

You can change the default list into new one near your country. Get example : if you live in ASIA, choose the Singapore or Japan mirror instead of main mirror. you just change the "archive.ubuntu.com" and "security.ubuntu.com" to "[country_code].archive.ubuntu.com" in file list /etc/apt/sources.list

Then run sudo apt update to test your new mirror :)

TIP : you can generate the new source list file based on this online tool : https://repogen.simplylinux.ch/generate.php

As default, docker runs your container as root user (uid=0). Although docker isolates your filesystem to protect docker host, but running processes as root is redundant and increasing attacking surface. Even it can make its written files having root owner, which can mess your docker-host filesystem permission.

This is example that show docker runs as root

$ docker run --rm alpine sleep 30

and open another terminal to check this

$ ps xau | grep 'sleep'
khanhic+ 15552  0.5  0.4 1264452 49280 pts/1   Sl+  17:37   0:00 docker run --rm alpine:3.9 sleep 30
root     15610  0.6  0.0   1520     4 ?        Ss   17:37   0:00 sleep 30
khanhic+ 15876  0.0  0.0  23076  1024 pts/2    S+   17:37   0:00 grep --color=auto sleep

You can see that the process sleep 30 is running as root with pid = 15610

To control which user docker container runs as, you can use the --user [userid]:[groupid] argument

Example

$ docker run --rm --user 1000:1000 alpine sleep 30

Then you will get this result

$ ps xau | grep 'sleep'
khanhic+ 16275  2.0  0.4 1411916 50124 pts/1   Sl+  17:41   0:00 docker run --rm --user 1000:1000 alpine:3.9 sleep 30
khanhic+ 16336  1.5  0.0   1520     4 ?        Ss   17:41   0:00 sleep 30
khanhic+ 16403  0.0  0.0  23076   984 pts/2    S+   17:41   0:00 grep --color=auto sleep

TIP : you can set a environment variable by add this line to ~/.bash_profile or ~/.bashrc

export DOCKER_UID="$(id -u ${USER}):$(id -g ${USER})"`

then use docker command like docker run --user $DOCKER_UID ....

When you use a load balancer stay in front of your app, and use SSL Passthrough mode. You will never get real IP of client, because Load balancer works like a TCP load balancer, which means it can not add extra HTTP headers into encrypted traffic from client when it doesn't handle SSL termination.

So if you use 1 domain or wildcard subdomains, it's better if you use SSL Termination mode.

To export data from MySQL by specified query to CSV file, you can use this command

$ mysql -B -u username -p database_name -h dbhost -e "SELECT * FROM table_name;" | sed "s/'/\'/;s/\t/\",\"/g;s/^/\"/;s/$/\"/;s/\n//g"

Tip from : https://stackoverflow.com/a/25427665

To run a shell command in all hosts, you can use the module name raw and provide shell command to module args.

Example:

• To list all CPU model name of hosts

$ ansible all -m raw -a "cat /proc/cpuinfo | grep 'model name'"

Alpine linux becomes the most base image for docker images because it's lightweight and handful package manager apk. Sometimes, you create an image that downloads the binary file but can not execute it. It shows something like this:

/entrypoint.sh: line ***: [your binary]: not found

The problem is your binary built within shared libraries, so it can't run without shared libraries dependencies. To findout which libraries is missing, use this

$ ldd [your binary path]

This is sample result

/usr/local/bin # ldd hugo
        /lib64/ld-linux-x86-64.so.2 (0x7fa852f2a000)
        libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7fa852f2a000)
Error loading shared library libstdc++.so.6: No such file or directory (needed by hugo)
        libdl.so.2 => /lib64/ld-linux-x86-64.so.2 (0x7fa852f2a000)
        libm.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7fa852f2a000)
Error loading shared library libgcc_s.so.1: No such file or directory (needed by hugo)
        libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7fa852f2a000)

So we need to install libstdc++ and libc6-compat before run the binary

RUN apk add --no-cache libstdc++ libc6-compat

Sometimes you want to manipulate the HTML DOM elements inside as a string, then you found a lot of HTML parser or DOM library from the Internet (but it supports only NodeJS). How to do it in a browser ?

The answer is "jQuery is your best friend in browser environment" :)

Then you try this :

const content = jQuery('<p><strong>Hello</strong></p><p>from</p><p><strong>KhanhIceTea</strong></p>');
content.find('p > strong').each(function(i, ele) {
    $(ele).css('color', 'red');
});
console.log(content.html());

What you expected

<p><strong style="color: red;">Hello</strong></p><p>from</p><p><strong style="color: red;">KhanhIceTea</strong></p>

But the console print

<strong>Hello</strong>

SURPRISE !?? JQUERY SUCKS ?

Nope ! The reason is simple, DOM data structure is a tree. And, any tree has a root, right ??? Now you understand the problem ? Then we fix it below

const html_content = '<p><strong>Hello</strong></p><p>from</p><p><strong>KhanhIceTea</strong></p>';
// Wrap all elements into one root element (div)
const content = jQuery('<div />').append(html_content);
// or const content = jQuery('<div>' + html_content + '</div>');
content.find('p > strong').each(function(i, ele) {
    $(ele).css('color', 'red');
});
console.log(content.html());

When you need to debug the docker host of your docker server inside macOS. You can connect to its tty screen by

$ screen ~/Library/Containers/com.docker.docker/Data/vms/0/tty

Then type ENTER to enter the screen and start debugging docker host.

To disconnect the screen, press "Ctrl + A > Ctrl + " and confirm "yes".

If you are in console of developer tool and you want to know what the function does, you can view its source by:

1. call .toSource() of variable or function name in Firefox

Example :

>> function hello(name) { return "Hello " + name; }

>> hello.toSource();
<- "function hello(name) { return \"Hello \" + name; }"

2. click 'show function definition' in Chrome

Cron is a UNIX service that helps user run command periodically. And crontab is simple tool to setup cron job as user, just type in the command crontab -e then setup your cron schedule.

Btw, sometimes you want to run the cronjob every seconds (5, 10 or 20 seconds), but crontab only support every minute. How to achieve your goal without using another tool ?

I got an idea that we can use the sleep command to make it done. So this is solution.

This below is crontab rule that run command every 10 seconds.

* * * * * [command]
* * * * * sleep 10 && [command]
* * * * * sleep 20 && [command]
* * * * * sleep 30 && [command]
* * * * * sleep 40 && [command]
* * * * * sleep 50 && [command]

It's simple, right ? ;)

I often use telnet and netcat to debug my TCP server and client. But these tool only support plain connection, mean every data transfer between server and client is unencrypted and unsafe.

So if you want to achieve the same result through secure connection (SSL or TLS), use this command

$ openssl s_client -host example.com -port 443
$ # or short syntax
$ openssl s_client -connect example.com:443

I made a function named telnets in .bash_profile to make it easier to use

function telnets() {
	openssl s_client -host "$1" -port "$2"
}

then I just type this on bash shell within same syntax of telnet

$ telnets github.com 443

TIP: To hide detail of certificates, add -quiet flag into command

More info, check openssl s_client -h

Today my team met a weird bug on Safari browsers (on all Apple devices), that input losts its focus after you type first character.

First, we thought it's javascript bug so we wasted a lot of time for debugging the behaviour of input. But nothing works !

So we continued searching on Google, then we found this wonderful answer

The root issues caused by one CSS overrided -webkit-user-select to none, so we have to prevent it by add this line to end of CSS file

input, input:before, input:after {
	-webkit-user-select: initial !important;
	-khtml-user-select: initial !important;
	-moz-user-select: initial !important;
	-ms-user-select: initial !important;
	user-select: initial !important;
} 

Hope it helpful for you next time ! :D

Time before, I often meet the situation that I forgot the parameters pass to a function (so I have to searching the API docs to read via Google). This progress can take you huge time if it repeats many times.

So I think one way to debug the parameters without reading API docs, that is pass console.log as a callback function parameter.

Examples :

[3,9,27].forEach(console.log);
// So you will get
// 3 0
// 9 1
// 27 2
// Then you know , first parameter is item, second parameter is indexed key

jQuery.ajax('/data.json').done(console.log).fail(console.error);

Sometimes, you have a interval timer to retrieve new data updates from AJAX requests. But even the response status code of response is 304 (no body), the browser will treat it as 200 response and copy the cached body from browser cache to response body. So it will trigger the re-rederning UI components.

The way we can detect it is via its response headers list.

This example is using fetch API (which is supported in major browsers nowaday)

fetch('https://some.thing/has/data')
.then(function (res) {
	if (res.headers.get('status') == '304 Not Modified') {
		return null;
	}
	return res.json();
})
.then(function (data) {
	if (data == null) return;

	// Render your data below
	renderUI(data);
});

When you get stucked in this error message "Access denied for user 'root@localhost' ...", you search the way to reset the root password on the Internet, but life is Hard ! (No answer makes you feel it's right way, even some do not work)

So to solve this problem, we need to understand MySQL Authentication

Step 1 : Disable MySQL Authentication by skip loading grant-tables on loading MySQL server

Open MySQL server config file, it might be in /etc/mysql/mysql.conf.d/mysqld.cnf. Add this line to section mysql

[mysqld]
skip-grant-tables

DANGER : BE CAREFULL ! AFTER RESETTING SERVER, YOUR MYSQL SERVER ALLOWS ANY CONNECTION FROM ANY USER FROM ANY HOST BY ANY PASSWORD

So safe way is to make sure that you are the only one connect MySQL, by

• change to listening port of the server

[mysqld]
skip-grant-tables
port=6033
bind-address = 127.0.0.1

• disable access through MySQL socket

$ sudo chmod 400 /var/run/mysqld/mysqld.sock

Step 2 : Restart the MySQL server

$ sudo systemctl restart mysql

Step 3 : Connect to mysql server by mysql cli, now you can connect free

$ mysql -h 127.0.0.1 -P 6033

Step 4 : Analyze mysql.user table

mysql> use mysql              
Database changed              
mysql> select Host, User, plugin, password_expired, account_locked from user where User = 'root';                                           
+-----------+------------------+-----------------------+------------------+----------------+
| Host      | User             | plugin                | password_expired | account_locked |
+-----------+------------------+-----------------------+------------------+----------------+
| %         | root             | mysql_native_password | N                | N              |
+-----------+------------------+-----------------------+------------------+----------------+
4 rows in set (0.00 sec)

These fields meaning :

• Host : allowed client host name or IP address
  • 127.0.0.1 : allow local clients connect via TCP
  • localhost : allow local clients connect via local UNIX socket file /var/run/mysqld/mysqld.sock
  • % : any wildcard, allow from all hosts
• User : allowed user name
  • root : allow root user
• plugin :
  • mysql_native_password : use hashing function of MySQL PASSWORD('YOURPASSWORD'), stored in authentication_string field (MySQL 5.7+) or password field (MySQL 5.6 or older)
  • auth_socket : use socket
• password_expired :
  • Y : password is expired
  • N : password is not expired (still working)
• account_locked :
  • Y : account is locked
  • N : account is not locked (still working)

Step 5 : Reset your password

Rewrite your sql command by replacing NEWPASSWORD and WHERE statement to match account we analyze in Step 4

MySQL 5.7+

mysql> update user set plugin = 'mysql_native_password', authentication_string = PASSWORD('NEWPASSWORD'), password_expired = 'N', account_locked = 'N' where Host = '%' and User = 'root';

MySQL 5.6 or older

mysql> update user set plugin = 'mysql_native_password', password = PASSWORD('NEWPASSWORD'), password_expired = 'N', account_locked = 'N' where Host = '%' and User = 'root';

Make sure that we changed 1 row by checking the result log : Query OK, 1 rows affected (0.00 sec)

Step 6 : Flushing privileges

mysql> flush privileges;
mysql> quit;

Step 7 : Rollback all config changes

Update your mysql server config file, make sure to comment out skip-grant-tables

[mysqld]
# skip-grant-tables
port=3306
bind-address = 127.0.0.1

$ sudo systemctl restart mysql

Trying to connect to MySQL server with your new password

$ mysql -u root -h 127.0.0.1 -p

If anything works perfectly, last step is enabling access to socket file

$ sudo chmod 777 /var/run/mysqld/mysqld.sock

HOPE IT HELP ! WE SOLVE PROBLEMS BY UNDERSTANDING IT !

You can overrride the Name Resolution with specific IP address without adding the hostname to /etc/hosts file by using --resolve option.

Syntax :

--resolve <host:port:address>

It will resolve IP address when connect to host on port

Example :

This will connect 127.0.0.1

$ curl --resolve google.com:80:127.0.0.1 "http://google.com/"

But this won't connect 127.0.0.1, because we use 443 port for https

$ curl --resolve google.com:80:127.0.0.1 "https://google.com/"

For cover all ports, use * wildcard

$ curl --resolve google.com:*:127.0.0.1 "https://google.com/"

Curl is great tool to do thing with HTTP in command line interface. Sometimes your want to get extra info from HTTP response and put in a variable. Here is the way :

Using --write-out is magical option help you to write out all info you want, or put it in a variable.

Example :

$ code=$(curl --write-out %{response_code} --silent --output /dev/null https://khanhicetea.com)
$ echo $code # get http response status code
200

$ tracetime=$(curl --write-out "%{time_namelookup} %{time_connect} %{time_appconnect} %{time_pretransfer} %{time_redirect} %{time_starttransfer} %{time_total}" --silent --output /dev/null https://khanhicetea.com)
$ echo $tracetime # Trace all timing of http connection (in seconds)
0.068 0.097 0.370 0.370 0.000 0.720 0.721

Use case :

Below code is cron bash script that checks if http response code equals :

• 502 (Bad Gateway), then restart the backend server (nginx -> apache2)
• not 200 then restart the frontend server (nginx)

#!/bin/bash

code=$(curl --write-out %{response_code} --silent --output /dev/null http://example.com)
[ $code -eq 502 ] && sudo systemctl restart httpd || echo "everything works fine"
[ ! $code -eq 200 ] && sudo systemctl restart nginx || echo "everything works fine"

More info variables, you can check it here

$PWD

Your current working directory, so you don't have to use CWD=$(pwd)

$OLDPWD

Your previous working directory

Note : You can jump directly to it by the command cd -

$SECONDS

The number of seconds the script has been running

You can use it for profiling or limiting timeout

TIME_LIMIT=60
START=$SECONDS
while [ $(($SECONDS - START)) -le "$TIME_LIMIT" ]
do
	## Your work here
done

echo "It takes $SECONDS seconds to get here !"

$RANDOM

Get random integer number, for getting random name or just roll a dice ;)

In previous TIL, I learned about the way to define getter and setter in Javascript

Today, I learned it in Pythonic way ;) (in Python3)

So here is example (the easy way to learn from a code) :

class A:
    def __init__(self,x):
        print ('init', x)
        self.x = x

    @property
    def x(self):
        print ('getter')
        return self.__x

    @x.setter
    def x(self, x):
        print ('setter', x)
        if x < 0:
            self.__x = 0
        elif x % 2 == 0:
            self.__x = x
        else:
            self.__x = x * 2

a = A(10)
print(a.x)
a.x = -1
print(a.x)
a.x = 2
print(a.x)
a.x = 7
print(a.x)

Output :

init 10
setter 10
getter
10
setter -1
getter
0
setter 2
getter
2
setter 7
getter
14

This is so cool feature, and even it has deleter property method, which triggers when we run del object.property :D

In reactive JS framework that they use getter and setter like a core of Reactive, let find out more next articles ;)

SQL databases support 2 main wildcards :

• % : represents zero, one, or multiple characters
• _ : represents a single character

I learned this from learn-python3

Example :

def append_if_multiple_of_five(number, magical_list=[]):
    if number % 5 == 0:
        magical_list.append(number)
    return magical_list

print(append_if_multiple_of_five(100))
print(append_if_multiple_of_five(105))
print(append_if_multiple_of_five(123))
print(append_if_multiple_of_five(123, []))
print(append_if_multiple_of_five(123))

Result :

[100]
[100, 105]
[100, 105]
[]
[100, 105]

So default parameters in Python are shared between function calls if it isn't passed from caller. So be careful because shared mutable object can affect your logic between function calls, where MAGIC was born !

One safe way to achieve the goal, use None as replacement like this

def append_if_multiple_of_five(number, magical_list=None):
    if not magical_list:
        magical_list = []
    if number % 5 == 0:
        magical_list.append(number)
    return magical_list

print(append_if_multiple_of_five(100))
print(append_if_multiple_of_five(105))
print(append_if_multiple_of_five(123))
print(append_if_multiple_of_five(123, []))
print(append_if_multiple_of_five(123))

Result :

[100]
[105]
[]
[]
[]

In Python, sometimes you will see this syntax

with something:
    do_something_else(something)

Then you ask yourself, why I have to use this with syntax ? What runs inside that statement ?

Here is how it works, it's called Context Managers in object

You can define context managers for a class of object to make sure some logic runs correctly without forgeting

Example :

class Animal:
    def __init__(self, name):
        self.name = name
    
    def __enter__(self):
        print("Enter the room !")
    
    def __exit__(self, exc_type, exc_value, traceback):
        print("Exit the room !")

kitty = Animal("Kitty Kat")
with kitty:
    print(kitty.name)

Result

Enter the room !
Kitty Kat
Exit the room !

So, when you start using with keyword on a object, it runs __enter__ method, when everything inside with block is runned, __exit__ will be called !

It's cool feature of Python !

Sometimes you want to use another private key to authorize to remote repository.

Just add an environment variable before the command you wanna run : GIT_SSH_COMMAND='ssh -i [your-private-key]

Example :

$ GIT_SSH_COMMAND='ssh -i ~/keys/key1' git pull

Sometimes you want to put .git data into outside directory (to use another disk partition or to protect your git data). Use --separate-git-dir= option to get that.

Example :

$ git init --separate-git-dir=/var/gitstorage/myproject

Bonus : to protect .git data from other users, use this option --shared within octal value (same to chmod)

Example : this will protect git file from writing by group and reading/writing by others

$ git init --separate-git-dir=/var/gitstorage/myproject --shared=0640

Curl is good lib and tool to simulate HTTP requests. One common usecase is reusing the cookies between 2 or more requests. So you don't have to copied last "Set-Cookie" of previous response then paste it to "Cookie" of next request.

To achieve that, you have to use a cookie jar (sounds fun) to store cookies then use that cookie jar in next request. We have two parameters :

-c [cookie_jar_file] : store response cookies in a file

-b [cookie_jar_file] : getting cookies from cookie jar file then send it in request

Combine them together we can simulate a real browser HTTP requests easily.

Example :

$ curl -c cookies.txt -b cookies.txt -XPOST -d "user=admin&password=hahahehe" http://example.com/login
......
$ curl -c cookies.txt -b cookies.txt -XGET http://example.com/dashboard
Hello admin !

You can encrypt and decrypt the file using openssl command line. Somehow you will need to encrypt your important file with a secret key.

Encrypt

openssl enc -aes-256-cbc -in [input_file] -out [output_file]

Then Enter 2 times your secret key (this should be hard to guess and don't loose it)

Decrypt

openssl enc -aes-256-cbc -d -in [input_file] > [output_file]

Then enter your secret key to decrypt the content !

Notice : should use a 10-char length secret with alpha nums and special characters

Today, I read my junior team-mate code and find this line

$number = rand(2,1000)*rand(2,1000);

This made me remember that's same idea of my own in many years ago. Then I ask myself, is it good to generate a random number from 2 random numbers ?

So the main reason that he wrote this was making probability of same number at same time must be low. We can know that this will return a random number from 4 to 1000000. So why don't we just do this ?

$number = rand(4,1000000);

This function will make probability lower than above function, because above function is commutative (AB = BA). So stay away from commutative function (+, *) when generate a random number.

Need more powerful ? Try Pseudorandom Number Generator (PRNG) !!