Keep learning forward
To be updated ...#TIL : View DNS history of a domain
You can check the history of a domain (A Record). It's useful in case you forgot the old IP of domain.
Check it here : http://viewdns.info/iphistory/
Example : this is Github A record history http://viewdns.info/iphistory/?domain=github.com
#TIL : Never autostart XDebug in cli environment
TLDR;
Never ever enable xdebug.remote_autostart in cli
Xdebug is handy extension helps you debug your PHP code. But it slows the performance in cli mode, especially run PHP cli tool like composer or phpunit.
So please disable Xdebug in cli mode or set
xdebug.remote_autostart=0
in INI file.#TIL : try, catch and finally in PHP
We have to deal with exceptions every moment we touch PHP web development, and so please be carefully with running order of exception catching.
Here is an example
<?php function a() { try { throw new Exception('dsads'); } catch (Exception $e) { return 'b'; } finally { echo 'c'; } } echo a();
Then the output is
cb
Than mean even
return 'b';
runs, the finally code must be runned before function result passes out.#TIL : Use temporarily data from another database in SQLite
Sometimes, we need to use temporarily data from another database file. There is simple and fast way to achieve that without transfering data from file X to file Y.
We connect to main database
$ sqlite3 main.sqlite
Then using the ATTACH command to attach another database as an alias in main database
> ATTACH another_db.sqlite as TEMP;
Let listing the tables
> .tables TEMP.users TEMP.posts users admins categories
As you can see, we got a list of tables, which are prefixed by defined alias
TEMP
we used in ATTACH command. So you only need to use these tables as normal tables.Example of copying data between 2 tables (same structure) :
> INSERT INTO users SELECT * FROM TEMP.users;
#TIL : Exporting environment variables on virtual env activate
You can put common environment variables to the file
venv/bin/activate
. So everytime we active the virtual env, everything is on the way# venv/bin/active content # export your env vars here export FLASK_APP=hello export FLASK_ENV=development export DATABASE=hello.sqlite3 export SECRET_KEY=secret_key_here
#TIL : Setup wildcard domains .test for development in MacOS
Too tired of setting your local domain each time you create new virtual development domain, etc helloworld.test, unit.test point to 127.0.0.1
There is a better way to achieve that by using dnsmasq, then set up a wildcard domains for development. In this case I use .test because .dev has been owned by Google and they strictly use HTTPS in mainly browsers.
Install dnsmasq
$ brew install dnsmasq
Adding .test wildcard to config file
$ echo 'address=/.test/127.0.0.1' > $(brew --prefix)/etc/dnsmasq.conf
Setup dnsmasq as a startup service
$ sudo brew services start dnsmasq
Then add
127.0.0.1
(dnsmasq IP) as first DNS resolverSystem Preferences > Network > Wi-Fi > Advanced... > DNS > add 127.0.0.1 > move it to top of the list.
Checking everything is worked by listing all resolvers
$ scutil --dns
Try it out
$ nslookup -type=a something.test $ ping helloworld.test
#TIL : Send a file through networking via netcat
If you're working on 2 machines in same networking and want to send a file from machine A to machine B. But you don't have USB, floopy disk :lol: or insanse Bluetooth. There is simple way to send a file to another computer without setting up SSH or SMB (althrough these way are safer than it).
On the machine A (with IP address : 192.168.1.2)
$ cat data.txt | sudo nc -l 0.0.0.0 6666
On the machine B
$ nc 192.168.1.2 6666 > here_the_data.txt
Have fun playing net😼 !! ;)
#TIL : Create tiny chat channel via netcat
In a network, you can create a tiny chatting channel using netcat. It's lightweight TCP protocol with plain-text transmission, so be carefully on using.
First, create a channel by picking port number (ex: 7777)
$ sudo nc -l 0.0.0.0 7777
Then, tell you friend your IP and channel port. He will use this info to connect the channel
$ nc 192.168.1.2 7777
Finnally, start chatting !! Each message will be send when you press [Enter]
Note: End the session by press
Ctrl + D
#TIL : Prepend line number to file
When you want to prepend line number in every line of file, use the
-n
flag ofcat
tool.Example :
cat -n a.txt
Or even from many file
cat -n a.txt b.txt c.txt
#TIL : Cut file content from line to line
In case you have a big file which contains a lot of content (2+ GB). And you only need a small part from the file (the part is continuous string from line X to line Y).
You have many ways to achieve that :
- Use
vi
editor and delete from line 1 to line (X-1) by press[X-1]dd
then go to line (Y-X+2) and delete to last line by pressdG
- Use
sed -n '[X][Y]p' [input_file] > [output_file]
. Example :sed -n '15,68p' a.sql > b.sql
- Use
head
andtail
trick :head -n[Y] [input_file] | tail -n[Y-X+1] > [output_file]
I personally recommend using the
sed
way, it's faster and simpler to remember.- Use
#TIL : Create a sequence of numbers
In the past, every time I want to create a sequence of numbers. I have to use something like MS EXCEL, then copy it and paste to text editor. It's tricky way and slow !
Now, I can use the handy tool
seq
to achieve thatman seq
SEQ(1) User Commands SEQ(1) NAME seq - print a sequence of numbers SYNOPSIS seq [OPTION]... LAST seq [OPTION]... FIRST LAST seq [OPTION]... FIRST INCREMENT LAST DESCRIPTION Print numbers from FIRST to LAST, in steps of INCREMENT. Mandatory arguments to long options are mandatory for short options too. -f, --format=FORMAT use printf style floating-point FORMAT -s, --separator=STRING use STRING to separate numbers (default: \n) -w, --equal-width equalize width by padding with leading zeroes
So we have 3 main arguments (same as for loop) :
- FIRST
- INCREMENT
- LAST
And 3 options :
- format : you can use string format like
This is number %g
- separetor : default is new line
- equal width : padding with leading zeroes
Example :
$ seq -f"This is number %g" 3 4 20 This is number 3 This is number 7 This is number 11 This is number 15 This is number 19
$ seq -w -s", " 10 01, 02, 03, 04, 05, 06, 07, 08, 09, 10
#TIL : Install CA root certificate on iOS device
Disclaimer : ⚠️ You can do it, but it's at your own risk !
Sometimes you want to accept a SSL firewall proxy or self-MITM proxy, the important step is installing its CA root certificate to your device. Because iOS apps almost use all https connections (that's new rule).
This is the way to install and enable custom CA Root cert :
- Step 1 : encode your certificate to binary-PEM (only need when you try
cat [ca-cert]
and see ASCII base64 characters)
openssl x509 -outform der -in [ca-cert] -out [new-ca-cert].crt
- Step 2 : Transfer the root certificate to your device (can use 1 of 2 methods : uploading cert to public webserver and open link in Safari app; or share certificate file through AirDrop - between 2 Apple devices).
Tips : use
ngrok
as a simple tunnel webserver if you don't have AirDrop supported PC.Step 3 : Click Install on install profile screen
Step 4 : Enable installed certificate, go to
Settings > General > About > Certificate Trust Settings
, then switch On your certificate item. (You could disable it when you don't need it)
;) Check the web connection !
- Step 1 : encode your certificate to binary-PEM (only need when you try
#TIL : Flush DNS cache on iOS device
There are 2 simple ways to clear DNS cache on iOS devices :
- (FASTER) Just enable the Airplane Mode, wait 10 seconds and disable it
- (SLOWER) Reboot the device ! ;) You know this always be classic answer for many questions :D
#TIL : Sending Cookie in AJAX CORs request
By default, browser will remove the cookie and authorization header from AJAX CORs request. So
before sending out the request, make surewithCredentials
must be true.In this case, CORs response must specify which origin is allowed (mean
no wildcard allowed origin rule).#TIL : Prevent source hacking from .git directory exposing
Many web project use Git as source version control tools. So in production
server, we could expose the hidden.git
directory - which contains all most
infomation about project source code.To "rip" a source code from a vulnerable website, we can use this tool : https://github.com/kost/dvcs-ripper#git
So to prevent this happens, try to deny all http access to hidden files and
directories (usually starts by.
character)Example of Nginx config
location ~ /\. { deny all; }
#TIL : Build lightweight image by using multistage
Docker is great tool to build a pull-n-run application. But sometimes, your image will be large if you build image from a big base image which has heavy compliling toolbox.
Ex:
One-stage build
FROM golang:1.9.2 WORKDIR /go/src/github.com/khanhicetea/test/ COPY . . RUN CGO_ENABLED=0 GOOS=linux go build . ENTRYPOINT ["/go/src/github.com/khanhicetea/test/test"]
Multi-stage builds
FROM golang:1.9.2 WORKDIR /go/src/github.com/khanhicetea/test/ COPY . . RUN CGO_ENABLED=0 GOOS=linux go build . FROM scratch COPY --from=0 /go/src/github.com/khanhicetea/test/test . ENTRYPOINT ["/test"]
So final image will only contains /test excutable file.
COPY --from=0
means COPY from build has index0
in Dockerfile.#TIL : List opening ports or listening UNIX sockets
In Linux, you can use
netstat
to list all opening ports and listening UNIX sockets$ sudo netstat -npl
Tip to remember command : network statistics - natual languge processing
;)
#TIL : Convert tabs to spaces
This is my config to use 4 spaces instead tab
filetype plugin indent on set tabstop=4 set shiftwidth=4 set expandtab
To convert existing file from tabs to spaces, use this command
:%retab
#TIL : Transaction style in Redis
In Redis, you can use transaction-style (mean queue commands then flush it once). It will improve performance in many case where latency or networking is slow.
> SET hoge 2 OK > MULTI OK > INCR foo QUEUED > INCR hoge QUEUED > EXEC 1) (integer) 1 2) (integer) 1
MULTI
command is begin transaction andEXEC
command is commit transactionThe result will be returned in order your command queue list.
#TIL : Check vcl file syntax before restarting
Like NginX, Varnish has a syntax checker function that helps us test the syntactic correctness.
$ varnishd -C -f [vcl file path]
Varnish will compile the file and output the result to stdout. If something goes wrong, it will throw a message like
> Message from VCC-compiler: > Expected an action, 'if', '{' or '}' > ('input' Line 74 Pos 6) > vcl_hash(req.http.Cookie); > -----########------------------ > > Running VCC-compiler failed, exit 1
#TIL : Using mark to bookmark checkpoints in files
Bookmarking a checkpoint will help you get back to it intermidately. Ex: your have to find some text to replace something but want to return back current position.
Set a mark
- [NORMAL MODE] , type
m
then follow by a letter from a-z (lowercase is filescope, uppercase for global scope - vim scope)
Go to a mark
- [NORMAL MODE] , type backstick ` then follow by the letter your marked above.
List all current marks
- [NORMAL MODE],
:marks
- It shows all marks included special ones :
Command Description `. jump to position where last change occurred in current buffer `" jump to position where last exited current buffer `0 jump to position in last file edited (when exited Vim) '' jump back (to line in current buffer where jumped from) `` jump back (to position in current buffer where jumped from) TIPS : Can use it as a motion with
c
hange,d
elete ory
ank- [NORMAL MODE] , type
#TIL : Basics of Elasticsearch
Last days, I developed a EFK stack to centralize my system logging. I really like the concepts of FluentD, it's better than original stack ELK of elastic company.
So I need to learn basics about Elasticsearch and Kibana
This is what I learned :
# Get all documents from elasticsearch node GET _search { "query": { "match_all": {} } } # Check nodes statistics GET /_nodes/stats # Check health of cluster (I don't know why it is always yello status) GET _cluster/health # Get list of indices (indexes) GET /_cat/indices?v # Delete a index (with its data) with name DELETE /[index-name]
#TIL : Use journalctl to check system logs
Logging and Monitoring are important factor for system admin. Checking the log will help you have a closer look into the issue. One tool could help you will handy features is
journalctl
.Here are simple options :
-f
: follow the log (tailf)-u [service]
: filter to show only [service] logs--since=[date]
: Show entries not older than the specified date--until=[date]
: Show entries not newer than the specified date
Example :
$ sudo journalctl -u nginx.service $ sudo journalctl -u nginx.service --since yesterday $ sudo journalctl -u nginx.service --since "2018-01-01" --until today
#TIL : Ansible running host pattern
Ansible supports pattern to select and filter running hosts from all hosts. Here is some common pattern
*
: wildcard, standalone meanall
group1,group2
: run hosts belong to group1 or group2*.web
: run hosts belongs to group matches pattern. Ex:backend.web
,frontend.web
all:!abc
: run all hosts exclude hosts belongs to groupabc
Infrastructure by code ;)
#TIL : Set up simple rate limiting on specified port using UFW
Allow unmetrered connections on networking is so risky. Attacker can use the brute-force attacks to comprosise your service (or simple DOS).
Linux has a cool firewall to hanlde this, via
ip-tables
. But it's so complicated to remember all the rule and syntax. That's whyUFW
was born to save us. :DYou can use simple command to manage your firewall
$ ufw default deny incoming # deny any incoming port, should be run before allow any port $ ufw default allow outgoing # allow any outgoing port $ ufw allow 80 # allow port 80 $ ufw deny 53/udp # allow udp protocol to port 53 $ ufw disable # disable firewall $ ufw enable # enable firewall $ ufw status # check all the rules $ ufw delete [num] # delete the rule by its order in status result $ ufw reload # reload all rule $ ufw limit ssh/tcp # finnaly, limit ssh (port 22 tcp), deny connections if an IP address has attempted to initiate 6 or more connections in the last 30 seconds
#TIL : Disable IPv6 to stop getting stuck in network
I know IPv6 will be future for networking, but at this moment "It's suck !" :(
Some service will be failed when trying to connect IPv6 destination :
- apt package manager
- smtp
- curl
So I decided to disable IPv6 on every production server.
$ echo "net.ipv6.conf.all.disable_ipv6 = 1" | sudo tee -a /etc/sysctl.conf $ echo "net.ipv6.conf.default.disable_ipv6 = 1" | sudo tee -a /etc/sysctl.conf $ echo "net.ipv6.conf.lo.disable_ipv6 = 1" | sudo tee -a /etc/sysctl.conf $ $ sudo sysctl -p
I will re-enable it when everything works perfectly !
#TIL : Tại sao biển xanh lại mặn ? :lol:
TLDR;
Biển xanh lại mặn bởi vì đá ở mặt đất cọ xát với mưa trên trời (chứ không phải cá nó đái 😂 )
Read more : https://oceanservice.noaa.gov/facts/whysalty.html
#TIL : Trigger event after setting val in jQuery
After setting value of an input via
val
method, we should call thechange
chaining method to trigger theonChange
event of element.$('#selectCity').change(function() { console.log($(this).val()); }); $('#selectCity').val('HaNoi'); // No trigger $('#selectCity').val('HoChiMinh').change(); // Fire trigger
#TIL : Tại sao biển xanh lại mặn ? :lol:
TLDR;
Biển xanh lại mặn bởi vì đá ở mặt đất cọ xát với mưa trên trời (chứ không phải cá nó đái 😂 )
Read more : https://oceanservice.noaa.gov/facts/whysalty.html
#TIL : Tracking changes of cookie on webpage
Using
Object.defineProperty
helper function as I wrote 3 days ago. We could track the changes of cookie on webpage.// Based on Vlad Shevchenko's script at https://stackoverflow.com/a/36826049 var cookieSetterOrig = document.__lookupSetter__("cookie"); // get origin setter function var cookieGetterOrig = document.__lookupGetter__("cookie"); // get origin getter function Object.defineProperty(document, "cookie", { get: function () { console.trace(); return cookieGetterOrig.apply(document); }, set: function () { console.log(arguments); console.trace(); return cookieSetterOrig.apply(document, arguments); }, configurable: true });
Notice : This code only works if cookie is changed by javascript, not http header request !