Keep learning forward
To be updated ...#TIL : Using Curl to check downtime and ssl cert expiration
Before I wrote a cron script to check website is down, but it can't check multiple endpoint and can't alert if ssl certificates is about to expired.
So, I created this bash snippet function to check multiple endpoints (and their SSL certificates)
#!/bin/bash function checkEndpoint() { URL="$1" FIND="$2" RESPONSE=$(curl -m 10 -s "$URL") CONTENT=$(echo $RESPONSE | grep "$FIND") if [ $? -ne 0 ]; then echo "Error: $URL same down. Please check!" return 1 fi # Check SSL certificate expiration HOST=$(echo "$URL" | sed -E 's|https?://([^/]+).*|\1|') EXPIRY_DATE=$(echo | openssl s_client -connect "$HOST:443" -servername "$HOST" 2>/dev/null | openssl x509 -noout -enddate | cut -d= -f2) EXPIRY_TIMESTAMP=$(date -d "$EXPIRY_DATE" +%s) CURRENT_TIMESTAMP=$(date +%s) DAYS_LEFT=$(( (EXPIRY_TIMESTAMP - CURRENT_TIMESTAMP) / 86400 )) if [ "$DAYS_LEFT" -le 7 ]; then echo "Error: $URL - SSL certificate will expire in $DAYS_LEFT days." return 1 fi return 0 } function checkAndAlert() { CHAT_ID="telegram_chat_id" BASIC_AUTH_BOT_PARAM="bot*****:*********" ALERT_MSG=$(checkEndpoint "$1" "$2") if [ $? -ne 0 ]; then curl -s -m 10 --get --data-urlencode "chat_id=$CHAT_ID" --data-urlencode "text=$ALERT_MSG" "https://api.telegram.org/$BASIC_AUTH_BOT_PARAM/sendMessage" return 1 fi } checkAndAlert "https://google.com/" "google" checkAndAlert "https://news.ycombinator.com" "Hacker News"
#TIL : PHP memory allocation when passing arguments into function
In PHP, when you pass variables into another function arguments, it
- Copy the variable memory if the argument is scalar data type (int, float, bool, ...), because it's cheap operations
- Copy a
shadow clone jutsu
variable zval struct (24-32 bytes), then point the real data pointer of dynamically data type (array, string, $object)
Below testing will say more than me
<?php class A { public $arr; public function __construct() { $this->arr = array_fill(0, 10000, "Hello world"); } public function push(){ $this->arr[] = "ok"; } } function haha() { $a = func_get_args()[0]; mem("inside func haha, after func_get_args"); return $a; } function a($b, $c) { $a = new A(); $str = str_repeat("Helo", 100000); mem("after create an object A"); $x = haha($a, $str, $b, $c); mem("after call func_get_args"); $x->arr[] = "there"; var_dump($a->arr[count($a->arr) - 1]); var_dump($x->arr[count($x->arr) - 1]); $x->push(); var_dump($a->arr[count($a->arr) - 1]); var_dump($x->arr[count($x->arr) - 1]); mem("after modify arr property"); // var_dump($a === $x); return $b+$c; } function mem($msg = null) { if ($msg) print_r($msg . " : "); echo (memory_get_usage(false) . " bytes\n"); } mem("start"); var_dump(a(1,2)); mem("end");
This is the result
start : 469464 bytes after create an object A : 1137200 bytes inside func haha, after func_get_args : 1137200 bytes after call func_get_args : 1137200 bytes string(5) "there" string(5) "there" string(2) "ok" string(2) "ok" after modify arr property : 1137200 bytes int(3) end : 469464 bytes
Surprise ! :D
#TIL : Detect ViteHMR dev server is running
Vite server has a middleware named
viteHMRPingMiddleware
in this source codeSo we have a simple way to check if Vite HMR server is running by calling a ping http request to the url endpoint (with timeout 1s)
function checkViteHMR() { fetch('http://localhost:5173/_ping', { method: 'HEAD', headers: { 'Accept': 'text/x-vite-ping' } }) .then(response => { if (response.status === 204) { // Do something here console.log('Vite HMR is running'); } else { console.log('Received response with status:', response.status); } }) .catch(error => { console.error('Error making request:', error); }); }
#TIL : Script tags in remote HTML request won't run when using innerHTML
By default, all script tags will be turned off when you try to insert to a element using innerHTML. So the tricks to make it run to create the new script tag using
document.createElement('script')
then copy the original scripts to new scripts. After then append the scripts to body or the parent element.<div> <h1>Hello from Test</h1> <script data-sign="hello"> console.log(document.currentScript); const parent = document.currentScript.parentElement; parent.style.color = 'blue'; </script> </div>
<!DOCTYPE html> <html lang="en"> <body> <div id="test"></div> </body> <script> const ele = document.getElementById('test'); fetch('test.html') .then((res) => res.text()) .then((content) => { ele.innerHTML = content; ele.querySelectorAll('script').forEach((script) => { console.log(script.dataset.sign || 'No signature'); // Your can use this to check the signature and minimize XSS attacks const newScript = document.createElement('script'); newScript.textContent = script.textContent; ele.appendChild(newScript); }); }); </script> </html>
HTMX Library use the same way to do the trick, to turn off using
htmx.config.allowScriptTags = false
#TIL : Cronjob monitoring web endpoint and alert to Telegram
This is shell script using
curl
andgrep
to send request then check if the keyword (FIND) include in the response text body. If failed, try to alertURL="https://news.ycombinator.com/" FIND="Hacker News" response=$(curl -m 10 -s "$URL") content=$(echo $response | grep "$FIND") if [ $? -eq 0 ]; then echo "OK!" exit 0 fi # Do watever you want to alert CHAT_ID="telegram_chat_id" TEXT="The web is down !" BASIC_AUTH_BOT_PARAM="bot*****:*********" curl -s -m 10 --get --data-urlencode "chat_id=$CHAT_ID" --data-urlencode "text=$TEXT" "https://api.telegram.org/$BASIC_AUTH_BOT_PARAM/sendMessage" exit 1
#TIL : XCode automatic format code on build time
XCode (the main IDE of Apple), but lacks much common features like auto formating code.
So we have to install a external tool (lol it's Apple tool itself) bybrew install swift-format
Then I have to use a trick in Build Phases settings in XCode, we create a bash script run format tool on every build time (before source code has been built, because sometimes your code buidling will failed)
Here is the bash script, make sure the Run script below "Run Build Tool Plug-ins" step
echo "Formating all source codes" cd $SOURCE_ROOT /opt/homebrew/bin/swift-format --ignore-unparsable-files -i -p -r . echo "Formated all source codes."
Important : You have to turn off Build Options named User Script Sandboxing (this diffenrence of App Sandboxing), to allow your shell scripts can affected real filesystem instead of shadowing sandbox files, below is the screenshot
#TIL : Generate routing stubs for IDEs
Some IDEs has good LSP or Intellisense can show link of class methods, so we can using this mechanism to listing all the routes to faster searching route and click method link to navigate actual code.
I make this Laravel command to generate a stub file into routes directory so the IDE can index
So sometimes when you need to re-generate stubs, run this command
$ php artisan routing_stubs
<?php namespace App\Console\Commands; use Illuminate\Console\Command; use Illuminate\Support\Facades\Route; class RoutingStubs extends Command { /** * The name and signature of the console command. * * @var string */ protected $signature = 'routing_stubs'; /** * The console command description. * * @var string */ protected $description = 'Generate routing stubs'; public function handle() { $routes = collect(Route::getRoutes()) ->map(fn (\Illuminate\Routing\Route $route) => explode('@', $route->getActionName())) ->mapToGroups(fn ($params) => [$params[0] => $params[1] ?? '']) ->filter(fn ($methods, $class) => class_exists($class)) ->map(fn ($methods, $class) => $methods->map( fn ($method) => sprintf("* @uses %s::%s", $class, $method) )->join("\n")) ->flatten() ->join("\n"); file_put_contents(base_path('routes/stubs.php'), "<?php\n/**\n" . $routes . "\n*/\n"); $this->info('Done'); } }
#TIL : Smallest inline dummy image in HTML
Learned from https://www.phpied.com/minimum-viable-no-image-image-src/
You can use this for lazy load images in pages
<img src="data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg'/>" ... >
Hmm, when I inspected his demo image, I saw new img attribute named
decoding="async"
, so interesting! So we can tell browser not to wait process this dummy image for later and priority the rest higher. (Good for LCP vital metric)#TIL : Transform array to object shorthand
To transform a array to stdClass in PHP, try to cast type using
(object)
before the array variable.$arr = ['a' => 1, 'b' => 5]; $obj = (object) $arr; echo $obj->b;
#TIL : Be careful when using built-in btoa from window to decode base64
window.btoa
is built-in function to encode string to base64 string, but it can't process any Unicode characters (each char must be fit in 1 byte, but Unicode is multi-byte char)So solution is transform the value to Latin character using some pre-processing function like
const new_btoa = (val) => btoa(unescape(encodeURIComponent(val))) const result = new_btoa('Hế lô xin chào')
So you have to pair the
atob
tooconst new_atob = (val) => decodeURIComponent(escape(atob(val))) console.log(new_atob(result))
Or using 3rd-lib like https://www.npmjs.com/package/js-base64 (I recommend use this way)
#TIL : Split a Collection items into 2 groups using 1 boolean function
Old fashion way
$groupTrue = $groupFalse = []; foreach ($items as $item) { if (some_logic_function($item)) { $groupTrue[] = $item; } else { $groupFalse[] = $item; } }
New way
list($groupTrue, $groupFalse) = collect($items) ->partition(fn ($item) => some_logic_function($item)) ->map(fn ($group) => $group->values());
tip: Using the last map then values to reset indexing of 2 groups after spliting.
#TIL : Reactive statement explicit dependencies like React useEffect
Svelte uses it compiler to detect the reactive dependencies, to trigger the reactive updater function. So our work need is make sure all dependent variables appear in the line.
I use the below logic trick like this (!0 always true so it won't run init array [x,z,y]), but compiler always see the x,z,y in dependencies. :D
result = (!0 || [x,y,z]) ? calc() : null;
OK, here is sample Svelve 4 code (sure Svelte 5 Runes is the saver)
<script> let x = 1; let y = 1; let z = 1; let result = 0; function calc() { return x * y * z; } $: result = (!0 || [x,y,z]) ? calc() : null; function handleClickX() { x += 1; } function handleClickY() { y += 1; } function handleClickZ() { z += 1; } </script> <button on:click={handleClickX}> x = {x} </button> <button on:click={handleClickY}> y = {y} </button> <button on:click={handleClickZ}> z = {z} </button> <p>Result = {result}</p>
#TIL : Warning on MacOS upgrade major system
Never ever upgrade OS without :
- Backup important files (better sync it to cloud every work hour)
- Unplug all devices or accessories from all USB ports
- Plugged in power cable
Yesterday, I upgraded macOS to another version when plugged to the USB-C hub (HDMI external screen, USB keyboard, USB for LED light). Then it restarted few times and hang with black screen. After waiting 10 mins nothing changes, I power off the macbook hard way (by hold power button). Then I never can boot it again (even to Recovery mode).
After researching whole Internet (Google, MacForum, Youtube) and trying 42 times, I failed to revive it! :(
But, keep searching I found my life saver's blog post , now I can restore the Macbook and reinstall clean MacOS Sonoma.
Thanks GOD :D
Btw, never stupid like me ! And if the thing works fine, keep it works ! If not, carefully upgrade it and always have a plan B.
#TIL : Cache storage utils using redis server
I created a small util Cache storage using redis server to cache seriable data, it stores serialzied data string in text then unserialize to wake the data up.
<?php namespace App\Library; use Predis\Client; use Predis\PredisException; class CacheStorage { public function __construct(private Client $redis) { } public function remember($key, $ttl, $dataFn) { try { $cached_str = $this->redis->get($key); $cached = is_null($cached_str) ? false : unserialize($cached_str); if ($cached) { return $cached; } $freshData = $dataFn(); $this->redis->set($key, serialize($freshData), "ex", $ttl); return $freshData; } catch (PredisException $e) { return $dataFn(); } } }
#TIL : Getting database from data range to array of enum objects
function getData(range, columnMap) { const dataValues = range.getValues(); const mapEntries = Object.entries(columnMap) return dataValues.map((row) => Object.fromEntries(mapEntries.map((colMap) => [colMap[0], row[colMap[1]]]))) }
Example:
You have a sheet with data from A1:D*** (*** is last row), with the field mappings :
- first col is product_code
- second col is total
- fourth col is rank1
So the code of main function is
function main(){ // Get the sheet you want var sheet = SpreadsheetApp.getActiveSpreadsheet().getSheets()[0]; var lastRow = sheet.getLastRow(); var dataRange = sheet.getRange("A1:D" + (lastRow)); var data = getData(dataRange, { "product_code": 0, "total": 1, // skip third column "rank1": 3, }) console.log(data) }
#TIL : JS DOM event deepdive using SvelteJS demo
The DOM event handling go through 2 phrases : Capture and Bubble (with support of preventDefault, stopPropagation, stopImmediatePropagation methods to modify behavior)
Try the demo here : https://svelte.dev/repl/99d1edbf7bd9426384be2c7763d2e872?version=4.2.0
<script> const onClickEle = (e) => { console.log((new Date()).toISOString() + " : Clicked " + e.currentTarget.tagName) } </script> <h5>Default (bubble)</h5> <div on:click={onClickEle}> <button on:click={onClickEle}>Click here!</button> </div> <h5>Default (bubble with stopPropagation)</h5> <div on:click={onClickEle}> <button on:click|stopPropagation={onClickEle}>Click here!</button> </div> <h5>Parent capture</h5> <div on:click|capture={onClickEle}> <button on:click={onClickEle}>Click here!</button> </div> <h5>Parent capture stopPropagation</h5> <div on:click|capture|stopPropagation={onClickEle}> <button on:click={onClickEle}>Click here!</button> </div> <h5>Self (ele = event target)</h5> <div on:click|self={onClickEle}> <button on:click={onClickEle}>Click here!</button> </div> <style> div { padding: 10px 30px; background: gray; cursor: pointer; display: inline } </style>
#TIL : ES6 Module import in client browsers
moduleA.js
export const message = "Hello from moduleA!";
moduleB.js
import { message } from './moduleA.js'; console.log(message);
app.html
<script type="module" src="moduleB.js"></script> // Print console log "Hello from moduleA!"
So moduleB.js acts like a module entry-point, it loads all dependencies deeply to run it-self, so you only need add script tag point to it.
#TIL : SvelteJS : Reactivity via variable assignment
Reactivity in Svelte works via assignment of variable, so we have this as example
let todos = []; // Don't work todos.push({id: 1, name: 'cleaning'}) todos.push({id: 2, name: 'coding'}) // Work todos = [...todos, {id: 1, name: 'cleaning'}] // or this even work as tricky way todos.push({id: 1, name: 'cleaning'}) todos = todos
Why ? I don't know in internal system how it implement, but I think because Svelte works as a compiler, so it triggers the update when it see the assignment variable line.
Different with another frameworks, they work based on proxy object
#TIL : Array group by function in JS
Object.prototype.groupBy = function(cb) { const groupByCategory = this.reduce((group, item) => { const category = cb(item); group[category] = group[category] ?? []; group[category].push(item); return group; }, {}); return groupByCategory; } // Example const people = [{"name":"John","age":28},{"name":"Alice","age":24},{"name":"Michael","age":32},{"name":"Emily","age":29},{"name":"David","age":27},{"name":"Sophia","age":38}]; const by_first_age_period = people.groupBy((person) => Math.floor(person.age / 10) * 10 + '+'); console.log(by_first_age_period);
#TIL : CSS Selector element has an attribute which contains some string
[abc*="something"]
means select element has attributeabc
, and its value contains "something" stringExample :
/* Warning all admin link is red text color */ a[href*="/admin/"] { color: red; }
#TIL : HTTP Status Codes : 301+302 vs 307+308
- HTTP 307 : Temporary redirect, keep METHOD and BODY
- HTTP 308 : Permanent redirect, keep METHOD and BODY
vs
- HTTP 301 : Permanent redirect, fallback to GET method
- HTTP 302 : Temporary redirect, fallback to GET method
#TIL : Becareful on using port mapping of Docker in Development
Many examples, articles and repository use insecure config in port mapping Docker.
This is example in official document of Docker
https://docs.docker.com/engine/reference/run/#detached--d
$ docker run -d -p 80:80 my_image nginx -g 'daemon off;'
or
from 12.2k stars repo in Github
https://github.com/chatwoot/chatwoot/blob/develop/docker-compose.yaml#L89
ports: - '5432:5432'
This mapping config means you will proxy all traffic from the container port to the docker host (your dev machine) with binding
0.0.0.0
. It means if you don't have a firewall or accidently open all ports, your contaier data will be public in the Local Network (so danger if you use in public places like coffee shops)So remember to add the IP, which you wanted to bind in docker host (or have a network firewall ON)
ports: - '127.0.0.1:5432:5432' - '172.17.0.1:5432:5432'
#TIL : Using extra hosts to add custom ip of hostnames to Docker container
By default, all docker container using DNS server from docker host, so if you want to overwrite the specific hostnames ip address, try the flag
--add-host [hostname]:[ip]
$ docker run -it --add-host db1:1.2.3.4 --add-host db2:2.3.4.5 alpine cat /etc/hosts 127.0.0.1 localhost ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters 1.2.3.4 db1 2.3.4.5 db2 172.17.0.2 0bdcf2fb2216
If you use docker-compose, then add the extra-hosts param below the service you want
extra_hosts: - "crm.dev:host-gateway" - "db1:1.2.3.4"
Bonus :
host-gateway
mean the ip address of the gateway of network container in, often be the docker host!#TIL : Laravel run scheduled command within parent environment
Last day I updated my Laravel app from version 8 to 9, all my scheduled command in Console Kernel has run in another environment like "production". Which causes my system didn't work as expected.
Example, I have 2-3 enviroment which I run cronjob like
* * * * * php artisan schedule:run --env=hello * * * * * php artisan schedule:run --env=world
But all sub scheduled commands has been run with "production" instead of "hello" or "world"
So this is my fix, I created a lambda function which pass current environment to the child command.
class Kernel extends ConsoleKernel { protected function schedule(Schedule $schedule) { // Added 2 lines to get lamda function $env = app()->environment(); $schedule_command = fn ($cmd) => $schedule->command($cmd, ['--env' => $env]); // Then schedule using new lambda function $schedule_command('check_product_stock')->everyTenMinutes(); $schedule_command('run_job_queue')->everyMinute(); $schedule_command('clean_old_data')->dailyAt('00:25'); } protected function commands() { $this->load(__DIR__ . '/Commands'); require base_path('routes/console.php'); } }
Enjoy ! :) Hope it works for you !
#TIL : Minimize Google PHP SDK before deploying on production
In case you use
google/apiclient
package to connect to Google API services, remember to cleaning unused services after deploying to production.So let me show you why
First, lets counting number of classes of Google API services inside composer autoload classmap file (Composer use this file as a mapping to lazyload your needed class)
$ cat vendor/composer/autoload_classmap.php | grep 'Google' | wc -l 15436 $ du -h vendor/composer/autoload_classmap.php 3.5M vendor/composer/autoload_classmap.php
As you can see, it has almost 15k classes having
Google
in class namespace. And yourautoload_classmap.php
has 3.5Mb in size (so large)So, let cleaning unused services (by keeping only services you need)
Editing your
composer.json
file by adding"pre-autoload-dump"
inscripts
and"google/apiclient-services"
inextra
, like below (replace services you need like Drive, Youtube, Sheets, etc..){ "scripts": { "pre-autoload-dump": "Google\\Task\\Composer::cleanup" }, "extra": { "google/apiclient-services": [ "Drive", "YouTube", "Sheets" ] } }
Finally, run
composer dump-autoload
again and see the result$ cat vendor/composer/autoload_classmap.php | grep 'Google' | wc -l 383 $ du -m vendor/composer/autoload_classmap.php 1.3M vendor/composer/autoload_classmap.php
Class map reduce from 15k to 400 lines, and size from 3.5M to 1.3M ! Such an optimize !
Beware : If you add services after cleaning, try to run
rm -r vendor/google/apiclient-services && composer update
to redownloading all services classes.#TIL : Getting back Whoops error pages in Laravel 9
Laravel 9 uses "Spatie/Ignition" as default error handling pages , which is beatiful but bloated within add-on service like Flare.
So if you want to get back the good-old "Whoops" error pages, this tutorial is for you :D
First, we remove "spatie/ignition" from composer packages
$ composer remove "spatie/laravel-ignition"
Then, add 3 lines in the
boot
function ofAppServiceProvider
use Illuminate\Contracts\Foundation\ExceptionRenderer; use Illuminate\Foundation\Exceptions\Whoops\WhoopsExceptionRenderer; class AppServiceProvider extends ServiceProvider { public function boot() { // Add 3 lines below between boot function if (config('app.debug')) { $this->app->bind(ExceptionRenderer::class, WhoopsExceptionRenderer::class); } } }
And last but not least, open the
config/app.php
then add to this line to array config (try to replace the name of Editor you use in this list )... 'editor' => 'vscode' ...
Now try to add this line to second line in
web.php
route file to test$x = 1 / 0;
#TIL : DirectAdmin change document root to public directory
By default, the document root of a domain in DirectAdmin (DA) has the format
/home/[your_username]/domains/[your_domain]/public_html
But in case you have to run modern PHP app like Laravel or Symfony framework (they use child public directory as their docroot). In some case, using .htaccess file to rewrite is a hacky way to solve problem (or this is the option when you don't have Admin account of DA)
So, if you have an Admin account, try this
Open menu Server Manager > Custom HTTPD Configurations, then click on your domain httpd config file, then click CUSTOMIZE
In the large textbox, add this line (remind to replace username and domain) to
|?DOCROOT=/home/[your_username]/domains/[your_domain]/public_html/public|
Click SAVE to save modification. Then go to Service Monitor and reload your webservers (Nginx, Apache or OpenLiteSpeed)
#TIL : Insert fastly current date time to cell in Google Spreadsheets
Yo! Back to my #TIL
Today, I find out a way to insert fastly current date or current time to a cell. It makes you faster 3-5 seconds and reduce confusing about what the time is?
- To insert current date : press
Ctrl + :
- To insert current time : press
Ctrl + Shift + :
Excel (Spreadsheets) never dies !
- To insert current date : press
#TIL : JSON Parse is faster than Javascript Object declaration
TLDR;
Use
JSON.parse("[your data in json string]")
if your data is big (>10KB for instance)Short Explaination :
JSON.parse
parses a string to object, so it has many strict requirements than Javascript parses the source code (more syntax, more context)Long Explaination :
#TIL : Use NGINX as a TCP,UDP load balancer
NGINX is well known as a simple and good web server right now, but not everyone knows that NGINX can act like a TCP-UDP loadbalancer. So you won't need to install HAProxy when you need a LoadBalancer.
This feature is released on NGINX 1.9+. So you can setup it by this rule
stream { upstream backend1 { server s1.backend1.com:12345; server s2.backend1.com:12345; } server { listen 54321; proxy_pass backend1; } upstream backend2 { server s1.backend2.com:7777; server s2.backend2.com:7777; server s3.backend2.com:7777; } server { listen 8888 udp; # add udp keyword if you want UDP server proxy_pass backend2; } }
To learn more, click here : https://docs.nginx.com/nginx/admin-guide/load-balancer/tcp-udp-load-balancer/