#TIL : Can not get real IP address from Load Balancer SSL Passthrough

When you use a load balancer stay in front of your app, and use SSL Passthrough mode. You will never get real IP of client, because Load balancer works like a TCP load balancer, which means it can not add extra HTTP headers into encrypted traffic from client when it doesn’t handle SSL termination.

So if you use 1 domain or wildcard subdomains, it’s better if you use SSL Termination mode.

#TIL : Sending Cookie in AJAX CORs request

By default, browser will remove the cookie and authorization header from AJAX CORs request. So
before sending out the request, make sure withCredentials must be true.

In this case, CORs response must specify which origin is allowed (mean
no wildcard allowed origin rule).

#TIL : Bypass CORS by using JSONP callback

Sometimes you are blocked from request a cross-origin resource. Instead of adding our domain to allowed list of them, we can use another way to retrieve data from their API by using JSONP (in case they support it).

The mechanism of JSONP is simple, instead of returning a JSON data. It will return a javascript text with passing your data into a function, whose name is declared in query string. So you just add a new script element with the URL and waiting the callback.

Example :

1
2
3
4
5
6
7
8
function callMeBaby(data) {
console.log(data);
}

var s = document.createElement("script");
s.type = "text/javascript";
s.src = "https://freegeoip.net/json/?callback=callMeBaby";
document.head.appendChild(s);

or using jQuery (hide magic)

1
2
3
4
5
6
7
8
$.ajax({
url: "https://freegeoip.net/json/",
jsonp: "callback",
dataType: "jsonp",
success: function( data ) {
console.log( data );
}
});

#TIL : HSTS rule in browser

HTTP Strict Transport Security (HSTS) is a web security policy mechanism which helps to protect websites against protocol downgrade attacks and cookie hijacking.

Enabling HSTS on your web will make your browser validate every SSL issues more strictly :

  • User can not visit http version on browser
  • User can not add SSL exception for the domain to ignore the warning. (when SSL cert expire or invalid common name)

Note : You can manually remove a domain from HSTS in Chrome by accessing this page URL chrome://net-internals/#hsts

So remember to add HSTS to your website !

#TIL : Using web proxy to bypass firewalls

Someday, you will be blocked by a firewall while trying crawling or accessing some website. The reason is they block your IP address from accessing the server.

One solution is using a web proxy (http proxy, socks4 or socks5) to bypass the firewall, by adding the middle-man server between you and target. It’s a bit unsecured but you could use for https site only.

Some HTTP Proxy supports https will stream TLS data from target to you (so don’t worry about proxy server can read you data). Btw, it only knows which domain and IP address you’re connecting.

To find a free proxy from the internet, try this service : https://gimmeproxy.com/

It provides a cool API to fetch new proxy from its database.

Example this endpoint will return JSON response including proxy anonymous, supports HTTPS, from Japan and minimum speed more than 100KB

1
http://gimmeproxy.com/api/getProxy?anonymityLevel=1&supportsHttps=false&country=JP&minSpeed=100

In case you need more requests per day, try a subscription (cancelable and refundable). I tried last days, and really like their service (although I cancelled subscription b/c I don’t need proxy anymore).

Break the rules ! ;)

#TIL : Ping Google to crawl updated content

When you post new content to your website, the fastest way is ping search engines to notify them. After that, they will try to crawl and index your page.

One way to ping search engines is using XMLRPC ping

This is a example XMLRPC request (HTTP POST request with xml body)

Request

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
> POST /ping/RPC2 HTTP/1.1
> Host: blogsearch.google.com
> User-Agent: curl/7.47.0
> Accept: */*
> content-type: application/xml
> Content-Length: 239
>
<?xml version="1.0" encoding="UTF-8"?>
<methodCall>
<methodName>weblogUpdates.extendedPing</methodName>
<params>
<param>
<value>Page Title</value>
</param>
<param>
<value>http://example.com/helloworld.html</value>
</param>
</params>
</methodCall>

Response

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
< HTTP/1.1 200 OK
< Content-Type: text/xml; charset=ISO-8859-1
< X-Content-Type-Options: nosniff
< Date: Tue, 08 Aug 2017 05:04:01 GMT
< Server: psfe
< Cache-Control: private
< X-XSS-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
< Accept-Ranges: none
< Vary: Accept-Encoding
< Transfer-Encoding: chunked
<
<?xml version="1.0"?>
<methodResponse><params>
<param><value><struct>
<member>
<name>flerror</name><value><boolean>0</boolean></value>
</member>
<member>
<name>message</name><value>Thanks for the ping.</value>
</member>
</struct></value></param>
</params></methodResponse>

Popular XML Servers

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
http://blogsearch.google.com/ping/RPC2
http://api.moreover.com/ping
http://bblog.com/ping.php
http://bitacoras.net/ping
http://blog.goo.ne.jp/XMLRPC
http://blogmatcher.com/u.php
http://coreblog.org/ping/
http://mod-pubsub.org/kn_apps/blogchatt
http://www.lasermemory.com/lsrpc/
http://ping.amagle.com/
http://ping.cocolog-nifty.com/xmlrpc
http://ping.exblog.jp/xmlrpc
http://ping.feedburner.com
http://ping.myblog.jp
http://ping.rootblog.com/rpc.php
http://ping.syndic8.com/xmlrpc.php
http://ping.weblogalot.com/rpc.php
http://pingoat.com/goat/RPC2
http://rcs.datashed.net/RPC2/
http://rpc.blogrolling.com/pinger/
http://rpc.pingomatic.com
http://rpc.technorati.com/rpc/ping
http://rpc.weblogs.com/RPC2
http://www.blogpeople.net/servlet/weblogUpdates
http://www.blogroots.com/tb_populi.blog?id=1
http://www.blogshares.com/rpc.php
http://www.blogsnow.com/ping
http://www.blogstreet.com/xrbin/xmlrpc.cgi
http://xping.pubsub.com/ping/

#TIL : Cloudflare Error 522 Connection Time out

If you are using Cloudflare as a proxied web server, it will provide many benefits about performance (assets caching, prevent DDOS and cheap CDN). But sometimes, you will face to this error “522 Connection Time out”.

The problems caused by :

  • Networking (CF can’t touch origin server : Firewall blocking, Network Layer #1,#2,#3 issue)
  • Timeout (origin server process too long than 90 seconds)
  • Empty or invalid response from origin server
  • No or big HTTP headers (> 8Kb)
  • Failed TCP handshake

Ref: