Reverse Proxy & Speed with NGINX

I like fast things

Warning: This article is not a tutorial but rather our experience using Nginx & Apache with SSL and Geo Blocking.

I like things working as fast as possible and have been fiddling with various mechanisms to improve the speed web pages are served to end users. At the same time I want to minimise resources used. As a 55 year old I come from an era when computer resources were limited and a computer with 4 Megabytes of RAM was considered fancy. A 20 Megabyte Hard Disk cost $600 or more. Optimisation was something we all did by default. Folks now who have loads of CPU power & Memory can’t fully appreciate how good they have it.

We tried squid proxy, Varnish and others. We did all the tuning we could think of but always ran into a roadblock somewhere along the way. For example, Varnish is very good and works as advertised, however, it will not do HTTPS unless we pay squillions of dollars per month for the privilege.

Nginx is more than a Web Server

NGINX has been around for a while and I’d heard of it but had never tried it. My understanding was it was a web server and often a replacement for Apache. I was an Apache kind of guy and that’s what we were sticking with because it was familiar like an old slipper. But I read somewhere (everywhere all over the internet) that Nginx could act as a front end Proxy/Cache for Apache, with excellent speed results. NGINX, here I come.

Cache Analogy – Game of Thrones

You’re watching Game of Thrones and want a cold beer. You pause the TV, stand up & walk to the kitchen, open the fridge and grab a beer, walk back to the TV room with a beer in your hand, sit down and continue watching.

Or, pause the TV, go to the fridge & grab three beers. Then each time you want a beer grab one without interrupting Game of Thrones. Effectively bypassing the walk to the fridge altogether.

With a well configured cache things happen faster for the end user, and they’re the folks you want to impress.

Now I’m not going to lie, I’ve had some issues along the way but overall my experience has been good. It has worked brilliantly for all our websites (static html, joomla and wordpress), except one. That one site became my biggest issue over a period of a month and my grey hair is even more grey as a result. It’s a WordPress site which appeared to be functioning most of the time, but after 24 hours the home page was not being served, but rather a random article. No matter what I did it would not load the home page. The only way to make it work consistently was to disable caching for that site. Obviously that wasn’t good enough for me.

Eventually it started working and I can’t for the life of me remember what fixed it. I struggled for so long and made so many changes, commented out variables & parameters etc. Whatever it was, I hope I don’t accidentally undo it at some point in the future.

Tha basics

Nginx receives all HTTP & HTTPS requests and serves the results (Joomla, WordPress, HTML etc.) from the cache at three times the speed compared to Apache on its own. If it is unable to fulfil a request it forwards it to Apache and stores it in the cache for next time. It is important to note Nginx can act as a front-end for various apps such as MySQL and everything else you can think of. The back-end apps can be on the same server or on other machines with limited or no internet access. Nginx does an excellent job of load balancing too. You might have an enormous load in which case you have multiple Apache back ends and have configure Nginx to send requests to the machine currently doing less work.

I discovered that Nginx by default is single threaded, but in your ‘Location’ blocks you can turn threads on. You should assign processes to specific CPU Cores too, for example; four workers with one per core. Right at the top of /etc/nginx/nginx.conf.

user                    nginx;
worker_processes        4;
worker_cpu_affinity     0001 0010 0100 1000;
worker_rlimit_nofile    5000; # You can increase this as required

# In your location blocks, add the following;
#    
#   aio threads;
#   sendfile on; # sendfile may fail on some Operating Systems
# 
# Obviously remove the # character :)

Geo Blocking

Like many WordPress operators I’ve found a real problem with Chinese, Russian, American and others who hammer wp-login.php attempting to stumble upon a valid username & password.

You should consider installing some form of Geo Blocking (google nginx geoip2 or geoip). Once configured you can block entire countries. The best method is to return code 444 which simply ignores the request. No error message or other information is sent to the potential hacker. The attempt can be logged so you can keep an eye on how well your block is working.

In my nginx.conf http block I have the following;

   geoip2 /usr/share/GeoIP/GeoLite2-Country.mmdb {
   auto_reload 5m;
   $geoip2_metadata_country_build metadata build_epoch;
   $geoip2_data_country_code default=AU country iso_code;
   $geoip2_data_country_name country names en;
   }

   geoip2 /usr/share/GeoIP/GeoLite2-City.mmdb {
   $geoip2_data_city_name default=Mandurah city names en;
   }

   geoip_country /usr/share/GeoIP/GeoIP.dat;
   map $geoip_country_code $allowed_country {
   default yes;
   RU no; # Russia
   CN no; # China
   A1 no; # Anonymous proxy
   }

Within your server block include the following;

   if ($allowed_country = no) { return 444; }

As a result the hammering of various files stopped, which is awesome. In addition to Geo Blocking I also use Fail2ban which is capable of picking up all sorts of attacks and banning the source IP Address. That’s a story for another day.

You will know Geo Blocking is working because your Nginx access log for your sites will, over time, have entries like which you can find using grep or similar tool;

Command: sudo grep '444 ' /var/log/nginx/the_access_log_filenname

[24/Oct/2019:19:51:01 +0800] "GET /wp-login.php HTTP/1.1" 444 0 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0"

Note the space in the grep command ('444 ').

SSL

Some sites on this server were HTTPS. When I converted to Nginx I did it the wrong way around. I was trying to maintain the working HTTPS status of my Apache Virtual Hosts when I should have been setting them up as standard HTTP.

The way to do SSL is to use Let’s Encrypt certificates and Certbot on the Nginx front-end with standard HTTP on the back-end. I had some trouble setting up multiple domains because Certbot wanted to manage the certificates. In the end, the best method was to create a stand-alone certificate and configure each site manually.

Now Nginx handles HTTPS and it works flawlessly. I have Apache on an address which is not accessible from the Internet, therefore Nginx adds a level of security which makes me feel good about myself.

Conclusion

If you are looking for speed, Nginx is the way to go. It performs brilliantly for both HTTP & HTTPS. Once you are up and running you can make fine tuning changes to various settings and see if you can get more out of.

I don’t know why I didn’t do this years ago.