Placing Magento 2 behind Varnish reverse proxy

Magento 2 & VarnishEven though Magento 2 Admin clearly states that Built-In Application cache is not recommended for production use, most Magento stores I found through BuiltWith don't have Varnish configured yet. Additionally, I have the feeling that most Magento 2 developers do not have Varnish configured locally while developing. While Magento 2 page cache has been developed in a way that you should be able to transparently switch caching backends, this probably will not be the case in real life therefore getting to know idea of reverse proxy could come in handy.

In attempt to address points outlined above, I'll write a few paragraphs describing my preferred setup, one powered by Nginx as Varnish backend. Even if store in question is powered by Apache, applying following information to your environment should just be matter of translating configuration file syntax, but the idea is same regardless to configured Varnish backend.

Before I proceed, I 'll assume you have fully functional Magento 2 installation happily running with Built-In Application caching backend. I'll also assume you used Nginx configuration file bundled with Magento 2 source code - nginx.conf.sample as starting point for your own Nginx configuration. Additionally I'll assume your server is powered by Debian based Linux operating system like Ubuntu, but as I already wrote, general concepts can be applied to any Linux distribution.

Magento

Before proceeding further, you should visit your Magento 2 Admin -> Stores -> Advanced -> System -> Full Page Cache section, and switch caching application to Varnish. Additional options will appear, what you're interested in is button for exporting ready-made configuration file for Varnish version you will use. We'll use Varnish 3.0 branch, so lets click the "Export VCL for Varnish 3" button to export version 3.x compatible varnish.vcl file and place it somewhere safe to be used for configuring Varnish later.

Nginx

So the idea of reverse proxy caching solution like Varnish is to hook it up to port 80 of your Magento store, and to configure your Nginx to listen on some other port (8080 in my case), and serve as data backend in case Varnish is not able to serve request at hand from its cache. Therefore the first thing you should do is unhook Nginx from port 80 by adjusting related virtual hosts file. Essentially you must change following directives:

# Other http context directives...
 
server {
    # Other server context directives...
 
    listen 80 default_server;
    listen [::]:80 default_server;
 
    # Other server context directives...
}
 
# Other http context directives...

into:

# Other http context directives...
 
server {
    # Other server context directives...
 
    listen 8080 default_server;
    listen [::]:8080 default_server;
 
    # Other server context directives...
}
 
# Other http context directives...

Besides adjusting virtual host that usually listens on port 80 in you Nginx configuration, there's one other thing requiring your attention - Varnish can not terminate SSL. In other words, Varnish is not capable of listening on port 443 and decrypting HTTPS traffic using your SSL certificate, therefore if you wish to secure areas of store in question, for example checkout and Magento 2 Admin, you'll need to offload SSL termination to another application. Since Nginx (as well as Apache) is fully capable of decrypting SSL traffic, we'll keep using it for that purpose, and we will then proxy decrypted HTTPS traffic to port 80 to be served by Nginx as usual. So here's how our port 443 Nginx virtual host looks like after required modifications.

# Other http context directives...
 
server {
    listen 443;
    listen [::]:443;
 
    # Other server context directives...
 
    ssl                  on;
 
    ssl_certificate      /etc/ssl/certs/certificate.crt;
    ssl_certificate_key  /etc/ssl/private/private.key;
    ssl_protocols        TLSv1.1 TLSv1.2;
 
    # Other ssl_* server context directives...
 
    # Proxy
    location / {
        proxy_pass http://127.0.0.1:80;
        proxy_set_header X-Real-IP  $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Forwarded-Port 443;
        proxy_set_header Host $host;
    }
}
 
# Other http context directives...

That'll do the trick for the most part, but there's just this thing where Magento will not know if request at hand originally arrived to port 80 or port 443, this is important for generating store wide URLs using proper protocol. That's where those X-Forwarded headers come into play - they are semi-standard headers used in order to pass some information about original request through our proxy. Even though Magento 2 could be monitoring the X-Forwarded-Proto header in order to decide whether request in question is secure or not, this is not the case. Magento 2 observes HTTPS server environment variable which is also semi-standard therefore it's required for us to modify our port 8080 Nginx virtual host to set HTTPS to on, whenever X-Forwarded-Proto is https. In order to do that we must populate flag variable using Nginx map functionality, and pass it along as HTTPS server environment variable using fastcgi_param. Therefore we will modify port 8080 Nginx virtual host by adding following outside of server context:

# Other http context directives...
 
# Vagrant - support HTTPS behind reverse proxy
map $http_x_forwarded_proto $fastcgi_https {
    default $https;
    https on;
}
 
server {
    listen 8080 default_server;
    listen [::]:8080 default_server;
 
    # Other port 8080 directives...
}

This code will populate $fastcgi_https variable with the default value of HTTPS that Nginx handles on its own when not behind proxy, and with value on when behind proxy and when X-Forwarded-Proto is https. Furthermore we must modify Nginx port 8080 virtual host location block which handles PHP files using fastcgi_param, in order to pass this flag to PHP $_SERVER superglobal array used by Magento to correctly detect protocol scheme:

# Other server context directives...
 
location ~ (index|get|static|report|404|503)\.php$ {
    # Other location context directives...
 
    # Support HTTPS behind reverse proxy
    fastcgi_param HTTPS $fastcgi_https;
 
    # Other location context directives...
 
    include        fastcgi_params;
}
 
# Other server context directives...

In order for our changes to take effect, we must test our Nginx configuration:

$ sudo nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

If configuration test is successful, we can restart Nginx service:

$ sudo service nginx restart

Varnish

Last thing we need to do is install and configure Varnish reverse proxy. First lets add official Varnish 3.0 branch repository for Debian based operating system at hand, Ubuntu 14.04 (codename Trusty) in my case:

wget -q https://repo.varnish-cache.org/GPG-key.txt -O- | sudo apt-key add -
sudo add-apt-repository "deb https://repo.varnish-cache.org/ubuntu/ trusty varnish-3.0"

Then we can proceed to installing Varnish:

$ sudo apt-get install varnish

Next, you need to configure Varnish to listen on port 80 by adjusting it's daemon options as well as instruct Varnish daemon to use varnish.vcl file we exported from Magento 2 Admin panel. First step is to use tool of your choice in order to copy varnish.vcl file to your production server, to /etc/varnish/varnish.vcl location.

Next, you need to adjust Varnish daemon options as described, and to restart Varnish.:

echo "DAEMON_OPTS=\"-a :80 -T localhost:6082 -f /etc/varnish/varnish.vcl -S /etc/varnish/secret -s malloc,256m\"" | sudo tee -a /etc/default/varnish
sudo service varnish restart

Now you just check if relevant services are up:

$ ps -e | egrep 'nginx|varnish'
 1174 ?        00:00:00 varnishd
 1175 ?        00:00:04 varnishd
 1860 ?        00:00:00 nginx
 1861 ?        00:00:00 nginx
 1862 ?        00:00:00 nginx

If you received output like the one from above, all systems are green and you're ready to check your Magento 2 frontend. For that purpose you can use varnishlog which comes bundled with Varnish, more precisely command crafted to show only requests for which Varnish had to contact it's configured backend, in this case Magento 2 running on port 8080. When you load specific Magento 2 page, request will get cached by Varnish, and on the subsequent load for the same page, request will get served by Varnish without contacting Magento.

Here's first request for product page with http://magento2.loc/hero-hoodie.html URL at my Magento 2 copy with sample data installed:

$ varnishlog -b | grep 'TxURL'
   13 TxURL        b /hero-hoodie.html
   15 TxURL        b /static/frontend/Magento/luma/en_US/mage/gallery/gallery.css
   13 TxURL        b /page_cache/block/esi/blocks/[%22catalog.topnav%22]/handles/[%22default%22,%22catalog_product_view%22,%22catalog_product_view_id_163%22,%22catalog_product_view_sku_MH07%22,%22catalog_product_view_type_configurable%22]/
   .
   .
   .

We can see that Varnish had to contact Magento 2 in order to obtain markup for /hero-hoodie.html. And here's second request to this same URL, as monitored by varnishlog:

$ varnishlog -b | grep 'TxURL'
   13 TxURL        b /static/frontend/Magento/luma/en_US/Magento_Checkout/js/checkout-loader.js
   15 TxURL        b /static/frontend/Magento/luma/en_US/Magento_Ui/templates/block-loader.html
   15 TxURL        b /static/frontend/Magento/luma/en_US/mage/gallery/gallery.html
   .
   .
   .

This time there's no /hero-hoodie.html in output of our varnishlog command, this means that markup was served by Varnish without contacting Magento. Please note that content requested with Cache-Control header of private or no-store, no-cache as well as static resources are not cached by Varnish.

Lastly, if you are interested in complete environment built around these concepts, especially if you are developer requiring Varnish setup for you local Magento 2 development, you can checkout my Magento 2 Vagrant box.

DevGenii

A quality focused Magento specialized web development agency. Get in touch!

3 thoughts on “Placing Magento 2 behind Varnish reverse proxy

  1. krystian

    hello. I have some problems with this setup, I get 502 bad gateway.
    I want my store to be full https://

    in magento2 admin area, base url is https://domain.com
    whereas you keep your server at 8080 port (which is http)

    what should be the settings in admin area in this case?
    secure url:
    non secure url:

    ?? thank you!

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *