Even though Magento 2 Admin clearly states that Built-In Application cache is not recommended for production use, most Magento stores I found through BuiltWith don't have Varnish configured yet. Additionally, I have the feeling that most Magento 2 developers do not have Varnish configured locally while developing. While Magento 2 page cache has been developed in a way that you should be able to transparently switch caching backends, this probably will not be the case in real life therefore getting to know idea of reverse proxy could come in handy.
In attempt to address points outlined above, I'll write a few paragraphs describing my preferred setup, one powered by Nginx as Varnish backend. Even if store in question is powered by Apache, applying following information to your environment should just be matter of translating configuration file syntax, but the idea is same regardless to configured Varnish backend.
Before I proceed, I 'll assume you have fully functional Magento 2 installation happily running with Built-In Application caching backend. I'll also assume you used Nginx configuration file bundled with Magento 2 source code - nginx.conf.sample as starting point for your own Nginx configuration. Additionally I'll assume your server is powered by Debian based Linux operating system like Ubuntu, but as I already wrote, general concepts can be applied to any Linux distribution.
Magento
Before proceeding further, you should visit your Magento 2 Admin -> Stores -> Advanced -> System -> Full Page Cache section, and switch caching application to Varnish. Additional options will appear, what you're interested in is button for exporting ready-made configuration file for Varnish version you will use. We'll use Varnish 3.0 branch, so lets click the "Export VCL for Varnish 3" button to export version 3.x compatible varnish.vcl
file and place it somewhere safe to be used for configuring Varnish later.
Nginx
So the idea of reverse proxy caching solution like Varnish is to hook it up to port 80 of your Magento store, and to configure your Nginx to listen on some other port (8080 in my case), and serve as data backend in case Varnish is not able to serve request at hand from its cache. Therefore the first thing you should do is unhook Nginx from port 80 by adjusting related virtual hosts file. Essentially you must change following directives:
# Other http context directives... server { # Other server context directives... listen 80 default_server; listen [::]:80 default_server; # Other server context directives... } # Other http context directives... |
into:
# Other http context directives... server { # Other server context directives... listen 8080 default_server; listen [::]:8080 default_server; # Other server context directives... } # Other http context directives... |
Besides adjusting virtual host that usually listens on port 80 in you Nginx configuration, there's one other thing requiring your attention - Varnish can not terminate SSL. In other words, Varnish is not capable of listening on port 443 and decrypting HTTPS traffic using your SSL certificate, therefore if you wish to secure areas of store in question, for example checkout and Magento 2 Admin, you'll need to offload SSL termination to another application. Since Nginx (as well as Apache) is fully capable of decrypting SSL traffic, we'll keep using it for that purpose, and we will then proxy decrypted HTTPS traffic to port 80 to be served by Nginx as usual. So here's how our port 443 Nginx virtual host looks like after required modifications.
# Other http context directives... server { listen 443; listen [::]:443; # Other server context directives... ssl on; ssl_certificate /etc/ssl/certs/certificate.crt; ssl_certificate_key /etc/ssl/private/private.key; ssl_protocols TLSv1.1 TLSv1.2; # Other ssl_* server context directives... # Proxy location / { proxy_pass http://127.0.0.1:80; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto https; proxy_set_header X-Forwarded-Port 443; proxy_set_header Host $host; } } # Other http context directives... |
That'll do the trick for the most part, but there's just this thing where Magento will not know if request at hand originally arrived to port 80 or port 443, this is important for generating store wide URLs using proper protocol. That's where those X-Forwarded
headers come into play - they are semi-standard headers used in order to pass some information about original request through our proxy. Even though Magento 2 could be monitoring the X-Forwarded-Proto
header in order to decide whether request in question is secure or not, this is not the case. Magento 2 observes HTTPS
server environment variable which is also semi-standard therefore it's required for us to modify our port 8080 Nginx virtual host to set HTTPS
to on
, whenever X-Forwarded-Proto
is https
. In order to do that we must populate flag variable using Nginx map functionality, and pass it along as HTTPS
server environment variable using fastcgi_param
. Therefore we will modify port 8080 Nginx virtual host by adding following outside of server
context:
# Other http context directives... # Vagrant - support HTTPS behind reverse proxy map $http_x_forwarded_proto $fastcgi_https { default $https; https on; } server { listen 8080 default_server; listen [::]:8080 default_server; # Other port 8080 directives... } |
This code will populate $fastcgi_https
variable with the default value of HTTPS
that Nginx handles on its own when not behind proxy, and with value on
when behind proxy and when X-Forwarded-Proto
is https
. Furthermore we must modify Nginx port 8080 virtual host location
block which handles PHP files using fastcgi_param
, in order to pass this flag to PHP $_SERVER
superglobal array used by Magento to correctly detect protocol scheme:
# Other server context directives... location ~ (index|get|static|report|404|503)\.php$ { # Other location context directives... # Support HTTPS behind reverse proxy fastcgi_param HTTPS $fastcgi_https; # Other location context directives... include fastcgi_params; } # Other server context directives... |
In order for our changes to take effect, we must test our Nginx configuration:
$ sudo nginx -t nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf test is successful |
If configuration test is successful, we can restart Nginx service:
$ sudo service nginx restart |
Varnish
Last thing we need to do is install and configure Varnish reverse proxy. First lets add official Varnish 3.0 branch repository for Debian based operating system at hand, Ubuntu 14.04 (codename Trusty) in my case:
wget -q https://repo.varnish-cache.org/GPG-key.txt -O- | sudo apt-key add - sudo add-apt-repository "deb https://repo.varnish-cache.org/ubuntu/ trusty varnish-3.0" |
Then we can proceed to installing Varnish:
$ sudo apt-get install varnish |
Next, you need to configure Varnish to listen on port 80 by adjusting it's daemon options as well as instruct Varnish daemon to use varnish.vcl
file we exported from Magento 2 Admin panel. First step is to use tool of your choice in order to copy varnish.vcl
file to your production server, to /etc/varnish/varnish.vcl
location.
Next, you need to adjust Varnish daemon options as described, and to restart Varnish.:
echo "DAEMON_OPTS=\"-a :80 -T localhost:6082 -f /etc/varnish/varnish.vcl -S /etc/varnish/secret -s malloc,256m\"" | sudo tee -a /etc/default/varnish sudo service varnish restart |
Now you just check if relevant services are up:
$ ps -e | egrep 'nginx|varnish' 1174 ? 00:00:00 varnishd 1175 ? 00:00:04 varnishd 1860 ? 00:00:00 nginx 1861 ? 00:00:00 nginx 1862 ? 00:00:00 nginx |
If you received output like the one from above, all systems are green and you're ready to check your Magento 2 frontend. For that purpose you can use varnishlog
which comes bundled with Varnish, more precisely command crafted to show only requests for which Varnish had to contact it's configured backend, in this case Magento 2 running on port 8080. When you load specific Magento 2 page, request will get cached by Varnish, and on the subsequent load for the same page, request will get served by Varnish without contacting Magento.
Here's first request for product page with http://magento2.loc/hero-hoodie.html
URL at my Magento 2 copy with sample data installed:
$ varnishlog -b | grep 'TxURL' 13 TxURL b /hero-hoodie.html 15 TxURL b /static/frontend/Magento/luma/en_US/mage/gallery/gallery.css 13 TxURL b /page_cache/block/esi/blocks/[%22catalog.topnav%22]/handles/[%22default%22,%22catalog_product_view%22,%22catalog_product_view_id_163%22,%22catalog_product_view_sku_MH07%22,%22catalog_product_view_type_configurable%22]/ . . . |
We can see that Varnish had to contact Magento 2 in order to obtain markup for /hero-hoodie.html
. And here's second request to this same URL, as monitored by varnishlog:
$ varnishlog -b | grep 'TxURL' 13 TxURL b /static/frontend/Magento/luma/en_US/Magento_Checkout/js/checkout-loader.js 15 TxURL b /static/frontend/Magento/luma/en_US/Magento_Ui/templates/block-loader.html 15 TxURL b /static/frontend/Magento/luma/en_US/mage/gallery/gallery.html . . . |
This time there's no /hero-hoodie.html
in output of our varnishlog command, this means that markup was served by Varnish without contacting Magento. Please note that content requested with Cache-Control
header of private
or no-store, no-cache
as well as static resources are not cached by Varnish.
Lastly, if you are interested in complete environment built around these concepts, especially if you are developer requiring Varnish setup for you local Magento 2 development, you can checkout my Magento 2 Vagrant box.
hello. I have some problems with this setup, I get 502 bad gateway.
I want my store to be full https://
in magento2 admin area, base url is https://domain.com
whereas you keep your server at 8080 port (which is http)
what should be the settings in admin area in this case?
secure url:
non secure url:
?? thank you!
What do you think about using Redis in place of Varnish?
Thank you so much!!!! I have spent hours trying to get this working and your explanation of the Nginx map functionality and $_SERVER superglobal solved it for me. Successfully implemented at: https://afrozania.com