Varnish Cache is a popular reverse proxy web application accelerator. It generally operates by caching generated pages in memory. Recently, version 4.0 was released which at a high level has the following enhancements:
Source: https://www.varnish-cache.org/content/varnish-cache-400
There were many changes to the VCL language and to some of the parameters that the “varnishd” binary accepts. This is not meant to be a comprehensive guide but lessons learned from converting a working “default.vcl” to the new VCL 4.0 syntax. I do not expect my previous “default.vcl” file to have been perfect either, but I did do some further optimization some while working on this conversion. Also I am running NGINX as my backend but none of this is specific to using that as the backend web server.
Topics Covered:
Please note that one big change is that “builtin.vcl” is automatically appended now to the user “default.vcl”. For reference, I am including the “builtin.vcl” at the end of this post of which I was only able to find by looking at the GitHub repository.
For Varnish Cache to know that it is using the new VCL 4.0 format use the following at the head of your default.vcl configuration.
# Required for VCL 4.0 vcl 4.0;
Note: I do not believe you can use VCL 3.4 syntax in Varnish Cache 4.0 by declaring a different version, but I am not positive.
This is not technically required but if you choose to use a probe and a check to see if the backend is healthy then you need to import the standard VMOD.
# Import VMod's import std;
One of the two new features available are that of limiting commands in the Varnish Admin CLI and limiting the use of inline-C code in you VCL files.
To limit commands available for use in the Varnish Admin CLI, add the following argument to your “varnishd” startup script (following parameters after the “-r” are an example of several commands to limit, mostly recommended by the linked blog entry):
-r cc_command,listen_address,user,group,vcc_allow_inline_c
The other security feature available is that, by default, inline-C code is disabled in VCL. Therefore to re-enable you need to do the following:
-p vcc_allow_inline_c=on
The only issue with the above is that I cannot find a way to actually make it work. I can find it in the source code and in theory it looks like it will work but in practice it does not appear to. In addition, I have tried “true”, 1, “on” and nothing works so far. I chose to put “on” in the example as that was what was in the source code.
One change that is not obvious is that the following will not work anymore:
ban url ~ EXPRESSION
The correct syntax is the following:
ban req.url ~ EXPRESSION
Source: https://raw.githubusercontent.com/varnish/Varnish-Cache/master/bin/varnishd/builtin.vcl
/*- * Copyright (c) 2006 Verdens Gang AS * Copyright (c) 2006-2014 Varnish Software AS * All rights reserved. * * Poul-Henning Kamp <[email protected]> * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * * The built-in (previously called default) VCL code. * * NB! You do NOT need to copy & paste all of these functions into your * own vcl code, if you do not provide a definition of one of these * functions, the compiler will automatically fall back to the default * code from this file. * * This code will be prefixed with a backend declaration built from the * -b argument. */ vcl 4.0; ####################################################################### # Client side sub vcl_recv { if (req.method != "GET" && req.method != "HEAD" && req.method != "PUT" && req.method != "POST" && req.method != "TRACE" && req.method != "OPTIONS" && req.method != "DELETE") { /* Non-RFC2616 or CONNECT which is weird. */ return (pipe); } /* We don't support chunked uploads, except when piping. */ if ((req.method == "POST" || req.method == "PUT") && req.http.transfer-encoding ~ "chunked") { return(pipe); } if (req.method != "GET" && req.method != "HEAD") { /* We only deal with GET and HEAD by default */ return (pass); } if (req.http.Authorization || req.http.Cookie) { /* Not cacheable by default */ return (pass); } return (hash); } sub vcl_pipe { # By default Connection: close is set on all piped requests, to stop # connection reuse from sending future requests directly to the # (potentially) wrong backend. If you do want this to happen, you can undo # it here. # unset bereq.http.connection; return (pipe); } sub vcl_pass { return (fetch); } sub vcl_hash { hash_data(req.url); if (req.http.host) { hash_data(req.http.host); } else { hash_data(server.ip); } return (lookup); } sub vcl_purge { return (synth(200, "Purged")); } sub vcl_hit { if (obj.ttl >= 0s) { // A pure unadultered hit, deliver it return (deliver); } if (obj.ttl + obj.grace > 0s) { // Object is in grace, deliver it // Automatically triggers a background fetch return (deliver); } // fetch & deliver once we get the result return (fetch); } sub vcl_miss { return (fetch); } sub vcl_deliver { return (deliver); } /* * We can come here "invisibly" with the following errors: 413, 417 & 503 */ sub vcl_synth { set resp.http.Content-Type = "text/html; charset=utf-8"; set resp.http.Retry-After = "5"; synthetic( {" <?xml version="1.0" encoding="utf-8"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <html> <head> <title>"} + resp.status + " " + resp.reason + {"</title> </head> <body> <h1>Error "} + resp.status + " " + resp.reason + {"</h1> <p>"} + resp.reason + {"</p> <h3>Guru Meditation:</h3> <p>XID: "} + req.xid + {"</p> <hr> <p>Varnish cache server</p> </body> </html> "} ); return (deliver); } ####################################################################### # Backend Fetch sub vcl_backend_fetch { return (fetch); } sub vcl_backend_response { if (beresp.ttl <= 0s || beresp.http.Set-Cookie || beresp.http.Surrogate-control ~ "no-store" || (!beresp.http.Surrogate-Control && beresp.http.Cache-Control ~ "no-cache|no-store|private") || beresp.http.Vary == "*") { /* * Mark as "Hit-For-Pass" for the next 2 minutes */ set beresp.ttl = 120s; set beresp.uncacheable = true; } return (deliver); } sub vcl_backend_error { set beresp.http.Content-Type = "text/html; charset=utf-8"; set beresp.http.Retry-After = "5"; synthetic( {" <?xml version="1.0" encoding="utf-8"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> <html> <head> <title>"} + beresp.status + " " + beresp.reason + {"</title> </head> <body> <h1>Error "} + beresp.status + " " + beresp.reason + {"</h1> <p>"} + beresp.reason + {"</p> <h3>Guru Meditation:</h3> <p>XID: "} + bereq.xid + {"</p> <hr> <p>Varnish cache server</p> </body> </html> "} ); return (deliver); } ####################################################################### # Housekeeping sub vcl_init { return (ok); } sub vcl_fini { return (ok); }