Bogus Link

Varnish Tips: Don't Cache 404 Responses

  • Server Tuning
  • Website Performance
  • Linux
  • Server Management

Introduction

Varnish Cache is an awesome piece of software, but it's not designed to work straight "out of the box". Although a lot of effort has been put in to providing sensible defaults, there is just too much difference between websites' requirements and configurations to expect it to work without learning about it and tuning it to your needs. Our series of "Varnish Tips" aims to assist you in your quest to master Varnish Cache and get the most out of this amazing piece of software.

Today's tip deals with how to prevent Varnish from caching 404 responses. For my needs, I prefer to never cache them, since they are trivial in terms of file size, and can occur simply because of a mistake (that I would prefer not to last beyond it being resolved). It also means all 404 requests get handled by the back-end and can be logged there (for example, Joomla handles and records 404s to assist with resolving them - having a count of the visits is helpful for deciding which ones need redirecting). If you prefer to cache them for a short period of time, then an option is provided for that too.

Details

Preventing Varnish from caching 404 responses is easy. If you are using version 4 or later, which you can test with varnishd -V on the command line (and you should be, because previous versions are beyond end of life), then the following code will do it.

sub vcl_backend_response {
    # Don't cache 404 responses
    if ( beresp.status == 404 ) {
        set beresp.ttl = 120s;
        set beresp.uncacheable = true;
        return (deliver);
    }
}

This code should be added at, or near, the top of your vcl_backend_response section as shown. What this code does is tell Varnish that any backend response with a 404 status should not be cached, and instead a "hit-for-pass" object should be stored in the cache, so requests in the next two minutes get passed straight through to the backend. Please see the references for more information on these choices and to see if this is right for you.

Alternative Approach

With the built-in response, which gets executed as long as no return statement is executed later in the vcl_backend_response, the above will happen anyway, so you can achieve the same result with simply:

sub vcl_backend_response {
    # Don't cache 404 responses
    if ( beresp.status == 404 ) {
        set beresp.ttl = 0s;
    }
}

However, I prefer to be explicit with what is supposed to happen, so any subsequent code will not change it. Again, please consider referring to the references and make your own choices on this.

Another Option - Short Cache Time

You may prefer to cache 404 responses for a short period of time. Perhaps you have a busy site and don't want them to be generated by the back-end too often. You can achieve this with the following code:

sub vcl_backend_response {
    # Don't cache 404 responses
    if ( beresp.status == 404 ) {
        set beresp.ttl = 30s;
    }
}

This sets a TTL (time-to-live) of 30 seconds, so 404 responses will be cached for that time.

Conclusion

And there we have it, an overview of how you might approach handling 404s with Varnish Cache. Please do let us know your experiences in the comments, and of course any errors you may have spotted. We are always looking to improve these articles and keep them up to date.


References

Disclaimer

The information in this article is provided "as is" and is not a substitute for your own competence. If you do not know what you are doing, and do not fully understand what the advice in this article is explaining and its implications, please do not make use of it without further research and training yourself, or seeking the help of a professional. The consequences of any actions you take, that are in any way informed by advice in this article, are your own to bear.

Professional Assistance

If you are in need of professional assistance to carry out a procedure similar to that outlined in this article, or have another system administration issue you need help with, please contact us and we will be happy to discuss options for assisting you.

Comments

We would love to hear from you about your experiences relating to the information in this article. If you have seen something here that needs correcting, please let us know so we can improve the information.

About Comments You Submit

Comments can be made using the form above, and will be posted here if appropriate for us to do so. Please let us know in your comment if you would prefer us not to publish it, in which case we will of course honour that request. Your email address will only be used to contact you regarding your comment, and will not be passed on to third parties or used in any other way. Any web address you provide will be used as a link from your name introducing your comment, if appropriate for us to do so.

Copyright and Republishing

This article is copyright Nigel Peck 2017. All rights reserved. Please contact us if you would like to republish this article, in whole or in part, and we will be happy to discuss options for that. Quoting from the article is permitted. Please provide a link back to the article, adjacent to the quote, and clearly indicate any modifications. Quoting anything more than two or three sentences should be discussed with us first.