I have been using Cloudflare for many years for their domain name server functionality, their SSL certificates and their content delivery network (CDN).
Cloudflare’s CDN perhaps does not boast as many configuration options as other CDN services, however it is free to use and their network boasts over 200 locations.
Over the years, I have taken a hands-off approach to Cloudflare’s content delivery network. Beyond enabling caching and defining a few basic settings, I have just allowed Cloudflare to manage everything for me.
For many people, Cloudflare’s simple approach to a CDN is appealing as other CDN services require you to manually set up push and pull zones.
Last week I upgraded to Cloudflare Pro to take advantage of their image optimisation tool Polish and their mobile acceleration tool Mirage.
Unfortunately, a few days later I noticed that Cloudflare was not caching my content or optimising my images.
This forced me to look more into how Cloudflare’s content delivery network works so that I could troubleshoot the issue.
In this article, I would like to explain the problem I had, show you how you can check the status of Cloudflare’s caching on your website and recommend some tools that you will find useful.
Hopefully these tips will help those of you who run into the same problem and need pointed in the right direction.
My Cloudflare Caching Problem
The main reason I opted to upgrade to Cloudflare Pro was to take advantage of Polish.
Polish is Cloudflare’s optimisation tool that can improve page loading times by reducing the file size of images.
If you choose lossless optimisation, metadata will be stripped from PNG, GIF and JPEG images, with no impact on the visual quality of the image.
Lossy optimisation will make files smaller, but there will be a noticable reduction in image quality.
The other feature of Polish is the ability to serve WebP images.
The WebP image codec was announced by Google way back in 2010, however it only started gaining mainstream support around 2019.
Google says that converting PNG images to WebP will reduce the file size by 45% and GIFs can lose 19% if you choose lossless and a whopping 64% if you choose lossy.
WebP images are still not supported by all browsers and Cloudflare cannot convert every image on your website if they have already been properly optimised, however WebP could be a great help to those of you who display many images on your website.
On this website, I have opted for lossless compression with WebP support.
A few days after I activated Polish, I noticed that not only were images not being optimised, but no images were being cached by Cloudflare at all.
This was baffling as I had not disabled the Polish feature on my Cloudflare dashboard.
I had noticed this issue shortly after switching from my test server to my production server for ten minutes in order to check some website design issues. This is simple to do as my hosting company Wetopi offers website staging.
After switching back to the new design, I noticed that blog posts were no longer getting good performance scores. Instead, I was being advised to serve scaled images and optimise my images.
I was genuinely confused about what was going on.
Before any of this happened, I had not really given much thought into how Cloudflare was optimising my images.
After reading a few articles about Polish, I discovered that:
- Cloudflare caches all images on your website
- Cached images are then optimised on Cloudflare’s servers
- The original images remain unoptimised on your website host
What this means in practice is that in order to deliver optimise images, Cloudflare needs to cache your website content.
Therefore, if there is an issue with caching your content, images cannot be optimised.
Understanding Key Cloudflare Caching Settings
When you’re a WordPress website owner, you become a little spoiled.
Technical problems rarely need to be tackled because developers have made WordPress plugins and themes user-friendly.
For example, when you change an important setting in a WordPress plugin, you will be advised the consequences of changing the setting and possibly be given other recommendations. Developers also integrate fail-safes to prevent you from creating errors.
This is an area where Cloudflare needs to improve upon.
With many Cloudflare features, you are kind of in the dark when something goes wrong.
Cloudflare have simplified the process of enabling and disabling features, but when you run into any problems, fixing the issue is not straightforward as you need to understand the significance of each Cloudflare setting and the downsides to using it.
OK, so how do you start troubleshooting an issue with Cloudflare caching?
As with many website problems, you first need to understand how everything works. You can then try and resolve any issues you face through trial and error.
I recommend starting with the caching information guides that are located in the Cloudflare support area.
The “Customizing Cloudflare’s cache” article will help introduce you to page rules, but you should read the full page rules tutorial to see examples of how they can be used.
You can use page rules to define exactly what is, and what is not, cached on your website.
Browser Cache TTL determines how long a visitor’s browser will cache your files.
The recommendations for this setting is all over the place, especially for WordPress users.
When you follow the Cloudflare integration process in WP Fastest Cache, the plugin sets Browser Cache TTL to six months. When you choose to optimise Cloudflare using the official Cloudflare WordPress plugin, it sets it to four hours.
I also tested “Respect Existing Headers” as Gijo Varghese, the developer behind Flying Analytics, recommends this in his article “Caching WordPress Pages using Cloudflare Page Rules” (I recommend browsing his blog WP Speed Matters as there are many great articles about WordPress optimisation).
What I was not initially aware of is that Cloudflare allows you to define browser cache duration in many different ways.
In “Understanding Origin Cache-Control“, Cloudflare states that:
Cloudflare respects whichever value is higher: the Browser Cache TTL in Cloudflare or the max-age header.
Cloudflare: Understanding Origin Cache-Control
This meant that when I had the Cloudflare Cache-Control max-age setting set to 604800 in WP Cloudflare Super Page Cache (the recommended value within the plugin), the value I declared on the main Cloudflare caching page was bypassed.
Later on the page, Cloudflare states that:
Browser Cache TTL Page Rules override max-age settings passed downstream from our edge, typically to your visitor’s browsers.
Cloudflare: Understanding Origin Cache-Control
In short:
- Cloudflare will look at the browser cache TTL value and the max-age header value and use the highest one
- These settings are overrided by any browser cache TTL values you define as a page rule for your website
The other value to understand is Edge Cache Expire TTL.
Edge cache expire TTL is the setting that controls how long CloudFlare’s edge servers will cache a resource before requesting a fresh copy from your server.
Cloudflare: Edge Cache Expire TTL: Easiest way to override any existing headers
One popular page rule you will see referenced frequently is “Cache Everything”. Normally, Cloudflare only caches static files such as Javascript, CSS and images.
With “Cache Everything”, Cloudflare will also cache HTML pages. This can be useful, but you do need to be careful when using this to ensure that private pages such as your website admin area aren’t cached.
So to summarise:
- Browser Cache TTL – Advises a visitor’s browser how long to cache files
- Edge Cache TTL – Advises Cloudflare when to request a fresh copy of files from your website server
- Cache Everything – Caches your whole website, not just static files
Do not be too concerned if all of this is a little confusing right now.
It starts to make more sense after reading all the caching guides and looking at suggested Cloudflare settings.
How to Troubleshoot Cloudflare Caching Issues
My first recommendation if you experience any problems with caching is to check Cloudflare’s network status website.
This shows you the operational status of Cloudflare data centres from around the world.
If there are no major network problems, the issue probably lies with either your Cloudflare caching settings or something in your website code that is preventing Cloudflare from caching your content.
The best way to verify this is to check your website headers.
When you use Cloudflare, the company inserts headers into your website code.
For example, cf-polished will be added when Cloudflare has used Polish to optimise an image.
There are many things you can check, however the two most important headers to check are CF-Cache-Status and Cache-Control.
CF-Cache-Status advises you whether Cloudflare has cached a file.
- CF-Cache-Status: HIT – File was served by Cloudflare
- CF-Cache-Status: MISS – File was served by your website host (i.e. your server)
- CF-Cache-Status: BYPASS – Not served by Cloudflare because you instructed them not to cache it
- CF-Cache-Status: DYNAMIC – A dynamic file that is not cached by Cloudflare by default
- CF-Cache-Status: EXPIRED – File was previously cached by Cloudflare, but has expired – Will probably be a HIT on the next attempt.
Dan Cruickshank’s article “Cloudflare’s CF-Cache-Status Headers Explained” has got a more expansive breakdown of what each status means. I recommend checking that out.
The other header you will need to understand is Cache-Control, which Cloudflare also refers to as “Original Cache Control”.
This header will inform you how Cloudflare is currently caching a file.
Max-age, for example, is defined in seconds and indicates when a response is stale. Therefore, cache-control: max-age=315360000, would mean that the file is stale after one day.
In summary:
- CF-Cache-Status: Advises you whether a file is being cached
- Cache-Control: Advises you the cache settings for that file
Let’s move on and look at how you can use these headers to get a better understanding of what is happening on your website.
Using the Developer Console (F12)
Every browser includes a developer console that allows you to troubleshoot issues with your website.
This can be accessed via the browser settings area, however it is quicker to launch it by entering F12.
Once the developer console has loaded, you need to refresh the page so that the tabs populate.
Headers for all files can be checked via the network tab.
For each file, you will see the status of Cache-Control, CF-Cache-Status and more.
Due to the nature of caching, you need to take a few steps to ensure you are viewing the latest version of your website.
On a WordPress website, for example, you should clear the cache of any WordPress caching plugin you have activated, particularly if you have applied changes recently. Cloudflare’s cache should be cleared after this step too.
Another thing to be mindful of is browser cache.
If you view your website in your regular browser, the developer console may show you an outdated version of your website that is stored on your computer.
One way to resolve this is to clear your browser cache for your website, but a more practical solution is to view your website in incognito mode and then load the developer console.
Checking Status with CF-Cache-Status
Another way to check the status of your website is to use the CF-Cache-Status tool.
This website will let you know if your website is properly configured to support Cloudflare.
All you have to do is enter a URL to perform a test.
In a standard Cloudflare caching setup, you can test images and other static files such as Javascript.
If you have setup a page rule to cache everything, you can also use CF-Cache-Status to test full pages such as your home page.
CF-Cache-Status highlights whether a file has been cached using the five statuses I mentioned earlier: Hit, Miss, Bypass, Dynamic and Expired.
Additional headers such as Cache-Control are also displayed.
You may get an expired message when you first use CF-Cache-Status, especially if you have recently purged your website’s cache.
When a user requests content from a website using a CDN, the CDN fetches that content from an origin server, and then saves a copy of the content for future requests. Cached content remains in the CDN cache as long as users continue to request it.
Cloudflare: How does content become cached?
This means that if you have purged your Cloudflare cache and no one has since visited that URL, there will be no copy of the page or files stored on Cloudflare.
This is why CF-Cache-Status regularly displays an EXPIRED message the first time to test a URL and then shows a HIT message once you test again.
CF-Cache-Status is a useful service that I recommend checking out.
As you are aware, you can check multiple files using your browser’s developer console.
With CF-Cache-Status, you can only view the status of one URL at a time, however it can save you some time as you do not need to worry about clearing your browser’s cache in order to do checks.
The Dr. Flare Chrome Extension
If you want the useful information from the developer console presented in a more digestible format, you may want to consider installing the Dr. Flare Google Chrome Extension.
This fantastic extension helps you see exactly where images and other files are being hosted.
To use the extension, you need to open up your browser’s developer console by hitting F12 and then navigate to the Dr. Flare tab.
By default, Google Chrome extensions are not enabled in incognito mode.
You can allow Dr. Flare to be used in incognito mode by clicking on the menu button and selecting more tools, extensions and then the Dr. Flare details button. Then simply allow the app to be used in incognito.
Unfortunately, Dr. Flare did not work correctly for me when used in incognito mode, therefore the only way I could effectively use the extension was to clear my browser’s cache.
Once you have opened up the developer console and navigated to the Dr. Flare tab, you will see images that are hosted by Cloudflare marked green and images hosted on your own server marked red.
It also marks external images as grey.
When you hover over an image that is marked green, you will see which Cloudflare features are used. For my cached images, it highlighted some images had utilised Polish.
When you hover over red images, you will see a message saying that it is missed. This means the image is being delivered by your own website.
If you expand the Dr. Flare console window, you will see a huge amount of data which shows what is cached, what is not cached, what is proxied and more.
The main overview page steals the limelight with its colourful pie charts and bar charts, however there are several tabs that detail exactly what is going on.
This gives you a lot of information that you just don’t see on Cloudflare, however I did notice that Dr. Flare said the Mirage feature was off, despite being enabled on Cloudflare.
As you can see from the screenshot below, Dr. Flare showed that only 3 files were being cached by Cloudflare. This confirmed that my content was not being cached correctly.
Later, when I had resolved my Cloudflare caching problem, Dr. Flare correctly showed 20 images and static files being cached as well as the number of dynamic files not being cached.
If I had used the “Cache Everything” page rule, the number of cached files would have been significantly higher.
Whilst I do love the breakdown in the main dashboard of Dr. Flare, if you are troubleshooting your website’s Cloudflare caching problem, you should open up the tabs and look at the information that is available for each file.
For each file you can see the request timings, request headers and response headers.
This information is vital when you are reviewing whether the settings you have configured on Cloudflare are correct.
I found Dr. Flare to be incredibly useful during the troubleshooting process as the extension takes all the information you will find in the network tab of the developer console and publishes it in a more presentable way.
Be sure to give it a try.
Configuring Cloudflare Caching in WordPress
Cloudflare is incredibly popular with WordPress users and there is great support for it in WordPress caching plugins and optimisation plugins.
There are many WordPress plugins designed to help you get the most out of Cloudflare too, but in this section I would like to share with you two WordPress plugins that will help you configure your WordPress website correctly for Cloudflare.
Both plugins are free to download, however please remember that neither solution is necessary to use Cloudflare’s CDN.
The Official Cloudflare WordPress plugin
The official Cloudflare WordPress plugin allows you to connect your WordPress website directly to Cloudflare using the API key that is located within your Cloudflare account area.
It was quite buggy in the past, however the developers appear to have resolved those issues.
The plugin has three pages: Home, Settings and Analytics.
Attack mode can be enabled from the top of each page too.
On the home page you can optimise Cloudflare for your website, purge cache and enable a feature where your Cloudflare cache is purged whenever you update the appearance of your website.
When you click to optimise Cloudflare for WordPress, the following settings will be applied to your website in your Cloudflare account:
| Security level | Medium |
| Caching level | Standard |
| Auto Minify | Enable Auto Minify for JS, CSS, and HTML |
| Browser Cache TTL | 4 hours |
| Always Online | On |
| Development Mode | Disabled |
| IPV6 Compatibility | Off |
| WebSockets | On |
| IP Geolocation | On |
| Email Address Obfuscation | On |
| Server-side Excludes | On |
| Hotlink Protection | Off |
| Image optimization (Polish and Mirage) | Off (unless on Pro or higher plan) |
| Rocket Loader | Off |
It is important to be aware that these settings are automatically applied when you click on the optimisation button and will override your existing Cloudflare settings.
Generally speaking, I believe these settings are a good starting point for configuring Cloudflare for WordPress, however you may have to adjust these settings so that Cloudflare works with other optimisation plugins you use.
For example, if you are using a caching plugin on your website, you may want to consider disabling minification for JS, CSS, and HTML, as your caching plugin may already be doing this.
The next page displays speed and security settings.
In the speed settings area, you can enable Cloudflare’s “Always Online” feature and image optimisation.
The development mode feature lets you bypass caching, which is useful when troubleshooting a host of issues on your website.
Business plan customers can also enable HTML caching from here.
In the security settings area, you can set your security level, enable the web application firewall and enable automattic HTTPS rewrites.
Advance DDoS options are available in this area for business plan customers.
On the analytics page you can view requests, bandwidth, unique visitors, threats, traffic served over SSL and bandwidth saved.
This data is just presented as a brief snapshot so you will have to login to your main Cloudflare account in order to filter dates and see full traffic information.
For the most part, I rarely check traffic stats or change settings through the official Cloudflare WordPress plugin, however it is worth installing the plugin so that you can purge cache directly from your website.
The optimisation feature is useful too as it will configure Cloudflare for WordPress.
If you wish, you can uninstall the plugin after you have processed the optimisations, as the settings will remain live on your Cloudflare account.
Controlling Caching with WP Cloudflare Super Page Cache
When troubleshooting my website’s caching problem, I read about the topic of managing cache control and ensuring that expire headers were being set correctly on websites.
This led me towards a useful WordPress plugin called WP Cloudflare Super Page Cache.
Developed by Salvatore Fresta of SpeedyWP, WP Cloudflare Super Page Cache allows you to use Cloudflare’s “Cache Everything” rule.
This feature can be enabled in Cloudflare using page rules, however WP Cloudflare Super Page Cache simplifies the task of setting cache expiration dates and bypassing caching of the WordPress admin area.
WP Cloudflare Super Page has over a dozen settings for controlling the lifetime of cache and what areas of your website are included and excluded.
All you have to do is connect to your Cloudflare account using your Cloudflare API key and then select how you want things setup.
Like the official Cloudflare WordPress plugin, there is an option to purge cache. WP Cloudflare Super Page extends this by offering an option to test cache and an option to purge cache from the WordPress admin bar.
Some WordPress caching plugins connect to Cloudflare using the Cloudflare API so that your Cloudflare settings can be adjusted, whilst others simply optimise your WordPress settings for Cloudflare, but no other solution gives you the same level of control that WP Cloudflare Super Page does.
It lets you define exactly what is cached and what is not, without having to create complex page rules.
You can define cache lifetime settings, cache behaviour settings and browser caching settings.
Last week, an update adding additional settings for WooCommerce, W3 Total Cache, WP Rocket and WP Super Cache, that gives you greater control over what is excluded from caching.
WP Cloudflare Super Page Cache is worth checking out if you want to take advantage of Cloudflare’s “Cache Everything” feature and cache dynamic content that is normally excluded by Cloudflare.
Cache Everything can be setup using page rules if you wish, but is is not as user-friendly or as customisable as WP Cloudflare Super Page Cache.
For the time being, I have decided not to cache non-static files on this website as it means that I have to clear Cloudflare’s cache whenever I make any change to my website.
I will therefore not need to use WP Cloudflare Super Page Cache at the moment.
How I Resolved My Cloudflare Caching Problem
I was always optimistic about resolving the caching issues on my website, yet throughout my research, I kept asking “What was causing all of this?”.
There are a few things I reviewed when I was troubleshooting the issue.
Firstly, I checked my WordPress .htaccess file and noticed a few problems there.
For the benefit of my upcoming website optimisation article, I had tested over a dozen caching WordPress plugins in order to determine the best solution. Unfortunately, even when I deactivated these plugins, a few of them had left behind unused code in my website’s .htaccess file.
So the .htaccess file was a mess and when I made my staging area went live, all of this unnecessary code was transferred to my live website.
This additional code was easy to tidy up, but tidying my .htaccess file did not fix my caching problem.
I then came across a help article entitled “Cloud Flare not caching images when cache-control is public” which talked about images having their CF-Cache-Status set to MISS.
This can sometimes be caused by JavaScript doing image handling or manipulation. This can be tested by disabling JavaScript in your browser and doing a hard refresh of the page using crtl+F5.
Episerver: Cloud Flare not caching images when cache-control is public
If doing this gives you a CF-Cache-Status header of “HIT” then there is something in the JavaScript that is causing CloudFlare to not serve a cached copy of the image, you will need to do a code review on your JavaScript to identify what might be causing this.
After reading this, I took a step back and thought about all the WordPress optimisation plugins I had installed which were combining Javascript files and deferring their loading.
So I decided to deactivate Async JavaScript, Autoptimize and WP Fastest Cache.
I then re-activated one WordPress plugin and once I saw files returning HIT for their CF-Cache-Status, I would activate the next one and repeat the checks.
This led me to resolving my website’s Cloudflare caching problem.
Whilst I do not know exactly what was causing the caching problem to arise, my guess is that it was a configuration issue stemming from one of the WordPress optimisations I had installed.
The mistake I made was to try and tweak my existing website settings and Cloudflare settings.
If I had to face this problem again, what I would do is deactivate all WordPress optimisation plugins and reset my Cloudflare configuration settings using the official Cloudflare WordPress plugin.
I would then adjust settings and verify caching is still working after each modification.
Final Thoughts
I hope you have found this guide useful.
Cloudflare is a simple service to use, however my experience over the last week has reminded me that the main dashboard in Cloudflare does not help you troubleshoot any problems.
Unfortunately, Cloudflare did little to help me with this issue.
Response times for Cloudflare pro customers is supposed to be an average of four hours, however they did not reply to my support ticket for five days and by the time they responded, I had already resolved the issue myself after days of troubleshooting.
Hopefully, if you face a similar issue with Cloudflare correctly caching your website, you will now be better prepared to tackle the problem.
For further help, I recommend browsing the Cloudflare community forums as there are many discussions there about common Cloudflare configuration problems.
Good luck.
Kevin