Cloudflare Worker recipes for frontend performance testing

Table of contents

Cloudflare Workers (CFW) (or Workers) have very much become part of my workflow when it comes to examining and ultimately improving frontend web performance (hopefully!). I use them both for work and personal projects. This post I’ve mainly written for myself, as I’ve found I’ve been repeating certain recipes over and over again. So it feels like a good idea to have a place to log them for reference in the future. I’d love to get feedback on what could be improved, or any other examples you may have. If you do, please do Let me know!

Note: If you are unsure where to start with these recipes I’ve written a step by step guide on how to get started with Cloudflare Workers here.

What are CFW’s?

Workers are a serverless offering from Cloudflare built upon their Content Delivery Network (CDN). They provide a lightweight JavaScript execution environment built on the v8 JavaScript engine. There are many uses for Workers. Many examples are listed on the Worker documentation page, including How to build a Slackbot, and How to build a QR code generator. But I’m going to focus on a particular area that I find them useful for: frontend web performance.

Why only frontend web performance?

Back in February 2012, Steve Souders published his now pretty famous blog post ‘the Performance Golden Rule’. To quote from the post:

80-90% of the end-user response time is spent on the frontend. [So] Start there.

I highly recommend reading the post, but a quick TL;DR; is essentially: once the server has received a request, done all the database lookups, generated the HTML, and delivered the response to the browser, the rest of the page performance now depends on how it is handled on the frontend. So if you want to make large web performance gains, then focusing on the frontend is a way to do it. I’m sure there are backend optimisations you can explore with Workers, but I won’t be focussing on them in this blog post (although I’d love to hear all about them).

Practical use case

To give you a practical example of where Workers come in useful I’ll look at some of the work I do at Government Digital Service (GDS). The department has built and maintains GOV.UK. It is the website for the UK government. It’s the best place to find policy, announcements, information about the government, and guidance for citizens. Since 2012 it has replaced 1,884 government websites with just one, to become the home of all central government’s online content and services.

What I’m trying to tell you is that it’s a big and complicated bit of software, both on the backend and frontend. The frontend that users see is made up of approximately 13 separate applications, and that’s not including any of the publishing applications. So what would be seen as a “small change” for a small static site, can actually turn out to be a huge change for a large complex site of its size. This is an issue pretty much every organisation and developer will run up against at some point. I’m sure many readers have been in the position where they’ve said: “If only we could change this small piece of code, and examine the difference it makes”, knowing full well that the change is potentially day’s worth of work (if not more).

What we need is a way to experiment with changing production code, without actually making any changes to production code. It doesn’t sound possible does it? But that’s exactly what we can do with Workers. The Worker can sit between the production server and our browser and make changes to server responses on the fly. We can then observe the difference these response changes make to performance. Here’s a tweet about an example I tried with GOV.UK, examining what difference the position of the JavaScript in the HTML source and the loading mechanism used (e.g. async / defer) has on performance. As I mention in the tweet thread, this set of changes would have taken much longer than the hour it took to create the experiments!

Boilerplate code

This code is all based off what Andy Davies has written in his excellent blog post all about the subject: ‘Exploring Site Speed Optimisations With WebPageTest and Cloudflare Workers’, which itself was based off the work Patrick Meenan mentions in his ‘Getting the most out of WebPageTest’ talk (slide 60). So with this post I really am kneeling on the shoulders of giants!

The heavily commented code that all these recipes are based off can be found in this gist.

Code quality

At this point it’s probably worth mentioning that the code used in the examples is pretty rough and ready “throwaway-code”. It’s simply there to transform certain server responses, rather than being production ready code. That being said, you should always try to login to the Cloudflare admin area and observe the CPU time that your worker is taking during execution. The free worker plan allows for up to 10ms CPU time per request. If you are going over these 10ms I’d imagine throttling will kick in. This is also a sure sign there are potential inefficiencies in the code which may lead to unrealistic final results (due to increased latency).

For example, I was recently looking to see what a really large HTML page can have on the CPU time in a Worker. I picked the NASA Astronomy Picture of the Day full Archive which returns 710 KB of uncompressed HTML. For certain DOM manipulation operations I managed to get it to spike to 35ms. And really it’s not a complicated DOM at all, it’s just large:

In the graph we see the spike in CPU time hitting 35ms for a single request.

So it’s worth checking the CPU graph if you notice latency issues during testing.

Recipes

Below you will find a set of recipes I’ve found useful when performance testing.

Checking all requests flow through the worker

The first thing that’s worth checking is that all the expected responses for your test site are actually flowing through your worker. There are cases when this may not be the case. For example, if some of your requests use relative URLs, and others use absolute URLs. Your absolute URLs will likely be pointing to the original site. This could lead to less accurate results since your browser will be creating a TCP connection to both the worker URL and the original server. Thankfully this pretty simple to fix using the HTMLRewriter API and the ‘Rewrite links’ example.

// domain rewriter vars
const OLD_URL = "https://www.example.com/";
const NEW_URL = "/";
//...
if(acceptHeader && acceptHeader.indexOf('text/html') >= 0){
  // store this particular HTML response for modification
  let oldResponse = await fetch(url.toString(), request)
  // create a new response
  let newResponse = new HTMLRewriter()
      // rewrite the links from the following page elements
      .on("a", new AttributeRewriter("href"))
      .on("link", new AttributeRewriter("href"))
      .on("img", new AttributeRewriter("src"))
      .on("script", new AttributeRewriter("src"))
      .on("image", new AttributeRewriter("src"))
      .on("meta", new AttributeRewriter("content"))
      // transform the old response
      .transform(oldResponse)

  // return the modified page
  return newResponse
}
//...
// https://developers.cloudflare.com/workers/examples/rewrite-links
class AttributeRewriter {
    constructor(attributeName) {
        this.attributeName = attributeName
    }
    element(element) {
        const attribute = element.getAttribute(this.attributeName)
        if (attribute) {
            element.setAttribute(
                this.attributeName,
                attribute.replace(OLD_URL, NEW_URL),
            )
        }
    }
}

In the above example we are picking out selected elements in the page and making sure they all use relative URLs. Note: you should only do this for assets that actually exist on your sites server defined in the site variable, else you’re just going to get a lot of 404’s!

Code: Complete code for this example can be seen in this gist.

Adding resource hints to the page <head>

In this example we’re going to look at how we can modify our page <head> to add a few resource hints, and give the browser information about the page before it is fully parsed. There are many articles available to read about how these tags can help improve the performance of a website.

var resourceHints = `
<link rel="preload" href="/assets/font/font1.woff2" as="font" type="font/woff2" crossorigin="anonymous">
<link rel="dns-prefetch" href="https://fonts.gstatic.com/">
<link href="https://cdn.domain.com" rel="preconnect" crossorigin>
`
//...
if(acceptHeader && acceptHeader.indexOf('text/html') >= 0){
  // store this particular HTML response for modification
  let oldResponse = await fetch(url.toString(), request)
  // create a new response
  let newResponse = new HTMLRewriter()
    // add listed resource hints to the page head
    .on('head', new addResourceHints())
    // transform the old response
    .transform(oldResponse)

    // return the modified page
    return newResponse
}
//...
class addResourceHints {
    element(element) {
        // notice how we are prepending the hints, right after the opening head tag
        // can be changed to append if you want them right before the closing tag
        element.prepend(resourceHints, {html: true});
    }
}

Notice how we have different resource hints being added. In the example I’ve added a preload, dns-prefetch, and a preconnect. You can add as many or as few as you need. Also note how these tags are being added directly after the opening <head> tag. You can change this too if you so wish (e.g. using .append()).

Code: Complete code for this example can be seen in this gist.

So we’ve added resource hints by modifying the HTML in the <head>, but it is also possible to use a response header instead. This is quite simple to do with a Worker, but there are a couple of caveats:

  • Be careful of absolute / relative URLs with your preloaded assets. Make sure the URL in the preload matches how the asset would usually load in the page. If they don’t match you will end up preloading and loading the asset twice (e.g. a font is preloaded and then also loaded through the usual page load).
  • If using preload make sure to include the ‘nopush’ string. Cloudflare converts a link preload to a HTTP/2 server push by default. This may (or may not) be the functionality you actually want.

In the below example I’ve also allowed for both HTML changes as well as modification of the response headers.

//...
if(acceptHeader && acceptHeader.indexOf('text/html') >= 0){
  // store this particular request for modification
  let oldResponse = await fetch(url.toString(), request)
  // create a new response
  let newResponse = new HTMLRewriter()
  /**
    * Insert any HTML modifications here
    */
  // apply above modifications to the response
  .transform(oldResponse)

  // Make the headers mutable by re-constructing the already modified response.
  let response = new Response(newResponse.body, newResponse)
  // add our Link headers
  response.headers.set("Link", '</assets/fonts/font-file-1.woff2>;rel="preload";as="font";crossorigin; nopush, </assets/js/js-file-1.js>;rel="preload";as="script"; nopush');
  // return the modified page along with custom headers
  return response
}
//...

Code: Complete code for this example can be seen in this gist.

Removing resource hints from the page <head>

I’ve given you an example of how to add resource hints to the <head> so I may as well give you an example of how to remove them too. Remember you can be very specific as to which ones you remove by making use of Attribute selectors:

//...
if(acceptHeader && acceptHeader.indexOf('text/html') >= 0){
    // store this particular request for modification
    let oldResponse = await fetch(url.toString(), request)
    // create a new response
    let newResponse = new HTMLRewriter()
        // blanket example of removing all resource hints
        .on("link[rel='preload']", new removeElement())
        .on("link[rel='prefetch']", new removeElement())
        .on("link[rel='dns-prefetch']", new removeElement())
        .on("link[rel='prerender']", new removeElement())
        .on("link[rel='preconnect']", new removeElement())
        // example were we only remove a selected preload hint for a font
        .on("link[rel='preload'][href*='our-woff2-font.woff2']", new removeElement())
        .transform(oldResponse)

    // return the modified page
    return newResponse
}
//...
class removeElement {
  element(element) {
    element.remove();
  }
}

Code: Complete code for this example can be seen in this gist.

Removing resource hints headers

So above we added the Link header for our resource hints, but what about if our server is already serving them and you want to test what difference removing them makes? Well it’s pretty much identical to the above code only we use the delete() method instead of set():

if(acceptHeader && acceptHeader.indexOf('text/html') >= 0){
    // store this particular request for modification
    let oldResponse = await fetch(url.toString(), request)
    // create a new response
    let newResponse = new HTMLRewriter()
    // remove all external scripts from the page
    /**
      * Make your HTML changes here
      */
    .transform(oldResponse)

    // Make the headers mutable by re-constructing the already modified response.
    let response = new Response(newResponse.body, newResponse)
    // delete our Link header(s)
    response.headers.delete("Link");
    // return the modified page along with custom headers
    return response
}

Code: Complete code for this example can be seen in this gist.

Modifying CSS and JavaScript response bodies

The great thing about CFWs is you can easily make changes to responses for other types of responses too. For example, in your Worker you could look for CSS and JavaScript files and make changes to their response bodies. You could quite easily remove whole sections of JavaScript code, or add in brand new selectors to the CSS and observe the results in the page. In the simple example below we are:

  • replacing a common CSS string (the font-family property value)
  • adding a simple console.log() to the end of a JavaScript file
//...
if(acceptHeader && acceptHeader.indexOf('text/html') >= 0){
  // store this particular request for modification
  let oldResponse = await fetch(url.toString(), request)
  // create a new response
  let newResponse = new HTMLRewriter()
    /**
      * Make your HTML changes here
      */
    .transform(oldResponse)

    // return the modified page along with custom headers
    return newResponse
} else if(acceptHeader && acceptHeader.indexOf('text/css') >= 0){// Change CSS here
    // grab the CSS response
    const response = await fetch(url.toString(), request);
    // extract the body of the request
    let body = await response.text();
    // modify the CSS response body
    body = body.replace(/Arial, Helvetica Neue, Helvetica, sans-serif;/gi,'Georgia, Times, Times New Roman, serif;').replace(/Arial,Helvetica Neue,Helvetica,sans-serif;/gi,'Georgia, Times, Times New Roman, serif;')
    // return the modified response
    return new Response(body, {
        headers: response.headers
    });
} else if(acceptHeader && acceptHeader.indexOf('*/*') >= 0){// Change JavaScript here (uses the generic Accept directive)
  // being granular we only modify a single response for a specific JavaScript file
  if(url.toString().includes('our-specific-js-filename.js')){
    // grab the JS response
    const response = await fetch(url.toString(), request);
    // extract the JS body of the request
    var body = await response.text();
    // using template literals we add a console.log to the end
    body = `${body} console.log('String added last');`;
    // return the modified response
    return new Response(body, {
      headers: response.headers
    });
  }
}
//...

There’s a caveat to the above code. The Accept header for scripts in all modern browsers is */*, so I’ve specifically targeted a single JavaScript file. You can easily do the same with the CSS using the same method.

Also, when you get to the point of var body = await response.text();, what you do next basically comes down to a JavaScript string manipulation exercise. You could easily add / remove / insert code into the files using the basic string manipulation methods.

Be sure to check to see if your <script> or <link> tags don’t have an integrity attribute. If they do you have Subresource Integrity enabled on your JavaScript and CSS assets. If you modify these files on the fly like in the example above, the hash of the file will have changed and the browser will refuse to load it. If that’s the case I’d remove the integrity attribute using the HTMLRewriter at the same time.

Code: Complete code for this example can be seen in this gist.

Removing elements

This is an incredibly simple example, as the HTMLRewriter API does all the heavy lifting for us.

//...
if(acceptHeader && acceptHeader.indexOf('text/html') >= 0){
  // store this particular request for modification
  let oldResponse = await fetch(url.toString(), request)
  // create a new response
  let newResponse = new HTMLRewriter()
    // remove a specific script
    .on("script[src*='name-of-our-script.js']", new removeElement())
    // remove the third meta tag in the head
    .on("head > meta:nth-of-type(3)", new removeElement())
    // remove all div elements that start with 'prefix'
    .on("div[class^='prefix']", new removeElement())
    // remove all link elements that start with '/assets/' and end with '.css'
    .on("link[href^='/assets/'][href$='.css']", new removeElement())
    .transform(oldResponse)

    // return the modified page along with custom headers
    return newResponse
}
//...
class removeElement {
  element(element) {
    element.remove();
  }
}

You can be really creative with your CSS selectors if you need to be, after all the Worker is powered by Chrome’s v8 engine, so it understands all the modern CSS selectors.

Code: Complete code for this example can be seen in this gist.

Clearing and adding inline scripts

The problem with inline scripts is that most of the time they are just a simple <script> tag. There are no attributes to use to select them, and their position in the page may not always be consistent in the DOM. So a method I’ve found useful is to remove all inline scripts from a page then rebuild and add them back in where you need them. This gives you fine grain control over the HTML markup coming from the Worker:

//...
if(acceptHeader && acceptHeader.indexOf('text/html') >= 0){
  // store this particular request for modification
  let oldResponse = await fetch(url.toString(), request)
  // create a new response
  let newResponse = new HTMLRewriter()
    // remove inline scripts from the page
    .on("body > script:not([src])", new removeElement())
    // reinsert some inline JavaScript back into the page
    .on("body", new reinsertInlineScript())
    // transform the page
    .transform(oldResponse)

    // return the modified page along with custom headers
    return newResponse
}
//...
class removeElement {
  element(element) {
    element.remove();
  }
}

class reinsertInlineScript {
  element(element){
    let inlineScript = `document.body.className = ((document.body.className) ? document.body.className + ' js-enabled' : 'js-enabled');`;
    element.prepend(`<script>${inlineScript}</script>`, {html: true});
  }
}

Using this method you could completely rebuild where inline scripts sit in the page source, and experiment with what they contain (since they are blocking after all).

Code: Complete code for this example can be seen in this gist.

Adding inline CSS

Using pretty much the same method as above you can add inline CSS into the <head> of your page. In doing so we are adding bytes to the browsers critical path, but it can be useful for experimentation. One particular use case could be you are wanted to explore is what difference adding critical CSS to a page does for rendering performance:

//...
if(acceptHeader && acceptHeader.indexOf('text/html') >= 0){
  // store this particular request for modification
  let oldResponse = await fetch(url.toString(), request)
  // create a new response
  let newResponse = new HTMLRewriter()
    // add the inline CSS to the head
    .on("head", new addNewCSS())
    // transform the page
    .transform(oldResponse)

    // return the modified page along with custom headers
    return newResponse
}
//...
class addNewCSS {
  element(element) {
    let newInlineCSS = `
      body {
        border: 10px solid red;
      }
    `;
    element.append(`<style>${newInlineCSS}</style>`, {html: true});
  }
}

Code: Complete code for this example can be seen in this gist.

Adding attributes

There may be times where you want to add certain attributes to elements. A great use case for this in web performance is adding defer and async attributes to <script> tags, and seeing how this changes the loading performance of a page. In the example below we create classes to add these attributes to scripts. These can be adapted to add any attributes you might want to add to an element:

//...
if(acceptHeader && acceptHeader.indexOf('text/html') >= 0){
  // store this particular request for modification
  let oldResponse = await fetch(url.toString(), request)
  // create a new response
  let newResponse = new HTMLRewriter()
    // add defer to this script
    .on("script[src*='script-to-be-deferred.js']", new addDeferAttribute())
    // add async to this script
    .on("script[src*='script-to-be-asynced.js']", new addAsyncAttribute())
    // transform the page
    .transform(oldResponse)

    // return the modified page along with custom headers
    return newResponse
}
//...
class addDeferAttribute {
  element(element) {
    element.setAttribute('defer', 'defer');
  }
}

class addAsyncAttribute {
  element(element) {
    element.setAttribute('async', 'async');
  }
}

Code: Complete code for this example can be seen in this gist.

Quickly adding scripts to the <head> and closing <body> tags

The position of scripts in the page source has become less of an issue since modern browsers have started to support the async and defer attributes, but there still may be cases where you want to position them very specifically in the source for experimentation. This is the method I used when I was investigating reducing the Cumulative Layout Shift (CLS) on GOV.UK in this tweet. I cleared all scripts from the page source and rebuilt it with the scripts repositioned as required:

//...
if(acceptHeader && acceptHeader.indexOf('text/html') >= 0){
  // store this particular request for modification
  let oldResponse = await fetch(url.toString(), request)
  // create a new response
  let newResponse = new HTMLRewriter()
    // remove all external scripts from the page
    .on("body > script[src]", new removeElement())
    // insert scripts back before the closing body tag
    .on("body", new reinsertBodyScripts())
    // insert scripts back before the closing head tag
    .on("head", new reinsertHeadScripts())
    .transform(oldResponse)

    // return the modified page along with custom headers
    return newResponse
}
//...
class reinsertBodyScripts {
  element(element) {
    var srcArray = [
      '/assets/js/body-script-1.js',
      '/assets/js/body-script-2.js'
    ]

    srcArray.forEach(function(val){
      element.append(`<script src="${val}"></script>`, {html: true});
    })
  }
}

class reinsertHeadScripts {
  element(element) {
    var srcArray = [
      '/assets/js/deferred-head-script-1.js',
      '/assets/js/deferred-head-script-2.js'
    ]

    srcArray.forEach(function(val){
      element.append(`<script src="${val}" defer></script>`, {html: true});
    })
  }
}

There are many combinations you may want to experiment with related to async, defer and standard blocking scripts. I can’t cover them all here. But hopefully this example gives you an idea of how you’d do it.

Code: Complete code for this example can be seen in this gist.

Testing performance

So, once you’ve made changes to your page you’re going to want to check what difference they have made to the performance of the page. There are a couple of ways to do this:

In the browser

The browser is a great way to check to see if the page is being modified in the way you expect (e.g. view the page source), so I’d recommend setting up this method anyway. Using the ModHeader extension you can easily add request headers to your requests. In our examples above we’d set x-host to the site we want to change (e.g. x-host: www.gov.uk), and x-bypass-transform to either true or false depending on if we want to toggle the Worker changes on or off. You could then use Chrome Lighthouse and run a performance audit with the changes both on and off. Using a tool like Lighthouse CI Diff you could then compare the difference between the two sets of results.

Using WebPageTest

I use the browser method above for verification and debugging of the changes, but my preferred method of comparing results is using WebPageTest. It gives you incredibly detailed performance information that you can compare across multiple experiments. The test setup uses the overrideHost scripting functionality to reroute any requests to the original domain through the Worker for modification. An example script I use for testing with GOV.UK can be found below:

setCookie https://www.gov.uk/ cookies_policy={"essential":true,"settings":true,"usage":true,"campaigns":true}
setCookie https://www.gov.uk/ cookies_preferences_set=true
setCookie https://www.gov.uk/ global_bar_seen={"count":0,"version":8}
addHeader x-bypass-transform:false
overrideHost www.gov.uk govuk-worker.nooshu.workers.dev
navigate https://www.gov.uk/

Enter the above script into the ‘Script’ tab of WebPageTest, configure all your test settings then run a test. Once completed you can use the compare view functionality to examine the difference between the different sets of results.

Summary

In this post I’ve covered what Cloudflare Workers are, and how they can be used for various frontend performance tweaks and testing. Moving onto how you can easily check to see you are seeing the changes you expect, and then performance difference they make.

If you’ve watched Harry Roberts’ talk ‘From Milliseconds to Millions’ where he talks about the importance of a pages <head> tag for page performance, it’s now possible to easily try this. Using a Cloudflare Worker you have a way of testing changes quickly and easily against your production environment, without having to touch a single line of production code! Imagine being able to sell in performance work internally (or externally) with actual metrics you could see if production code is changed. Note: The results may not be 100% accurate compared to your production environment, but they are certainly a good way to give you an indication as to what is possible.

I personally think that’s a pretty great tool to have in your web performance toolkit!

I’d love to hear any feedback you have on the blog post above, or if there are any other recipes you use, please do let me know!

Further Reading

There’s been more written about using Cloudflare Workers for web performance optimisation here:


Post changelog:

Loading

Webmentions