We give you an overview of compression in the modern web stack. We also detail a recent scenario where we sped up queries on our codebase significantly by enabling brotli compression, an algorithm popularized by Google.

We've been experimenting with quite a few methods to speed up larger request payloads on our GraphQL endpoints, and we found that compression has done wonders.

Jay Khatri

At Highlight, we're super focused on keeping our app snappy and fast. Given our product, we take responsibility for making sure that our users are able to quickly find the session/error/trace that identifies an issue as fast as possible.

Recently, we've been experimenting with quite a few methods to speed up larger request payloads on our 

 endpoints, and we found that compression has done wonders. Below, I'll share an overview of compression in the modern web stack as well as some code that can get you started!

What is compression and how can it help me?

 recognize patterns in data and use these patterns to reduce the payload size sent across the wire. For the purpose of this discussion, we'll focus on text compression, but for the most part, these concepts roll over to other types as well (sound, video, etc..).

In a modern web stack, because text is being sent over HTTP quite frequently (between your frontend and backend servers, for example), we can make use of patterns in this text to dramatically decrease the size of data. If this can be successfully done, your web server won't have to send as much data which saves bandwidth. Beyond that, if the amount of time taken to compress the data is minimal, this can also give your app a performance/speed boost. Overall, because there are loads of libraries and frameworks out there that will do this for us, experimenting with compression in your web app is well worth the effort!

What are common compression algorithms out there?

From 1992 to 2013 or so, the most common compression algorithm used on the web was 

. Because it was quite fast and decreased size by a lot, it quickly became very ubiquitous, gaining support in all major web browsers and 

. Around 2013,  Google published a new algorithm called 

, which performed even better than Gzip, and has been gaining popularity since. At this point, Brotli is supported by all major browsers and has a clear benefit over Gzip both from a speed and efficiency perspective, and this is what we use at Highlight!

. For browsers that don't support Brotli, however, the compression method will fall back to gzip based on the request's `Accept-Encoding` header. This means that there's little to no downside to enabling Brotli; it's an easy performance win.

, we saw response sizes decrease around 40% with no latency increases (requests were faster!). We're now using this on all of our API endpoints and there's a clear difference in load times and bandwidth usage.

Because of its ubiquity, most languages/frameworks will likely have a Brotli library that can be wrapped around the web server. At Highlight, we write a lot of Go, so below is an example for implementing Brotli compression in a Chi router.

# Install the Chi, older versions of Chi don't support Brotli
go get -u github.com/go-chi/chi
# Install the package that will do the Brotli compression
go get -u gopkg.in/kothar/brotli-go.v0

r := chi.NewMux()
compressor := middleware.NewCompressor(5, "/*")
compressor.SetEncoder("br",func(w io.Writer, level int) io.Writer {	
    params := brotli_enc.NewBrotliParams()	
    params.SetQuality(level)return brotli_enc.NewBrotliWriter(params, w)})
r.Use(compressor.Handler)

The `"/*"` means to compress all content types that can be compressed. These are the supported types:

var defaultCompressibleContentTypes = {
        "text/html",
        "text/css",
        "text/plain",
        "text/javascript",
        "application/javascript",
        "application/x-javascript",
        "application/json",
        "application/atom+xml",
        "application/rss+xml",
        "image/svg+xml",
}

Compression: The simple, powerful upgrade for your web stack

Other articles you may like

Get the visibility you need