Server-Timing Compression

Introduction

For domains that use our RUM product, through the power of the Edge, we are now injecting up to 3 Server-Timing entries per resource (including basepage). Because we don’t sample and because we beacon back every timer of every resource requested from every non cross-origin <IFRAME>, we’ve already been applying custom compression based on tries to save data going out of the browser. For Server-Timing, we have a lot of work to do, because the data can get pretty verbose and redundant.

Example

Let’s imagine a scenario where we have five resources each with 2-3 Server-Timing entries:

All five of those resources emit cdn-cache and edge entries. Three emit an origin entry. Built on the assumptions that developers are emitting mostly the same Server-Timing header name values, that desc is essentially an enum, and that dur values will be mostly unique, We augmented the resource timing compression library we use to compress all collected Server-Timing data.

There are two parts to the compression:

• a lookup data structure shared by all resources
• a compressed string (for each Server-Timing entry) that “keys into” the shared lookup

Here’s how it works. First, we iterate over the Server-Timing entries for all resources counting up the unique name / description tuples like so:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

{
  "cdn-cache": {
    "count": 5,
    "counts": {"HIT": 2, "MISS": 3}
  },
  "edge": {
    "count": 5,
    "counts": {"": 5}
  },
  "origin": {
    "count": 3,
    "counts": {"": 3}
  }
}

Then, we take that data structure and convert it into an array of arrays, ordering by most popular (cdn-cache and edge before origin, "MISS" before "HIT"):

1
2
3
4
5

[
  ["cdn-cache", "MISS", "HIT"],
  ["edge", ""],
  ["origin", ""],
];

Remember, the goal here is to save as many bytes on the wire as possible. Let’s take advantage of the fact that there are no descriptions specified for any of the edge or origin entries. This leaves us with our final lookup:

1

[["cdn-cache", "MISS", "HIT"], "edge", "origin"];

This lookup tells us that for all of the Server-Timing entries for all of the resources we have collected:

• there are exactly three unique name values: cdn-cache, edge, and origin
• for cdn-cache, there are exactly two non-empty descriptions ("HIT" and "MISS") and "MISS" appears as many or more times that "HIT"
• there are no non-empty descriptions for either edge or origin

Now we can figure out how our individual resources will “key into” our lookup. Here’s a view of the Server-Timing entries we’ve collected:

For each row, let’s apply two changes:

• replace the name with the index into the name value of the lookup, as “name index”
• replace the description with the index into the description value for that name of the lookup, as “desc index”

Next, let’s apply the template ${duration}:${name index}.${desc index}, as “Compressed #1”:

Now, let’s strip out those inferrable zeroes, as “Compressed #2”:

Finally, let’s concatenate multiple entries for each resource with a comma, as “Compressed #3”:

It is at this point that we can stick both the lookup and the compressed per-resource Server-Timing entries into the trie on our beacon.

(A clever reader might notice a failure to lop off durations of value zero, which might be quite common. 😦 The writer intends to personally apologize to each byte needlessly sent over the wire and promises to rectify this egregious and regrettable omission as soon as possible.)