What was wrong with v1?

years ago, back in 2009, when the first version of the DNS Benchmark was designed and released, websites relied upon many fewer 3rd-party domains. Most of a webpage's content was delivered by its own web server. This meant that caching that domain's IP would allow most of a webpage to be delivered.

That is no longer true for today's web.

Today's webpages are loaded with crap. They're sourced from a ridiculous number of different servers which provide code libraries, scripting and feedback mechanisms, unsolicited “AI” chatbox pop-ups, cookie permission banners, visitor comment systems, “likes” tracking and audience profiling and tracking, not to mention more advertisements than ever.

That content is sourced from 3rd-party services under contract with the website. Each one of them is identified by a domain name whose IP needs to be looked up by the user's operating system or web browser before that content can be loaded.

The two DNS resolver rankings shown above were taken immediately one after the other to capture each Benchmark's view of the same resolvers and the same network. The length of the bars signifies the time required to receive a DNS reply from a DNS query, so shorter bars are faster and better.

Because the v1 Benchmark prioritizes resolver cached performance (the red bar) over uncached lookup performance (the green and blue bars), the two Cox Communications resolvers shown at the top of the left chart receive top ranking because they have the shortest red bars. Those two resolvers are circled by a black outline because they're the two DNS resolvers the system is currently using.

The problem with v1 was that it only used a resolver's other critical performance measurements – its uncached lookup performance (the green and blue bars) – as a tie breaker when cached performances were identical. In other words, at the top of the left chart, v1 completely ignored the lengths of the green and blue bars merely because the red bar was the smallest.

But consider the 8th item on the left chart showing the performance of the resolver at IP 129.250.35.251. That resolver's cached lookup delay (red) bar is only a tiny bit longer, but its green and blue (uncached) delays are much shorter (faster) than those two top-ranked resolvers. That resolver would be a better choice than the two that it put in first and second place because many more domain names are being looked up today than were being looked up years ago in 2009.

Now look at the ranking created by the new v2 Benchmark

In the v2 chart on the right, the vertical black lines show the average of all three performance measures. Using v2, we see that the Cox Communications resolver at IP 68.4.16.25, which v1 ranked in first place, now ranks well down the list below many faster resolvers even though its performance did not change. The v2 DNS Benchmark considers the overall performance of every resolver. Even though that Cox Communications resolver has the fastest cached performance, its uncached lookups slow down its overall performance so that in it would not be right choice.

The Internet has grown and changed dramatically in the
years since the v1 benchmark was designed and published

By spending a year to significantly update our DNS Benchmark, converting it into an inexpensive user-supported commercial product, the DNS Benchmark will be able to support itself and we will be able to continue investing in its evolution as the Internet continues to evolve.

For $9.95 one time, you can purchase version 2 of GRC's famous DNS Benchmark, receive all future improvements for free, use it for the rest of your life, and support this and all of GRC's services and other freeware: