[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[public-dns-discuss] Re: Popular websites are often not present in cache?



aurandal wrote:
Popular websites such as google.comwikipedia.com, and youtube.com do not appear to be present in the resolver's cache for large periods of time (at least a day). I noticed this by running the command "dig @8.8.8.8 domain.com +norecurse" on multiple websites, for a few months, once a second. As far as I can tell, Google DNS returns a SERVFAIL response if a DNS request is made with the Recursion Desired bit unset. Google.com, wikipedia.com, and youtube.com consistently return SERVFAIL. However, many websites of similar popularity according to the Alexa top websites ARE cached. I am hoping someone can tell me why these websites are not present in cache when their popularity would intuitively imply that they should be. Does Google DNS deliberately not cache certain websites? Is my measurement technique flawed? Any assistance you can give me would be very welcome.

Your analysis of RD flag is correct.

You are missing the fact that google.com, wikipedia.org, and youtube.com all use DNS name servers that support EDNS Client Subnet (ECS), which fragments the cached based on the /24 or /56 subnet of the client address. While these are popular domains, unless someone from a network in the same range queries Google Public DNS without RD=0, our resolvers will not have any cached data. The other "similarly popular" domains probably do not support ECS, so the cache is not fragmented and you will see returned cache records.

--
You received this message because you are subscribed to the Google Groups "public-dns-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to public-dns-discuss+unsubscribe AT googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/public-dns-discuss/3091353f-fb5d-4590-b65c-8f3a7e4dbd83%40googlegroups.com.