[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[public-dns-discuss] Popular websites are often not present in cache?

Popular websites such as google.comwikipedia.com, and youtube.com do not appear to be present in the resolver's cache for large periods of time (at least a day). I noticed this by running the command "dig @ domain.com +norecurse" on multiple websites, for a few months, once a second. As far as I can tell, Google DNS returns a SERVFAIL response if a DNS request is made with the Recursion Desired bit unset. Google.com, wikipedia.com, and youtube.com consistently return SERVFAIL. However, many websites of similar popularity according to the Alexa top websites ARE cached. I am hoping someone can tell me why these websites are not present in cache when their popularity would intuitively imply that they should be. Does Google DNS deliberately not cache certain websites? Is my measurement technique flawed? Any assistance you can give me would be very welcome.

For context, I am a PhD student at University of California San Diego studying Internet measurement, and a three-time Google intern. One of my current projects involves measuring the behavior of different DNS resolvers, which is why I'm interested in this. This is a cross-post from the Google DNS Issue Tracker, because I posted there two weeks ago but haven't gotten a response yet.

Has anyone else noticed this behavior?

Thank you.

You received this message because you are subscribed to the Google Groups "public-dns-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to public-dns-discuss+unsubscribe AT googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/public-dns-discuss/dac62961-7a37-4d70-9deb-e46cd540c935%40googlegroups.com.