DocumentRoot and Google Webmaster Tools
Google Webmaster Tools is a nifty service from Google that allows you to keep tabs on how Google interacts with the sites you own, giving reports on search rankings, missing files, and incoming links. I use it on this domain and also for Alacatia Labs.
Verification
When you initially add a domain to your account, Google provides very limited information about it — namely, the last time Googlebot crawled the site. To gain access to the wealth of information Google has collected, you must first verify your account in one of two ways:
- Add a meta tag to the HTML header for the homepage
- Create a file named googleXXXXXXXXXXXXXXXX.html in DocumentRoot (where the Xs represent a unique 16-digit hexadecimal string)
Interestingly, the meta tag is unique for every domain; the googleXX.html file name is the same across all domains on your account.
Cutting to the Chase
Several sites allow you to register an account and get a URL at the top level. Some examples are 6URL, Wishlistr, and del.icio.us. While these all filter their usernames, some sites don’t think to parse them, and thus are vulnerable to a user crafting an account whose name matches the verification filename for their Google account.
Canonical Ltd.’s Launchpad was one such site. As this screenshot shows, I was able to register a project with my Google verification filename and convince Google that I was a Launchpad webmaster.
It took less than 12 hours for Canonical to patch it based on the bug report I filed. Thanks to James Henstridge and Tom Haddon for getting things fixed up.
Under the Microscope
What about countless Wiki sites that use the URL format http://example.com/ArticleName? Google has a clever way of protecting against that. Here is a portion of an Apache access log of the Google’s verification process.
As you can see, Google not only requested my verification file, but also a secondary one starting with “noexist”. Because Apache returned a 200 OK status code for this (nonexistent) file, Google deduces that my site is not properly reporting 404 errors, and won’t let me verify through the file method.
December 7th, 2007 at 6:57 pm
Very interesting post, Ryan. And, kudos to you on finding such a potential security exploit and reporting it to Canonical.
I wasn’t able to view your bug report though when logging in with my Launchpad account. Have you set some sort of permissions that don’t me allow to view the contents of the bug report?
I do think Google Webmaster Tools is a terrific tool and a vast improvement from its previous incarnation that required you to create an account that was totally separate from your Google account and login on an obscure five-digit port. All that then allowed you to do was remove content from Google’s index. This allows much more, in particular, customizing the quick links that appear below your search result and allow people to directly navigate to sections of your website.
Cheers, Doug
P.S. Ryan, could you contact privately, on AIM or by e-mail? I have something unrelated I wanted to discuss.