Monday, August 08, 2005

Google search results that require subscriber-only login? Disable JavaScript!

Today I experienced something I've seen many times before, but it pissed me off enough to investigate this time. Many times, when I search for some info with google, some of the results that I click do not include the matched content, but instead take me to a login page, saying the content is restricted to subscribers only.

Today, the site was techtarget.com. I've found useful information on that site before, and it was the #3 result in my search (I searched for "tracking down wireless network interference"). Sometimes, it's possible to view these pages by viewing google's cached copy. This time, that didn't work either. Even the cached copy redirected me to a login page.

So, i tried looking at the http headers with LiveHttpHeaders extension in FireFox, and that's where I saw these URLs as the first two requests:

First, google's cache:
http://64.233.187.104/search?q=cache:JGSD_2iKWo0J:
searchmobilecomputing.techtarget.com/tip/1,289483,
sid40_gci1109438,00.html+tracking+down+wireless+
network+interference&hl=en&client=firefox-a


Second, a javascript page from techtarget:
http://searchmobilecomputing.techtarget.com/
pageScripts/0,294327,sid40,00.js


Everything after that was related to the techtarget login page.

So, the redirect is in the javascript, eh? So I used 'wget' to download the javascript file ("0,294327,sid40,00.js"), then opened it in my text editor ( TextPad), and searched for 'redirect'. I think the following is causing the redirect, although I haven't looked thoroughly through the whole script, because I didn't need to:


/*
Force login if content is members only.
Redirect to top in case document is loaded
in a frame or iframe.
*/
var forMembersOnly =
(typeof JSmembersOnly != "undefined" &&
JSmembersOnly == "on") ? true : false;
if (forMembersOnly && loggedIn != 1) {
var NextURL = window.top.location.href;
if (NextURL.indexOf(
"/ttCMAv2/Production_Center/" +
"Preview_Form_v2/1,2563,,00.html") == -1) {
var redirectURL;
if (typeof JSmembersLoginURL == "undefined" &&
JSmembersLoginURL == "") {
redirectURL =
(typeof JSlinkLogin != "undefined") ?
JSlinkLogin :
"http://searchMobileComputing.techtarget.com/" +
"loginMembersOnly/1,289498,sid40,00.html";
} else {
redirectURL =
((JSmembersLoginURL.indexOf(
"http://searchMobileComputing.techtarget.com") ==
0) ? "" :
"http://searchMobileComputing.techtarget.com") +
JSmembersLoginURL;
}
window.top.location.replace(redirectURL +
"?NextURL=" + escape(NextURL));
}
}


So, I just disabled javascript in my browser before clicking the link!
This is VERY EASY to do with FireFox and the
Web Developer Toolbar extension.

In my opinion, there's nothing wrong with doing this. They made the content available for google to index, but then deliberately prevented me from viewing it (bait and switch?). I just viewed it without javascript. It probably would have also worked to just use a text-only browser like links or lynx. This will probably not get around ALL cases of this annoying search result behavior, but it did help in this case.