Reply to comment

Microsoft Live Bogus Search Referrals

Fri, 12/26/2008 - 07:57 - peter | |

For some time, I was using Google Analytics for most of my sites. It's got some good technology for drilling-down, and it's handy because they store all your data and take the CPU hits. But I switched over to Piwik which is an open-source solution where you keep your own data locally. The main reason I did this initially was because I didn't want to have google using my data to penalize sites I administer. While it's hearsay, I've read that google's algorithms penalize sites if they appear to the algorithms to be linked deceiviously (whether or not there is any malicious action); and certainly with a new website, there is a high bar to getting included in the SERPs.

Enter piwik. At first I was angered that piwik wasn't properly tracking the substantial search traffic I had been receiving from live.com (according to google analytics). Then when I investigated more closely, I realized that piwik is tracking properly. Microsoft's search quality bots send bogus search referrals! Barry Schwartz wrote about thgese Microsoft Search Quality Tests over a year ago, and it's clearly still going on, perhaps in somewhat modified form (most of the pseudo-searches I've found are for words like "central" that are on my pages but for which my site doesn't rank at all in live.com).

Here is an example of a bogus search referral from live.com on one of my sites (Central America Forum):

65.55.110.172 - - [11/Dec/2008:17:05:57 -0500] "GET /forums/gardening-and-agriculture-in-central-america?sort=asc&order=Topic HTTP/1.0" 200 22328 "http://search.live.com/results.aspx?q=central" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322)

It looks like someone with a real computer running windows NT w5.2 searched for "central" at live.com and was referred to our site. But actually what happened is that this was a microsoft bot, probably checking for cloaking - to see whether the site returned the same web page when the requesting agent is a disclosed bot, as it does when the requesting agent is a search referral. I guarantee, nobody searched for "central" on live.com and then clicked through to this one of my sites.

You can also investigate the IP addresses of these bogus referrals and this also verifies - they are all from 65.55.x.x.

Reply

The content of this field is kept private and will not be shown publicly.
CAPTCHA
This question is for clevery testing whether you are a human visitor and to prevent automated spam submissions.
Image CAPTCHA
Enter the characters shown in the image.