Today, Microsoft announced changes to their Live Search crawler intended to reduce bandwidth resources during the crawl of a site. MSNbot (upgraded to version 1.1) now supports both HTTP compression ...

Via Search Engine Roundtable comes news of refreshing candor from a Bing employee about the performance of Bing’s crawler, which is still referred to as MSNbot. The admission happened yesterday in the ...

Here is one more oddity to add to Microsoft Bing's web crawler, MSNBot. Why on earth are people reporting that MSNBot is crawling the same page twice, once for the compressed version and then once ...

Microsoft has a post today on its Bing Community blog alerting website owners that they may start seeing a lot of pings from the user agent “msnbot/2.0b (+http ...

Microsoft has been using MSNBot to crawl websites for its search engine. Despite changing their search engine to Live Search and then to Bing, Microsoft did not rename the bot that crawls the web to ...

Last December, we reported that MSNBot was failing a reverse DNS lookup. Well, guess what folks - MSNBot is failing again on some IP addresses. The thread has other IP addresses with MSNBot acting ...

Months ago MicroSoft announced that it would get serious about MSN Search, perhaps breaking its dependency on LookSmart, and it seems that they have jumped one of the first hurdles by developing a new ...

While MSNBot crawls billions of web pages, not every page that we crawl is indexed. For a site to be indexed, it must meet specific standards for content, design, and technical implementation. For ...