Finding “SlurpConfirm404″ in Your Logs? Here’s Why.   - 5,422 Views, 1 Comment

First, Yahoo Slurp is what Yahoo calls their Yahoo Web Crawler - their website indexing engine that crawls around the world wide web, indexing (cataloguing) all of the websites, and all of the web pages on those websites.

Now, many search engines and other web indexers, including Google and Yahoo, are interested in knowing what happens when someone comes to your website and tries to find a page on your site which doesn’t actually exist. What sort of error or message does your website return? Typically this error is error number 404 - page not found, and these search engines want to know how your site handles these. They want to see a proper “404 - not found” response for this, and want to make sure that your site is not returning such other response.

When the Yahoo Slurp web crawler wants to test your site to see what your site does with a query for a non-existant page (which should return some sort of “404 - page not found” error), it asks for a page on your site called “SlurpConfirm404″, on the assumption that you won’t have such a page, and so it will be a good test of what your site returns for such a request. In other words, it’s Yahoo’s Slurp’s way of confirming a 404 response - hence “SlurpConfirm404″.