Our SEO team recently asked us to fix an issue with a client website that we built, whereby a 404 error was being returned as a 200 (ok) response.

As you may know, any decent website has custom error pages to give the user more information, support and to avoid a browser displaying the default ugly page for the error. In .NET, we define custom errors in web.config or at the IIS level. a custom page is a redirect and would return a 200 response since the page was loaded fine although the original response to the invalid URL was a 404.

We needed to fix this because a search engine would keep the missing URLs in its index and these invalid URLs would end up in search results which is a bad user experience from a search perspective (you might say who cares about the search engine) but also from a website perspective (I guess a user may tend to click other domain links if they get 404s on your domain).

Also, Google for example states: “Don’t create multiple pages, subdomains, or domains with substantially duplicate content.” If it finds many of these, it will remove them from the index (which is fine, they were error pages anyway) but could lower the site ranking (which is real bad) because crawl times increase.

So, how can this issue be fixed?
In the case of .NET, just add: <% Response.Status = “404 Not Found” %> in the page and you’re good to go.
Of course, this could be done in the code-behind too.

You could apply this to all your custom-error pages if you want to be diligent.

Advertisements