By Howard Osborne
The HTTPArchive provide an excellent resource for analysing trends. As well as being able to analyse individual sites, aggregate stats are gathered for the top 100, top 1000 and all sites as identified by Alexa.
Here are the percentage of pages with errors for the top 100, top 1000 and all sites:
Figures for July 2015 show that 35% of all pages requested from all sites returned an error code. This would be either a 4xx client error, such as 404 page not found, or a 5xx server error, such as 500 internal server error. The top 1000 sites fared better with 18% of pages returning such an error and the top 100 sites returned the lowest score at just 10%.
But before the top 100 feel too smug, this is how they compare to four years ago:
In July 2011 only 3% of pages returned an error for top 100 sites, with the top 1000 returning 12% and 26% of all sites pages returning an error.
What would happen if this trend were to continue?
By January 2043, virtually all pages for all sites would return an error except for the top 100 and top 1000 sites, where 60% of pages would fail. If this happened, the internet would be as good as dead.
But before we all start relearning the art of letter writing, perhaps we should take a look at the potential causes.
As well as technology trends, are there any non-technical causes, such as in the way we work?
To put it in a word: Agile.
The growth of Agile practices across software development has been little short of a revolution and it is hard to find a place that doesn’t profess to be Agile. Delivering working code quicker can put pressure on quality. However let’s not forget, the pressure was always there in overdue waterfall projects and testing was the area most squeezed. But making sure testing is at the heart of an Agile process is not just about finding defects, it’s about re-focussing development effort on quality.
So will errors end up killing the internet?
Of course not, but it might kill your part of it…