Finding and Fixing Website Link Rot with Python, BeautifulSoup and Requests
When hyperlinks go dead by returning 404 or 500 HTTP status codes or redirect to spam websites, that is the awful phenomenon know as “link rot”. Link rot is a widespread problem; in fact, research shows that an average link lasts four years.
In this blog post, we will look at how link rot affects user experience, using Full Stack Python as our example. We’ll build a Python script that detects link rot in Markdown and HTML files so we can quickly find and fix our issues.
A Link Rot Example
The site now has over 145,000 words and 150 pages, including:
- 2400+ links in the repository
- 300+ HTML files
- 150+ Markdown files
And there ...