SEO Crawler
A free, fast & flexible SEO website crawler to help identify technical SEO issues. Crawl up to 350 URLs for free!
Crawler features:
- It's fast*. View real-time feedback of your crawl.
- Crawl on the move - runs on mobile phones & tablets including iPad, iPhone and more.
- Crawl up to a maximum of 350 URLs for free
- Use advanced filters to customize your crawl to sub-sections of your website
- Web reports show:
- URL
- Page title
- HTTP status code
- Details of redirect chains
- Meta robots details
- Followed external links & total links on-page
- rel=canonical URLs
- Any automatically detected SEO issues
- An Excel-compatible TSV report can be exported after the crawl showing:
- All of the above, plus:
- Meta description
- Total external links
- Meta keywords
- Meta refresh
- rel=author details
- rel=alternate (hreflang & media) details
- rel=next/prev details
- Use regular expressions to customize URL patterns to include, ignore, or strip parameters/folders out of the crawl.
- Search for content within a site's pages using regular expressions - for example, use
UA-123456-1
to look for presence of a Google Analytics Profile ID - Respects robots.txt rules & rel=nofollow
- Test how your site behaves with different user-agents such as Desktop, Mobile, Tablet and search bots
- Identify duplicate page titles, duplicated meta tags, server errors, redirect chains and more.
* Speed is limited to avoid impact on server load