Crawler – Recent errors

Administration → Web Crawler → Recent Errors

Like the queue, but with status=failed pre-selected and without status filter. Custom filters: Tenant, URL substring, date range.

Workflow

1. Sorted by completed_at DESC — most recent errors first
2. Click on a row opens the job detail page with the full error message + recommendation
3. Recurring errors for a URL: in the detail page disable auto-recrawl directly (or via bulk action on the queue page)

Common Errors

• HTTP 4xx/5xx — URL not (no longer) accessible
• robots.txt blocked — ends up as status=skipped with reason
• Content unchanged — not an error, but optimization (same hash as last crawl)
• Timeout / DNS — provider issues or URL entered incorrectly