Crawler – Queue

Administration → Web Crawler → Queue

Detailed view of all crawl jobs (queued / processing / completed / failed / skipped / cancelled).

Filter

Status, Source (manual / sitemap / recrawl), Tenant (Picker with search), URL substring, Date range (queued_at).

Bulk Actions

Checkboxes per row + Select-all. Toolbar appears as soon as at least one row is selected:

• Cancel — queued/processing → cancelled
• Requeue — failed/skipped/cancelled → queued (deletes error message, started_at, duration)
• Move to front / back — Priority ±10 (only queued)
• Disable Auto-Recrawl — kb_crawl_sources.is_active = false for (tenant_id, url) tuple

Each action pops a confirm dialog ("X of Y processed"). Selection is reset on page change.

Detail View

Click on a row → /admin/crawler/<id> with all job fields, the recurring crawl status (Auto-Recrawl flag), content hash and skip reason when status=skipped (robots.txt, content-type, unchanged).