Crawler
SilentChatBot
SilentChatBot is the SilentChat crawler. On behalf of our customers it indexes their own verified website content, so the AI assistant in the chat widget can answer accurately from the knowledge base.
SilentChatBot/1.0 (+https://silentchat.de/bot)Every request carries exactly this identifier. We do not disguise ourselves and do not rotate user agents.
Verified customer domains only
The bot fetches only domains whose ownership the customer has proven in the dashboard. It does not crawl third-party or public websites.
robots.txt is respected
Allow and Disallow rules (including * and $) for SilentChatBot or * are honoured; disallowed paths are never fetched.
Polite & rate-limited
A strict per-domain rate limit applies. On 429/503 we back off (honouring Retry-After); repeated blocks or a 403 stop the crawl instead of hammering the server.
Honest identity
A fixed, contactable user agent — no browser spoofing, no shifting identifiers.
Gentle & deduplicated
URLs are canonicalised (tracking and session parameters removed) and unchanged pages are skipped, so your server isn't queried needlessly.
Contact & opt-out
Since we only fetch our customers' own websites, a note from the site operator is usually enough to disable it. For questions about the bot's behaviour or opt-out requests, contact support@silentchat.de. Alternatively, you can restrict SilentChatBot via robots.txt — User-agent: SilentChatBot.