How did this answer come about? Reading the trace of a bot reply
For every reply from your AI bot and every handover to your team we store a trace: what the bot searched for, which articles in your knowledge base matched, what of that went into the instruction to the AI, and whether a safeguard stepped in. That lets you see why the bot said something and improve your knowledge base where it matters.
Where to find the trace
- In the inbox: Under every bot message you will find How did this answer come about?. One click opens the trace.
- In a ticket: If a ticket belongs to a conversation with the bot, the block Bot answers in the conversation lists every answer with its trace.
- As an overview: Under Analytics → AI quality you can see across all conversations how often the bot answered without evidence.
Everyone in your workspace who may read conversations can see the trace. Your visitor never does.
The chain: nine steps
On the left each step appears with a short result, on the right the details of the step you click.
| Step | What it shows |
|---|---|
| Input | The visitor’s message, the channel and the detected language. |
| Gates | Whether something held the bot back: budget, platform limits, muting after a human replied, maximum bot replies and handoff keywords. |
| Search query | The text the knowledge base was searched with, and the language searched in. |
| Retrieval | The articles found, by rank, with their distance to the question. What went into the instruction to the AI is marked. |
| Confidence | How well the best match fits, and whether it is above your threshold. |
| Prompt assembly | Which sections the instruction to the AI was made of (for example safety rules, guardrails, knowledge base). |
| Model | Which model answered and how long it took. |
| Post-processing | Whether a safeguard changed the answer. If it says “no change”, the text was delivered as the model wrote it. |
| Result | Answer sent or handed over to your team, with the reason for a handover. |
Reading the retrieval
In the Retrieval step the articles are listed by rank. Several sections of the same article appear as one row; the other articles move up. A row shows rank and distance of its best section. The smaller the distance, the better the article fits the question.
- Green is what went into the instruction to the AI. “in prompt · 2 sections” means two sections of the same article were used.
- The yellow line separates what was in the prompt from what was a narrow miss. An article just behind the line is a good candidate if the answer was off: check whether it should answer the question more clearly.
- “+3 more sections” on a row means this article has further matches that are hidden. Hover over the row or select it with the Tab key and the sections appear on the right, joined by lines, with rank and distance. A click keeps the view open until you click elsewhere, click the row again or press Escape. On a phone the sections open below the row.
- Below the list you see which ranks are hidden. The table with all matches is collapsed underneath.
Confidence and “No evidence”
Confidence is the distance of the best match, converted: the higher, the better the article fits. The levels are high from 0.75, medium from 0.50, below that low.
If no match is above your threshold the bot does not always answer: whether it hands over to a human or still answers from the weak matches depends on “Hand over on off-topic questions too”, see Handoff rules. If it answers anyway, the message carries No evidence and the conversation shows as Bot unsure in the list. The threshold is called Match Threshold in your AI settings; how to set it is described (in German) in KB-Trefferqualität tunen.
On a message in the inbox you can tell at a glance:
| Label | Meaning |
|---|---|
| Sourced | A match was above the threshold. |
| No evidence | No match was above the threshold; the model answered anyway. |
| Confidence high · 0.81 | Level and value of the best match. |
| Link removed | A safeguard removed an address (see below). |
| Competitor replaced | A safeguard took a competitor out of the answer. |
| Regenerated | The bot wrote the answer a second time, for example because of a repetition. |
Safeguards: always on
Two rules run on every answer and cannot be switched off:
- Links: An address stays only if it comes from your knowledge base or your settings, or was mentioned by your visitor or your team in this conversation. An address the model made up is removed; the sentence stays. Which address was removed is shown in the Post-processing step.
- Competitors: The bot does not recommend competing products. If an answer names a competitor, the bot words it once more; if that fails, the affected sentences are removed or replaced by a neutral answer. The bot also does not search the internet freely.
What is stored and for how long
- Stored: the search query, the titles of the articles found with their match quality, the sections of the instruction to the AI, the model and the duration, and the interventions of the safeguards. The content of the articles is not stored.
- The search query contains parts of your visitor’s message. It is shortened to 400 characters. If “Mask contact details before the AI” is on, email addresses, phone numbers, IBANs and card numbers are replaced in the trace too.
- The trace is deleted after 180 days, at the latest together with the conversation. If you delete a visitor, the trace is gone as well; it is part of the access given to the visitor.
- It is visible to your workspace and to our support, who can open it to clarify enquiries and for quality assurance. Every access by support is recorded in your audit log.
Common questions
The bot answered nonsense. Where do I start? Open the trace and look at the Retrieval. Were the right articles there? If not, knowledge is missing or the question is worded differently from your article. Add the words your visitors use. If the right articles were there and the answer was still wrong, check the text of the article itself.
Why is the same article in the prompt twice? Because two sections of this article fitted well. That is intended; the row shows “2 sections”.
Why is an article in another language in the retrieval? The bot searches in the language of the question first. If your knowledge base has too few matches in that language, it fills up with other languages. Which language was searched is shown in the Search query step.
I see no link under a bot message. Then you lack the right to read conversations, the answer is from before October 8, 2026 (no traces were stored before that), or it is older than 180 days.