SilentChat

A/B tests — compare widget variants

Last updated: November 18, 2026

Available from the Professional plan. On smaller plans the page shows an upgrade notice. Experiments you already created stay saved but are not served until your plan includes them again.

The experiment only counts visitors who have given consent. A variant is only assigned when "Require Consent (GDPR)" is switched on for the widget and the visitor has consented. Every other visitor always sees variant A, nothing is stored in the browser for the experiment, and their conversations are not included in the statistics. Without "Require consent" an experiment therefore runs empty: everyone sees A and no one is counted.

How an experiment works

  1. Create two (or more) widgets, e.g. a copy of your standard widget with a different greeting.
  2. Under AI & Automation → A/B tests, create an experiment: a name, the widgets to compare and the split (e.g. 50/50).
  3. Click Start. From then on every visitor with consent is assigned a variant by a deterministic hash. The same visitor lands in the same variant on the next visit.
  4. For each conversation of a consenting visitor SilentChat stores the variant_assignment ({group_id, label}), so the statistics stay attributable even after the experiment ends.

Who lands in which variant?

Without consent: always variant A. Variant A is the variant whose name comes first alphabetically. This also applies to the visit in which the visitor consents: when the widget loaded there was no consent yet, so the visitor saw A. Conversations from that visit are not counted; from the next visit on, the visitor is assigned and counted.

Why are visitors without consent not counted as A? Otherwise every visitor without consent would sit in A, and the comparison with B would be skewed.

With consent the assignment is deterministic: sha256(visitor_fingerprint + group_id) mod 100 yields a bucket from 0 to 99. With 50/50, buckets 0-49 get variant A and 50-99 variant B. As a backup, the widget stores the result in localStorage (sc_variant_<public_key>) only with consent, and deletes it when consent is revoked.

Statistics and winner

Per variant the dashboard shows conversations, messages, conversions, the rate and the lower bound of the Wilson 95% confidence interval. A winner badge appears only with at least 200 visitors per variant and non-overlapping confidence intervals.

Data protection

Storing an entry on the visitor's device requires consent under § 25 TDDDG unless it is strictly necessary — and an A/B test is not. Therefore:

  • Without consent the widget reads and writes nothing in the browser for the experiment.
  • With consent it writes sc_variant_<public_key> with the value { "group_id": "<uuid>", "label": "A" } until consent is revoked, the browser storage is cleared or the experiment ends.

For your privacy policy:

If you have consented to processing in the chat, we store a local entry (sc_variant_<id>) to keep you in the same test variant. It contains only a test ID and a variant name (e.g. "A"). Purpose: comparing two versions of our chat window; legal basis: your consent (§ 25(1) TDDDG, Art. 6(1)(a) GDPR), which you can revoke in the chat window.

Frequently asked questions

Do I need consent for the localStorage entry? Yes, and SilentChat enforces it: without consent the widget does not write the entry and the visitor sees variant A. Switch on "Require Consent (GDPR)" for the widget, otherwise the experiment counts no one.

Why does my experiment show fewer visitors than the visitor statistics? Only conversations of visitors who had consented before the visit are counted. Visitors without consent and conversations from the visit in which they consented are left out on purpose.