Written by: Kacper Osiewalski, Lead Backend Engineer, Digital Colliers

The TikTok internal doc that surfaced this month is going to become a reference point for regulators in every consumer-facing vertical. The claim, as reported, is that a safeguarded algorithm was withheld from around 10% of US users as part of an experiment. Whether that framing survives scrutiny or not, the shape of the problem is now on the table: if you A/B test a safety feature, you are, by definition, denying that feature to a control group. And you have to be able to explain why.

For iGaming operators running experiments on responsible-gambling nudges, deposit-limit prompts, session reminders, cool-off flows, the question stops being academic. Your A/B test log is evidence. Treat it that way from day one.

The pattern regulators will look for

The UK Gambling Commission's Remote Customer Interaction guidance came into force in August 2022 and was expanded in 2024. It puts the burden on operators to identify and act on markers of harm, not just to have a policy on the shelf. Affordability triggers already kick in at £150 net deposits over a rolling 30 days. If you're running an experiment that changes when or how a nudge fires for a customer approaching that threshold, you need a defensible record of who saw what, when, and why the experiment was justified.

About 1 in 4 UK-licensed operators already fails to hit a satisfactory AML rating on first assessment. The margin for a messy experimentation setup is thin.

The minimum ledger

Here's the smallest data model I'd want to defend in a regulator meeting. One row per (user, experiment, assignment window). Joined to RG outcomes downstream.

• experiment_id and variant_id. Human-readable, versioned, immutable once shipped.
• hypothesis_ref. Pointer to the written hypothesis and the ethics or compliance sign-off that approved it. Not a Slack link. A durable document ID.
• user_id, assignment_ts, unassignment_ts. When the user entered and left the cell. If you rebalance traffic mid-flight, that's a new row.
• variant_payload_hash. A hash of the actual UI, copy, and thresholds shown. If someone edits the nudge text, the hash changes and you get a new variant.
• safeguard_delta. This is the important one. For each variant, an explicit description of what safety behaviour is added, removed, or delayed relative to the production default. If a cell removes or weakens a nudge, flag it.
• guardrail_metrics. Pre-declared metrics that will stop the experiment. Self-exclusion rate, deposit velocity above threshold, reversed withdrawals, complaint volume. With pre-declared bounds.
• stop_ts and stop_reason. Populated when the experiment ends. Never null after the end date.

That's the spine. Everything else, statistical power, segment cuts, revenue impact, hangs off it.

Joining to RG outcomes

The ledger is useless if it lives in your experimentation tool and your RG outcomes live in a separate warehouse that nobody joins. The join has to be routine. For every experiment touching a safety surface, you want a standing query that answers: for users in variant X, what was the 30-day and 90-day rate of self-exclusion, affordability trigger breach, and reversed-withdrawal activity, compared to control.

Run it while the experiment is live, not after. Pre-declare the guardrail thresholds. If the weakened-nudge cell shows a lift in deposit velocity past the £150 rolling window without a proportional lift in affordability checks completed, that is a stop condition, not a discussion topic.

What auditable actually means

Auditable means someone who did not build the system can reconstruct, months later, exactly which users were in which cell and what safety behaviour they were exposed to. A few practical tests:

1. Can you produce, for any user who filed a complaint, every experiment they were enrolled in during the 12 months prior, with variant payload hashes intact?
2. Can you show, for any experiment that touched a safety surface, the written pre-approval and the guardrail metrics as they were declared at start, not as they were rationalised at end?
3. If your experimentation platform disappeared tomorrow, is the ledger still readable from cold storage?

If the answer to any of those is no, you have a TikTok-shaped risk sitting in your stack. The upside is that this is a solved data-modelling problem. It's just work that most operators keep deferring because nothing has forced the issue yet. The disclosure this month probably counts as forcing the issue.

The operators who'll be fine in 2026 aren't the ones with the fanciest nudges. They're the ones whose experiment ledger reads like a lab notebook.

Sources

• UK Gambling Commission
• UK Gambling Commission