Blog

When multiple concurrent experiments target the same audience on the same view of the game flow, the combination of experiment variations may alter their behaviors. These behaviors could be an effect of interaction between the experiments, making the results of experiments skewed and uninterpretable.

Use exclusion groups to prevent those interaction effects! We added this feature to increase versatility and agility in your experimentation practice to achieve more without making you wait for prior experiment completion.

With an exclusion group, you limit the participating target audience (players) to participate in one experiment at a time. This ensures that the same players don't see overlapping experiments within an exclusion group. This means that the data collected for one experiment is not affected by any other experiment, even if both the experiments are set up on the same game flow and targets the same set of players. Thus, making the experiment mutually exclusive.

For example, let’s say you want to run two experiments A & B defined as below:

• Tutorial room (A): Add a tutorial room as a section of gameplay that walks the player through the actions, they need to know to play
• Contextual lesson (B): Add a thematically relevant contextual lesson throughout each stage of the gameplay.

Considering each experiment receiving a total of 25% in “% traffic allocation”, the distribution looks like this:

In the above example, if you run both experiments at the same time to track the increase in engagement on the game without an exclusion group, you cannot be sure which change delivered positive or negative results. The reason being players who see both A & B behave differently from players who see just A or just B. Whereas, with the exclusion group, the traffic allocation happens without overlapping players’ data between the experiments, making A & B mutually exclusive.

In an exclusion group, the traffic is allocated and distributed at random in different buckets of players for each experiment. This ensures clean results where the change in conversion rate is attributed to the correct experiment without bias and overlap of the target audience.

We’ve heard your feedback and enabled the capability to also run longer duration experiments for up to 90 days.

We’re excited that PlayFab Experiments help you achieve more with exclusion groups and longer duration of experiment. Visit the documentation site to learn more and get started!

If you have questions or feedback, we would love to hear from you. Please leave a comment in our Forum.