Thats right! Another roadmap item ticked off!
A/B testing thumbnails is here.
What is A/B Testing?
A/B Testing allows you to compare how two different thumbnails preform inside of discover. Each thumbnail will be shown to a % of users, and will generate analytics. These analytics include clicks and the click through rate!
How long can A/B Tests run?
A/B Tests run until enough data is generated to obtain sufficient results*. Tests typically take a couple of days, but can run for a maximum duration of 90 days. Tests can be ended manually at any time.
*Test duration depends on audience size
Note from EPIC
A/B testing compares the performance of two thumbnails by tracking how many players click on each. We use a 90% confidence threshold to determine statistical significance. So for example, if the Creator Portal tells you that thumbnail B’s CTR (1.52%) is 21.09% higher than thumbnail A's (1.23%), you can be ≥90% confident that the result is because thumbnail B performs better.
How can I conduct an A/B Test?
You can now upload 2 thumbnails during the "Upload Promotional Media" step of the submission process. Results from an A/B Test will be displayed under the release tab of your island.
Need Further Info?
Have a look at Epics Docs page to learn more!
How have the community reacted?
The community love the idea of A/B testing, as it is another great tool from Epic Games, allowing us creators to make our islands as great as possible.
Although this is just the start... Creators like DolphinDom call out for further filters to be added, to remove Bias CTR.
Personally, I love the addition, and agree with Dom that more filters would make the tool even more useful!
Thats it for now!
See you in the next one!
Olilz
Olilz
Hey! I am Olilz! I Have been playtesting maps for FCHQ for over 175 weeks! I enjoy creating maps myself & creating new experiences. I love to code and make yt videos!