New 'Experiments' Functionality Now Live for Google Play Developers

"Experiments" for Google Play made its way into developer consoles today. Only a few short weeks after initial rumors circulated, access has been rolled out and testing has begun. We've got screenshots to share, and some updated info on what tests developers will actually be able to run.

Capabilities

To get started, you'll need to specify whether an experiment will be global or localized. Global experiments are limited to graphics, so to test profile page copy, developers will have to go local.

So what, exactly, can be tested? Choose from any of the following, but note that Google advises against testing too much at once. Being able to identify which metadata update generates success (or lack thereof) is incredibly important for these tests.

Up to three new versions of each item (a.k.a variants) can be added, and their performance will be compared to your current listing (or control group).

Developers will need to indicate how many of their users will be part of the test group. Google gives a great rundown of how this works:

If you type 30% as your audience, the remaining 70% of visitors to your store listing page will see your page’s current version. Your audience percentage will be divided equally between your experiment variants. For example, if you have a 30% audience and two variants in your experiment, each variant will be shown to 15% of users.

Once a user has been assigned to a test group, they will see the same test version every time they access the app's profile page until the experiment is shut down.

Most exciting are the data points Google will provide to help developers review their experiment, and ultimately decide which changes to implement. Installs by variant will be the strongest indicator of which metadata performs best for the specified audience. A "Performance" metric will reveal "expected change in install performance compared to the current version."

In order to get more great visuals on how the review panel functions, we started running an experiment of our own earlier today. Graphic components won't populate until Google feels enough data has been gathered, so we'll post again soon with more screenshots.

Additional Factors to Consider

• Despite what was anticipated, app purchase price cannot be tested using Experiments at this time. Neither can an app's title.

• Each app can only have one active experiment at any given time.

• Optimizing graphics plays an important role in ASO, so testing audience response to various icon and screenshot sets will be an incredible benefit to Android developers. Working to improve visibility and conversions simultaneously will be infinitely more seamless.