seatgeek open sourced seatgeek/sixpack-java

A Java client for the Sixpack A/B testing framework

The SeatGeek engineering team is excited to announce the latest addition to the Sixpack A/B testing framework’s client list: sixpack-java. Designed with the goal of making A/B testing Android applications easy and painless, sixpack-java has a straightforward API and an easy setup process that should make measuring and analyzing your application design decisions a breeze.

If you’re unfamiliar with Sixpack or A/B testing in general, you can read more about it here.

Let’s take a look at how you might integrate sixpack-java into your Android app.

Android app integration

Note: it is assumed that before integrating the java client into your app that you have set up a running instance of Sixpack-server; for information on setting one up, check out the instructions here

First, you’ll need to add sixpack-java to your application dependencies. sixpack-java is available on the Sonatype snapshots repository while it’s in beta, so you’ll need to add the following to your build.gradle:

1
2
3
4
5
6
7

repositories {
    maven { url 'https://oss.sonatype.org/content/repositories/snapshots' }
}

dependencies {
    compile 'com.seatgeek:sixpack:0.1-SNAPSHOT'
}

Now that you’ve resolved your dependencies, you can add an A/B test to your app. Let’s initialize a Sixpack client by creating a new Sixpack instance using the SixpackBuilder:

1
2
3
4

Sixpack sixpack = new SixpackBuilder()
        .setSixpackUrl("http://api.mycompany.com/sixpack")
        .setClientId(getCachedClientId())
        .build();

For reference, your getCachedClientId() method might look something like this:

1
2
3
4
5
6
7
8
9

public String getCachedClientId() {
    SharedPreferences prefs = context.getSharedPreferences("sixpack", Context.MODE_PRIVATE);
    String clientId = prefs.getString("sixpack_client_id");
    if (clientId == null) {
        clientId = Sixpack.generateRandomClientId();
        prefs.edit().put("sixpack_client_id", clientId).apply();
    }
    return clientId;
}

There are a two important things to take note of here: - You need to replace the url used in the setSixpackUrl() call with the url that points to your Sixpack-server deployment - The client id. The client id is the identifier used by the Sixpack-server to keep track of which clients have received which alternatives in the experiments they’re participating in. It is very important that the client id doesn’t change between sessions or else your users may see one alternative one time visiting the app and then another the next. That will probably have a fairly significant impact on your results too, so, just don’t do it. We recommend generating the client id once (there’s a helper method available, Sixpack.generateRandomClientId()) and caching that value in SharedPreferences so that it can be used again later (see the above code for an example of how that might work).

Note: we recommend maintaining a singleton instance of Sixpack using your favorite DI implementation, we use dagger for this

Alright, now you can create an experiment for testing the color of a button in your UI:

1
2
3
4
5

Experiment buttonColor = sixpack.experiment()
        .withName("Button Color")
        .withAlternative(new Alternative("Red"))
        .withAlternative(new Alternative("Green"))
        .build();

This will create a new experiment called “Button Color” with two alternatives, “Red” and “Green”. Once your client starts participating in this experiment, you’ll see it show up in the Sixpack-web dashboard. So let’s start it!

You start the test by calling Experiment#participate() and passing in the appropriate callbacks. Java 8 lambdas are used here for brevity.

1
2
3
4
5
6
7
8
9
10

buttonColor.participate(
        (participatingExperiment) -> {
            // success! save the participating instance for later so that we can convert it and set our button color
            this.participatingExperiment = participatingExperiment;
            button.setBackgroundColor(participatingExperiment.selectedAlternative == redAlternative ? R.color.button_red : R.color.button_green);
        },
        (experiment, error) -> {
            // failure, check network connection and try to participate again, you should also likely fallback to a default
        }
);

Now your test is live in the dashboard!

Finally, when the user clicks the button (assuming that’s the action that you’re measuring in this example) you can go ahead and fire the convert() message to Sixpack.

1
2
3
4
5
6
7
8
9
10

public void onClick(View button) {
   participatingExperiment.convert(
            (convertedExperiment) -> {
                // success!
            },
            (experiment, error) -> {
                // failure, check network connection and try to convert again
            }
    );
}

And that’s it, you’ve now successfully tested that button color!

Advanced usage

You might take a look at this API and say that there are several calls that do not need to be repeated more than once, and you’re not wrong! For instance, if you have a heavily trafficked part of your app under test, it will be to your advantage to call participate early in your application’s lifecycle and cache the ParticipatingExperiment in a way that will make fetching the selected alternative and calling convert() as easy as possible as to not hold up the rest of your UI from initializing. That said, participating early is risky and isn’t appropriate for most cases because you don’t want to participate in a test that your user never gets a chance to convert. Be careful and be sure to validate your tests are calling participate and convert at the right times before going into production by using a proxy or logging.

Here are a few “advanced usage” scenarios that you may want to consider for improving the sixpack-java integration in your app:

1. You could expose your ParticipatingExperiments as RxJava Observables and .cache() the results
2. In addition to having a singleton Sixpack instance in DI, you can put your Experiments and ParticipatingExperiments in your dagger modules so that they can have a lifecycle outside some of your application’s components
3. Instead of generating a random client id, for your users that are registered and have ids within your own system, you can use their user uuid as the Sixpack client id and later use Sixpack-server’s API to join your A/B testing results with your application’s users!

Conclusion

We can’t wait to see the great things you’ll build with sixpack-java! We’ve been using sixpack-java internally, but it is beta software at the moment, so please integrate it into your apps and let us know if you have any issues with it here at our github.

Have fun building!

P.S. If you’re interested in helping us create the best mobile event ticketing experience on Android, we’re hiring.