Hi Team,

I have been actively working on Mix-panel AB Test from a while now. I see there are few flaws that would have been taken care to make the AB Test experience little smooth.

As per the documentation and implementation -

The sequence of events are mentioned below:

• alias() method is called to tie the existing profile to the new ID you wish to use
• identify() and getPeople().identify() are both called with your ID upon the user signing up
• getPeople().set() is called to assign properties to the user
• flush() any waiting data to Mixpanel
• wait at least 2 seconds | 10 seconds | 20 seconds mostly get a callback after 60 seconds (but can't hold the user till that time, it will look like ANR)
• flush() again to receive a new response from Mixpanel
• OnMixpanelUpdatesReceivedListener will be triggered when an updated status is received
• OnMixpanelTweaksUpdatedListener will be triggered whenever a tweak value is updated

- Listener implementation ON-CREATE of the application (before setting people property as that will be received on Login Or Sign-Up)

public class MyAwesomeApplication extends Application  {
    private final MixpanelAPI mMixpanel =  MixpanelAPI.getInstance(this, "TOKEN");
    private Tweak<String> mFirstTweak = MixpanelAPI.stringTweak("myTweakName", "myTweakDefaultValue");

    @Override
    public void onCreate() {
        super.onCreate();
        mMixpanel.getPeople().addOnMixpanelUpdatesReceivedListener(new OnMixpanelUpdatesReceivedListener() {
            @Override
            public void onMixpanelUpdatesReceived() {
                mMixpanel.getPeople().joinExperimentIfAvailable(); // this will apply tweaks async. OnMixpanelTweaksUpdatedListener will be called right after
            }
        });
        mMixpanel.getPeople().addOnMixpanelTweaksUpdatedListener(new OnMixpanelTweaksUpdatedListener() {
            @Override
            public void onMixpanelTweakUpdated(Set<String> updatedTweaksName) {
                // Read your tweak here - will contain the correct value
                String serverValue = mFirstTweak.get();
            }
        });
    }
}

I am in touch with Mix-panel support from a while now. Let me describe the issues I have faced -

1. Sometimes listener not getting called, for the new user profile creation in mix panel usually sign up the case (frequency 2 out of 5 times)
   
As per the documentation, it says values are received with 10 seconds or more depending upon when the decide API gets called. The developer has to handle the first screen AB test by holding the user until callbacks are received in OnMixpanelUpdatesReceivedListener followed by OnMixpanelTweaksUpdatedListener.
   Drawback - Sometimes even if the user profile is created in mix panel it takes it own sweet time to give the experiment back. Even if you flush the values at regular intervals also. If the developer has the method to know when exactly the decide API is called and whether the experiment is received or not, to continue ahead it would be very helpful.

2. Opting out a user from the AB Test
   
I have been trying to opt out the targetted user from AB test if decision point has been crossed and tweak value is not received, but as far as I know and as per the discussion with support, we don’t have any such option from the client end to do.
   Let me give you a scenario - 


• After logging out the user, the user gets re-register for the update and tweak the listener on the onCreate method of the application.
  
_Mix panel replies -_ After you reset and unregister properties, Mix-panel sees the next events as a new user.



• The value received after re-registering is totally different from what we received earlier i.e. FALSE (FREE_TRIAL_DISABLED) – People Property Not Set
  
_Mix panel replies -_ I would expect the value received after re-registering to be either TRUE or FALSE because it is randomly assigned in the A/B test.



• Once the same user logs in again, ideally he should get the same tweak value i.e. TRUE (FREE_TRIAL_ENABLED)

  Mix panel replies - I would expect if the same first user re-logs in  that they would get the same tweak value as when they registered TRUE

Drawback - If the decision point is crossed and user went ahead with the default user flow (tweak value), it doesn’t make sense to give the same user experiment again and there is no guarantee that he will receive the same user flow tweak value. There is the possibility he can receive different tweak value as well.