Wikimedia Apps/Team/Android/Activity Tab Experiment

In a Nutshell

This project is testing a redesigned Android tab—transforming the Edit tab into a richer, more inclusive Activity tab that surfaces insights into reading and donor behaviors (in addition to editing). Leveraging users’ local data enhances personalization while preserving privacy.

This A/B test aims to boost logged-in reader retention and multi-day engagement, contributing to the efforts of the 2025-2026 annual plan where we engage core consumer audiences.

Background

Android app users engage with Wikimedia projects in many ways beyond editing. The Android team is testing whether the Edit tab can evolve into a more engaging space for deep readers and app donors. The goal is to highlight personal insights while keeping sensitive data private. Each module in the Activity tab can be turned on or off, and reading history can be deleted, giving users full control over what they see.

Experiment Requirements

• A/B test setup: control (Edit tab) vs. experiment (Activity tab).
• Toggle controls for all modules.
• Info page for transparency on local data usage.
• Clickable modules with engaging CTAs.
• Navigation bar label updated to “Activity.”
• Maintain that users must be logged-in to access insights
• Modules should include
  • Time a user spent using the app
  • Monthly reading insights (e.g. articles saved)
  • Monthly editing insights (e.g. # of contributions)
  • All time impact (e.g. total thanks received)
  • Game Stats (e.g. average score)
  • Last in app donation (only available for known app donors)
  • Timeline of reading and editing behavior

How does the feature work?

• Some users in the experiment group see the new Activity tab, while others continue to see the Edit tab (control).
• Each module (reading, editing, donor history) can be toggled on or off.
• Empty modules display calls-to-action (for example: “Make an edit” or “Discover new articles”).
• Readers have the option to clear their history from the tab menu or Settings.
• Users can see a timeline of editing and reading behavior all in one place.

How success will be measured

Key Indicators

• The Activity tab has a 5% higher multi-day engagement than the Edit tab.
• App retention is statistically higher for the experiment group than the control group.

Guardrails

• No more than 25% of users turn the activity tab feature off.
• At least 80% of user feedback is neutral or positive.

Additional questions of interest

• How many users customize their modules?
• Which modules are turned off most often?
• How do new users (with 0 edits) use the tab?
• Do some users begin editing after seeing the tab?
• Do users engage more with reading lists, games, or donation features?

FAQ

• Q: Will my private data be shared?
  • A: No. Only public edit data is used externally; reading and donor data remain on your device.
• Q: How can I customize the tab?
  • A: Click the overflow menu and go to Customize. Use the toggle controls to hide or show modules as you prefer.
• Q: Why may modules be empty?
  • A: Empty modules include CTAs encouraging engagement (e.g., “Make an edit” if no recent edits), which are triggered if there is no reading or editing activity or if all modules have been disabled.
• Q: Why isn't my editing data updated?
  • A: Your editing insights are only updated once per day. Our current API has limitations, but we are working on a solution that will provide more real-time updates in a future release.
• Q: I noticed a bug or have a feature request, now do I report it?
  • A: Create a task and tag the Android App board or send an email to android-support@wikimedia.org.
• Q: Where can I find Suggested Edits?
  • A: It is available via More
• Q: What do the icons in the timeline indicate?
  • A: The pencil represents that you made an edit to the article shown. The magnifying glass indicates an article you landed on via Search. The bookmark icon indicates an article you've saved to your reading list. The link icon indicates an article you arrived at by clicking a link.

Next Updates & Feedback

Updates will appear on the Android main page each month. At the end of the experiment, we will share results and next steps on this project page.

Feedback? Leave a comment on the project discussion page—your insights shape this experience!

How does this work fit into the Wikimedia Foundation's Annual Plan?

Under the Wikimedia Foundation's Infrastructure Goal, and within the group of objectives focused on Wiki Experiences, is an objective related to improving the experience of consumers:

Wiki Experiences 3: Consumer experience Objective: Readers from multiple generations engage, and stay engaged, with Wikipedia, leading to measurable increases in retention and donation activity

Wiki Experiences 3.3 (WE3.2) Key Result: Demonstrate a practically significant increase in logged-in reader retention, as measured through A/B testing of one feature per platform, by the end of Q2 FY2025-26.

Android team hypothesis	Timeline	Phabricator epic
Wiki Experiences 3.3.2 If we expand the existing Edit tab on Android into a personalized activity hub that includes insights into reading and non-editing participation, we’ll see a 5% increase in multi-day engagement with the tab compared to the original version.	1 July 2025 – 30 September 2025	T399727

Several Wikimedia Foundation teams are committed to working on projects under the WE3.3 Key Result: Draft Hypotheses.

Wikimedia Foundation teams are approaching annual planning more iteratively this year, so rather than committing to larger year-long projects, this hypothesis is fairly narrow in scope. This should allow us to learn from a brief experiment and deliver value in smaller increments throughout the year, while also ensuring we have the flexibility to pivot as we learn.

Results

• Summary:
  • Our hypothesis was: If we expand the existing Edit tab on Android into a personalized activity hub that includes insights into reading and non-editing participation, we’ll see a 5% increase in multi-day engagement with the tab compared to the original version.
  • The Activity Tab’s return rate for logged-in readers was 11.6%, 30.6% higher than the Edit tab’s return rate of 8.9%. We can be 95% confident that this result is a consequence of the changes made and not a result of random chance. For that reason we consider the hypothesis supported.
  • For overall app retention, we did not see a statistically significant increase. App return rate for the control and experiment groups remained at 83%.
  • Next steps: Due to the feature yielding positive retention rates and not having a negative impact in overall app retention we will scale the feature to all users and explore adding it to the iOS app.

• Validation
  • KR 1.1 - 5% higher multi-day visits of Activity Tab among logged-in readers who have the new Activity Tab vs those that have the Edit Tab
    • Actual: The Activity Tab user return rate for logged-in readers was 11.6%, 30.6% higher than the Edit Tab user return rate of 8.9%.
  • KR 1.2 - A statistically significant higher retention rate of experiment group compared to control group
    • Actual: There was not a statistically significant change in the overall app retention of users.

• Guardrails
  • GR 2.1 - No more than 25% of users turns feature completely off
    • Actual: Only 0.7% of Activity Tab users turned the Activity Feature off.

• Curiosities
  • CR 3.1 - What % of people switch their new tab experience to something other than the default?
    • Actual: The Tab Activity feature was customized from default by 1.5% of logged-in uniques, and 3.7% of logged-in and logged-out uniques made changes to Activity Tab features.
  • CR 3.2 - Which modules are people turning off and at what rate?
    • Actual: The Editing Insight, Impact and Games Modules were the most frequently deactivated Activity Tab modules. None of the modules were turned off by more than 1% of Activity Tab unique users.
  • CR 3.4 - How many new account creations go on to make an edit after seeing the activity tab?
    • Actual: Over 1000 accounts were created during the 15-day period directly as a result of the feature. 2.1% of those accounts went on to make a productive edit, which is on par with the Android app editor ratio. However, 28.7% of new accounts went on to add an article to a reading list and 37.5% went on to play the Wikitrivia game through the Activity Tab CTA.