Readers/Reader Growth/Expanded Mobile Sections
Mobile sections are currently collapsed by default in order to aid navigation on pages. However, this can slow users down when trying to find content. We conducted an A/B test in December 2025 that auto-expanded all sections and introduced a pinned, collapsible section heading to improve browsing. The hypothesis was that this change will increase session length and time on page.
We actually found that this change actually lowered the retention rate for readers by about 1.5% and shortened the amount of time they spent onwiki. We suspect that auto-opening all the sections on mobile ended up causing navigation difficulties by creating a wall of text, resulting in readers feeling overwhelmed or frustrated and leaving. As a result, we decided not to build this feature.
Background
[edit]Mobile sections are currently collapsed by default, but users can configure them to expand by default in mobile settings.
Sections were originally auto-collapsed to save users time in navigating from section to section as they scroll through long paragraphs of text.
However, this also makes it difficult to browse and find content since users are required to first open individual sections before reading.
We wanted to test reader behavior to see whether with auto-expanded sections they are able to more quickly find the content they're looking for and more easily open and close sections.
This A/B test expanded all mobile sections regardless of user preference.
Hypothesis
[edit]If we launch an A/B test of a version of the mobile site which introduces navigation that opens all sections by default, we will see early indicators that signal towards an increase in session length.
Results
[edit]We had targeted an increase in session length as our primary success metric, but we saw actual session length of 52.2 seconds for Control compared to 44.1 seconds for Treatment, a -15% relative decline. For our 21-day retention guardrail metric, we targeted no harm, but we saw actuals for the 7-day cohort window (the first window to mature) at 42.3% for Control compared to 41.6% for Treatment, a -1.5% relative decline.
Given the strong signal from session length as an indicator metric and early retention metrics, we ended the experiment when the 7-day cohort windowâs 21-day cumulative retention was mature, so that we would get a view of the impact on 21-day cumulative retention without extending a negative treatment for longer just to allow the 14-day cohort window to mature.
We suspect auto-opening all the sections on mobile created navigation difficulties (wall of text in T1, compared to the Control providing many of the section headers at a glance as a proxy table of contents), resulting in readers feeling overwhelmed and bouncing.
Experimentation
[edit]We proposed that we run an A/B test which proposes a test version of paragraph collapsing on the mobile site, which does the following:
- Opens all sections by default
- Pins the section header the user is currently reading to the top of the page
- Makes it so tapping on the section header will collapse the section and allow the user to switch to another section
The main metrics we explored in this A/B test were:
- Session length
- Time on page
- Guardrail
- Retention
Nice to have/secondary metrics included:
- Internal referrals
- Images clicked
The experiment affected 10% of mobile users on Arabic, Chinese, French, Indonesian, and Vietnamese Wikipedias, beginning the week of December 8, 2025 and running for four weeks.
Experiment timeline
[edit]Phase 0 (2022): Identify problem
[edit]This has been a problem space we have been exploring over the past few years. The general problem we are trying to solve is:
- The mobile website is our most popular surface. Yet many of our sessions on mobile are short, raising the risk that readers are not finding what they are looking for.
- One of the obstacles to this is the fact that all content with the exception of the first paragrah and infobox are collapsed by default on mobile. This might make it difficult to find specific pieces of information
- We are interested in making it easier to find specific pieces of information on mobile
Phase 1 (Nov - Dec 2025): Experiment with feature idea
[edit]Steps:
- We propose testing opening all sections by default.
- The A/B test will try a mobile version that opens all sections by default, pins the active section header, and lets users collapse sections by tapping the header. It will measure session length, time on page, retention, and other secondary engagement metrics
- The experiment will run for 2-4 weeks
- Turn feature off
- Analyze data
- Report back on results and propose next steps
Questions this phase will answer:
- Is this feature useful to readers? (measured by overall engagement with sections)
- Does it help readers browse more content (measured by an increase of time on page)
- Does it help readers come back more frequently to Wikipedia? (measured by retention)
- Should we build this feature?
Hypothesis evaluation: Was the hypothesis correct?
â If yes â continue with discussing the proposed idea with communities
â If no â stop the project and document what we learned
Phase 2 (Jan 2026 - March 2026 if experiment is successful): Build feature
[edit]Based on our experiment results, we decided not to build the feature. Instead we opted take these learnings to create a new hypothesis to test adding a Table of Contents on mobile web in addition to auto-opening all article sections, as a way to get readers information faster while addressing the navigation difficulties mentioned above. The work of this new hypothesis is documented in WE3.10.2 Minerva Table of Contents.
Design
[edit]
Prototype: https://vector-2022.vercel.app/sticky-section-headers
Note that this prototype shows sections as collapsed by default. We would expect sections to be open by default, but section header behavior is as expected.
Mobile user options
[edit]We decided to disable the mobile user setting that expands or collapses the sections if enrolled in the experiment to prevent confusion in the user interface.
