Hello @Bobocicada! Would it be possible for you to review the specification of the dataset in the description above, as a last check, before I start generating it?

Hi @Urbanecm_WMF,

Thank you for setting this up so quickly! The specification looks good overall. Two small requests:

Opt-out users: The current spec excludes users who explicitly opted out. Would it be possible to either include them with their original mentorshipState value, or add a separate field like optedOut: true/false? I'd prefer to have the complete picture and handle the filtering on my end.

"Controlled by MediaWiki" filter: Could you clarify what this excludes exactly? For example, does this exclude users whose mentorship was enabled via a manual mentor claim rather than the automatic randomization?

Everything else — the fields, the scope (Jan 2021 – Feb 2025), and the use of userId for correlation with the growthmentorship dump — looks great.

In T420387#11721059, @Bobocicada wrote:

Opt-out users: The current spec excludes users who explicitly opted out.

That is done to protect the user's privacy. The user might have various motivations to opt out from mentorship, including "my mentor is abusive to me". We do not want to point extra attention to those users, to make retaliation more difficult. I don't think we would be comfortable to make that dataset public.

Would it be possible to either include them with their original mentorshipState value, or add a separate field like optedOut: true/false?

For the record, opted out status is considered to be a mentorshipState value (it is essentially a tristate field, with the values of enabled/disabled/optedout).

I'd prefer to have the complete picture and handle the filtering on my end.

I understand you might prefer to have more control and understanding. Is there any reason why the research couldn't progress if this dataset was not available? If so, we might consider its limited release (to you specifically, provided you'd sign an NDA with us). However, I'd like to understand the reasoning before considering a limited release. Would you be able to provide more details about the need here?

"Controlled by MediaWiki" filter: Could you clarify what this excludes exactly? For example, does this exclude users whose mentorship was enabled via a manual mentor claim rather than the automatic randomization?

User options are technically freely editable by the user they concern (it is possible to call the action=options API and set the value to anything you'd like). For this reason, the actual value of the user property might not be recognised as valid (eg. if the user sets it to "thisisaninvalidoption", MediaWiki wouldn't be able to interpret that as a mentorship state). This point of the specification excludes user options that are not set to either enabled/disabled/optedout. There is no UI-exposed way how to override this particular preference (besides claiming, as discussed on the other ticket). In turn, this would only exclude users that are somehow tampering with the user property via the API. In retrospective, I probably could have written this in a better way.

Everything else — the fields, the scope (Jan 2021 – Feb 2025), and the use of userId for correlation with the growthmentorship dump — looks great.

Thank you for confirming!

Hi @Urbanecm_WMF,

Thank you for the thorough explanation and for being transparent about the privacy considerations — I fully respect that.

Why the opted-out data matters:
I'm comparing outcomes between users who were assigned a visible mentor at registration and those who were not. If a user originally had access to a mentor (growthexperiments-homepage-mentorship-enabled not equal to 0), that experience may have already influenced their behavior — even if they later opted out. Excluding these users means the "assigned" group would only contain users who stayed in the program, which could make the program appear more effective than it actually is. The research can progress without this data, but the results would carry this potential bias, which I'd need to report as a limitation.

Three technical clarifications:

1. Could you share roughly how many users in the dataset scope (Jan 2021 – Feb 2025, enwiki) have the opted-out status? This would help me assess whether the bias from excluding them is negligible or significant enough to warrant the NDA option.

2. In T419002, you mentioned:

we automatically set the growthexperiments-homepage-mentorship-enabled user property to 0 (meaning "mentorship module disabled") if the randomisation determines the user should have mentorship disabled

And in this ticket, you described mentorshipState as:

it is essentially a tristate field, with the values of enabled/disabled/optedout

Just to confirm: are these referring to the same underlying property? And what are the actual stored values — numeric (0/1) or string (enabled/disabled/optedout)? I want to make sure I interpret the dump correctly once it's generated.

3. Is it possible for a user whose growthexperiments-homepage-mentorship-enabled is set to 0 (not selected by the rollout) to opt out? Or is opting out only available to users whose value was set to 1? I expect the answer is no, but I want to make sure I'm not making incorrect assumptions.

Thanks again!

In T420387#11723445, @Bobocicada wrote:

Why the opted-out data matters:
I'm comparing outcomes between users who were assigned a visible mentor at registration and those who were not. If a user originally had access to a mentor (growthexperiments-homepage-mentorship-enabled not equal to 0), that experience may have already influenced their behavior — even if they later opted out. Excluding these users means the "assigned" group would only contain users who stayed in the program, which could make the program appear more effective than it actually is. The research can progress without this data, but the results would carry this potential bias, which I'd need to report as a limitation.

Thank you for the clarification. This makes sense to me. Let's make the final decision based on the clarifications below!

Three technical clarifications:

1. Could you share roughly how many users in the dataset scope (Jan 2021 – Feb 2025, enwiki) have the opted-out status? This would help me assess whether the bias from excluding them is negligible or significant enough to warrant the NDA option.

As of today, 1,416 English Wikipedia users who registered in this period opted out of mentorship.

2. In T419002, you mentioned:

we automatically set the growthexperiments-homepage-mentorship-enabled user property to 0 (meaning "mentorship module disabled") if the randomisation determines the user should have mentorship disabled

And in this ticket, you described mentorshipState as:

it is essentially a tristate field, with the values of enabled/disabled/optedout

Just to confirm: are these referring to the same underlying property? And what are the actual stored values — numeric (0/1) or string (enabled/disabled/optedout)? I want to make sure I interpret the dump correctly once it's generated.

The underlying data is stored as integers. I was suggesting translating the integers into strings. I can provide the raw value to you instead (excluding the 2s, meaning "opted out"), and/or provide the code I used to generate the dataset. Up to you :). Both the description here and the discussion in the other ticket refer to the same concept.

3. Is it possible for a user whose growthexperiments-homepage-mentorship-enabled is set to 0 (not selected by the rollout) to opt out? Or is opting out only available to users whose value was set to 1? I expect the answer is no, but I want to make sure I'm not making incorrect assumptions.

Not directly (via the UI the users have available). They can call the API in theory, but I doubt that would happen very often. In addition to that, if an user not selected by the rollout is claimed by a mentor, their status will change to 1 (selected). At that point, they can opt out.

Hi @Urbanecm_WMF,

Thanks for the detailed clarifications — very helpful!

After discussing with my advisor (Prof. Ceren Budak), we've decided to go with the non-NDA option, i.e., excluding the opted-out users. 1,416 is a small enough fraction of the full registered population that the impact on our estimates should be minimal, and we'll note it as a limitation in the paper.

For the export format, would it be possible to use the string labels (enabled/disabled/optedout) rather than raw integers? I think that would be easier to interpret and less error-prone on our end.

Also, once the dump is generated, how should I access it? Will it be uploaded to a public dump location, or is there another way to receive it?

Thanks again for all the help on this!

Hi @Bobocicada, thank you for the answers. This makes sense. With all the questions answered, I'm moving this further through the team's board, so it can be picked up in one of the next sprints.

For the export format, would it be possible to use the string labels (enabled/disabled/optedout) rather than raw integers? I think that would be easier to interpret and less error-prone on our end.

Sure, will do!

Also, once the dump is generated, how should I access it? Will it be uploaded to a public dump location, or is there another way to receive it?

We'll post a link to the dataset on this ticket once it's generated and ready for use. We'll also share the code used to generate it for transparency and posterity.

Hi @Urbanecm_WMF,

Thank you so much for the update and for accommodating my formatting request!

I noticed the task was recently moved to the "Up next Sprint" board. Do you happen to have a rough estimate of how long it might take before the dataset is generated and ready to access?

Thanks again for all your help!

I have finally got to this task and prepared the dataset. I published my outputs at https://analytics.wikimedia.org/published/datasets/one-off/growth/growthexperiments-mentorship-enabled-T420387/. Both the Jupyter notebook and the dataset are published there.

Before I close this, @Isaac, would it be possible for you to review this?

Hi @Urbanecm_WMF,

Thank you for publishing the dataset and the notebook -- Initial analysis with the released data points to a positive and statistically significant retention effect of the mentorship module assignment (and I do see some very interesting patterns). But before I share any findings, I'm staying cautious in interpreting it though, because of three specific things I noticed while working with the data:

1. Raw up_value column

In our earlier discussion I opted for the string labels. After workingwith the data I realized I should have asked for the raw integer instead. Specifically: for users registered between commit 8c1d665b5 (2021-09-14) and e3f70f699 (2022-03-02), HomepageHooks::onLocalUserCreated only ran for users in the Homepage rollout (~25% at the time). The remaining ~75% kept the property unset, and convert_up_property() maps NaN to "enabled" the same as an explicit '1'. With the current two-column release I cannot distinguish the two groups and have to drop this 5-month window.

Would it be possible to add a column with the underlying integer (excluding 2)? You mentioned earlier this would be feasible.

2. Residual above the configured rollout percentage

Aligning the released file with the rollout commits and excluding users in Special:Log/growthexperiments?subtype=mentorassignmentchanges (those manually claimed mentee), the observed P(enabled) per window is:

p10 (2022-03-07 .. 2023-07-10): 11.75% vs configured 10%
p25 (2023-07-11 .. 2023-10-04): 26.15% vs 25%
p50 (2023-10-05 .. 2025-02-02): 50.57% vs 50%
p75 (2025-02-03 .. 2025-02-16): 73.91% vs 75%
p100 (2025-02-17 onward): 98.65% vs 100%

Since the release filter restricts to event.isSelfMade == true, the autocreated / byemail / create2 paths are already excluded. Within that filtered population, what mechanisms could produce a value other than the configured percentage at registration? I want to make sure I characterize this residual correctly.

Note: I think this residual is definitely not mostly driven by the our agreed exclusion of opt-out users (1,416 total across 4 years, contributing at most a sub-0.1% downward shift in enabled fraction per segment).

3. Users in dataset.tsv with no row in growthexperiments_mentor_mentee

I cross-joined the released file (dataset.tsv from https://analytics.wikimedia.org/published/datasets/one-off/growth/growthexperiments-mentorship-enabled-T420387/, generated 2026-06-05) against the 2026-05-30 enwiki growthexperiments_mentor_mentee dump (20260530-enwiki-growthexperiments_mentor_mentee.sql.gz from https://dumps.wikimedia.org/other/growthmentorship/), and observed that a small fraction of users in dataset.tsv have a mentorshipState value but no row in the assignment dump (under any gemm_mentor_role), AND no event in Special:Log/growthexperiments referencing them. In total, 16,812 users (~0.50% of the 2022-03..2025-02 enwiki cohort I'm looking at) fall into this category. The fraction rises monotonicallyover the cohort window:

• 2022-03 to 2022-12: 0.22 – 0.38%
• 2023-01 to 2023-12: 0.29 – 0.54%
• 2024-01 to 2024-12: 0.49 – 0.91%
• 2025-01: 1.29%
• 2025-02: 1.86%

Could you help me understand what produces this discrepancy — i.e., why these users appear in dataset.tsv with a mentorshipState value but have no corresponding row in growthexperiments_mentor_mentee and no entry in Special:Log/growthexperiments?

Thanks

Hi @Bobocicada,

Thank you so much for the review! This is appreciated. I changed the convert_up_property() function to the following:

def convert_up_property(val):
    if pd.isna(val):
        return 'unset'
    elif val in ['0', '1', '50']:
        return val
    else:
        return None

which should produce the correct data.

I pushed the updated code to analytics.wikimedia.org; it should be available there soon(ish). I'll take a look at the other questions soon.

Users in dataset.tsv with no row in growthexperiments_mentor_mentee

I took a look at this. First, I merged the two datasets you mentioned (I used the real-time internal replica of growthexperiments_mentor_mentee, rather than the dump, but that shouldn't affect much):

>>> from wmfdata import mariadb
>>> import pandas as pd

>>> dfMentorMentee = mariadb.run('SELECT * FROM growthexperiments_mentor_mentee', 'enwiki', use_x1=True)
>>> dfPublished = pd.read_csv(
    '/srv/published/datasets/one-off/growth/growthexperiments-mentorship-enabled-T420387/dataset.tsv',
    sep='\t'
)

>>> dfMerged = dfPublished.merge(dfMentorMentee, left_on='userId', right_on='gemm_mentee_id', how='left', indicator=True)
>>> dfMerged = dfMerged[dfMerged['_merge'] == 'left_only'].drop(columns='_merge')

>>> dfMerged.userId.count()
552669

>>> dfMerged.loc[(dfMerged.mentorshipState != 'unset') & (dfMerged.mentorshipState != '0')].userId.count()
4

It seems we have 4 users once we exclude the unset and 0 mentorshipStates. This is still unexpected, but much less concerning than the percentages reported in your comment. Unfortunately, given the data retention period for logging data (90 days), I'm unable to check what happened much, and I can speculate. Notably, all four missing users have mentorshipState of 50, which should trigger a proactive assignment of mentor (cf. MentorManager::setMentorshipStateForUser). However, the proactive mentor assignment happens via a job and as such, it is possible it didn't run properly (eg. due to a job infrastructure error). Given it affects merely 4 users, it should be ignorable I believe.

For the values of unset (NaN) and 0, missing rows are more interesting. I did some review, and it appears that in early 2021, growthexperiments_mentor_mentee rows were only provisioned if homepage was enabled (ie. depending on GEHomepageNewAccountEnablePercentage), cf. HomepageHooks at that time. This changed in Feb 2021 via r666432.

However, those two explanations would explain a monotonic _decrease_, not a monotonic _increase_. Would it be possible to share the code you used to determine it increases as you described? That would allow me to take a look at specific users and values.

Hi @Urbanecm_WMF,

Thank you for the quick fix on convert_up_property. Here's a short summary of what I found; the full analysis with code is in here if you want the details:

On the users in dataset.tsv that have no row in growthexperiments_mentor_mentee snapshot. I did the same comparison on the public dump and got 552,377 such users, close to your 552,669 from the internal replica. I went through them one by one to see why they are missing: about 524,000 registered before mentorship program was active on en.wiki (their local registration timestamp is earlier than 2021-06-01), and about another 21,000 are currently indefinitely blocked on en.wiki. The remaining few thousand are mostly old global accounts whose home wiki is not en.wiki and therefore have no local en.wiki registration row — none of them registered after the mentorship rollout. One small question I'd love your read on: do you know why these pre-rollout users end up in the published dataset.tsv at all? I don't have a clean explanation from the released data.

On the values in the new release. After downloading the updated file I only see three values of mentorshipState: unset (2,524,086), 0 (2,457,315), and 50 (32) — there is no 1. Could you confirm this is expected, i.e. that the explicit-enabled path simply does not write 1 in current code, and that unset is the canonical value for "the user did not turn off the mentorship UI"? I want to make sure I'm not misreading the absence of 1 (as when I treat unset as 1, mentorship module rollout rate starting from 2022-03-07 is quite the same as the rate you set).

On the 2021-09-20 → 2022-03-06 window. In the released data, 80.386% of users registered in this window have mentorshipState = unset. According to the system configuration at the time, the Newcomer Homepage was only shown to 25% of new users, and within that 25%, the mentorship module was randomly enabled for 20% of them. Under that configuration, only 25% × 20% = 5% of users should have ended up with mentorship enabled, and 25% × 80% = 20% should have ended up with it explicitly disabled. The remaining 75% never saw the Newcomer Homepage at all. The number 80.386% is essentially 75% + 5%: the 75% who never saw the homepage, plus the 5% who saw it and were randomly assigned to the enabled arm.

The problem is that for those 75% who never saw the homepage, the mentorship assignment code never ran. Their mentorshipState stayed unset, and convert_up_property maps unset to enabled, so they end up in the released file as if they were in the mentorship-enabled group. But they were never actually assigned to either arm — the code that decides "enabled vs disabled" was never invoked for them.

Conceptually, the 75% of users who never saw the newcomer homepage are not really part of the mentorship-module experiment — they were the population covered by a different rollout at the time (the Newcomer Homepage rollout). The value unset currently carries two different kinds of users at once: those who were in the homepage rollout and did not explicitly turn mentorship off, and those who were not in the homepage rollout and for whom the mentorship assignment code never ran. Once these two groups are collapsed into the same bucket, it becomes hard to isolate the effect of the mentorship module on its own during this window.

Would either of these be feasible on your side?

• (a) Add a column in the release — for users registered in 2021-09-20 → 2022-03-06 — flagging whether the user was in the Newcomer Homepage rollout at the time (equivalently, whether onLocalUserCreated actually wrote a value for them), or
• (b) Pre-filter those users who were never shown the homepage out of dataset.tsv directly, so the released file only contains users for whom mentorship is a meaningful treatment contrast.

One more question, could you clarify how a user ends up with mentorshipState = 50? Specifically, I want to know whether it is written automatically by the system at registration time (in the same random-assignment path that produces unset), or whether it is set later through some adjustment path — for example, a user who was initially assigned no mentor module (0) and later had their state changed.

Hi @Bobocicada,

Thank you for the response and for the notebook, this is very helpful.

On the values in the new release. After downloading the updated file I only see three values of mentorshipState: unset (2,524,086), 0 (2,457,315), and 50 (32) — there is no 1. Could you confirm this is expected, i.e. that the explicit-enabled path simply does not write 1 in current code, and that unset is the canonical value for "the user did not turn off the mentorship UI"? I want to make sure I'm not misreading the absence of 1 (as when I treat unset as 1, mentorship module rollout rate starting from 2022-03-07 is quite the same as the rate you set).

Yes, that is expected. In MediaWiki's user properties system, each user property has a default value. The user_properties database table contains "overrides" of that default (so if the value is something other than the default, it would be visible as a row in user_properties). This is done for space management reasons (the default value is supposed to be the most common one). Note that conditional defaults exist (where certain user properties can have a different default based on a specific condition), but they are not enabled for the mentorship-enabled property.

For the mentorship-enabled property, the default value is 1 (to the best of my knowledge, it has never been changed). For this reason, if something attempts to set the value to 1, the system detects that and responds by deleting the user_properties row rather than writing 1 there.

As a consequence, it is not possible to tell apart "row was never written" and "row was written as 1". This is also why the gradual mentor rollout code works by "the user has a random chance to NOT have mentorship" (rather than the other way around) – the default value is 1, so we need to define when we want a zero written, not the other way around. This is different to the "homepage enabled" property, which defaults to 0, and we write it to 1 for all new users.

One more question, could you clarify how a user ends up with mentorshipState = 50? Specifically, I want to know whether it is written automatically by the system at registration time (in the same random-assignment path that produces unset), or whether it is set later through some adjustment path — for example, a user who was initially assigned no mentor module (0) and later had their state changed.

50 is never written on signup. It is reserved for situations in which mentorship is enabled later. If you see 50 for a specific user, their mentorship state was previously different (either disabled [and they were claimed by someone] or they opted out and then opted back in). Hope this makes sense.

Would either of these be feasible on your side?
(a) Add a column in the release — for users registered in 2021-09-20 → 2022-03-06 — flagging whether the user was in the Newcomer Homepage rollout at the time (equivalently, whether onLocalUserCreated actually wrote a value for them), or
(b) Pre-filter those users who were never shown the homepage out of dataset.tsv directly, so the released file only contains users for whom mentorship is a meaningful treatment contrast.

Sure thing! I'll update the notebook and post here when it's updated.

The remaining few thousand are mostly old global accounts whose home wiki is not en.wiki and therefore have no local en.wiki registration row — none of them registered after the mentorship rollout. One small question I'd love your read on: do you know why these pre-rollout users end up in the published dataset.tsv at all? I don't have a clean explanation from the released data.

Very good question! I'm looking into this, as the explanation is not immediately clear to me.

Hello @Bobocicada,

Thank you for your patience. I have published a new version of the data, which includes the new column you asked for. Unfortunately, I was not able to find out why event_sanitized.serversideaccountcreation reports created users that aren't recorded in the DB. I added a JOIN on mediawiki_user (a monthly copy of the actual user table from production), which should filter out those users. I'll fill a follow-up ticket to learn more on this.

Would you mind taking another look at the dataset, please?

@Isaac mentioned they would review the code today or tomorrow as well.

Just chiming in to say that I took a pass on the code and everything looks good with one clarifying question below. Thanks @Urbanecm_WMF for the work on this and @Bobocicada for close engagement/questions!

Would either of these be feasible on your side?
(a) Add a column in the release — for users registered in 2021-09-20 → 2022-03-06 — flagging whether the user was in the Newcomer Homepage rollout at the time (equivalently, whether onLocalUserCreated actually wrote a value for them), or
(b) Pre-filter those users who were never shown the homepage out of dataset.tsv directly, so the released file only contains users for whom mentorship is a meaningful treatment contrast.

Sure thing! I'll update the notebook and post here when it's updated.

I see the homepageState property to show whether a particular user didn't have the homepage enabled, which is meant to identify users who didn't technically have mentorship disabled but also wouldn't have seen the homepage so had no way to access a mentor. After the Homepage rollout was complete though in 2022, would those users who previously were in the control condition (not shown Homepage) had that value updated? Or would they still have a homepageState value of 0 assuming they didn't explicitly go in and enable it somehow, which I suspect isn't a huge proportion?

In T420387#12027011, @Isaac wrote:

I see the homepageState property to show whether a particular user didn't have the homepage enabled, which is meant to identify users who didn't technically have mentorship disabled but also wouldn't have seen the homepage so had no way to access a mentor. After the Homepage rollout was complete though in 2022, would those users who previously were in the control condition (not shown Homepage) had that value updated? Or would they still have a homepageState value of 0 assuming they didn't explicitly go in and enable it somehow, which I suspect isn't a huge proportion?

We never enabled access to the Homepage for legacy users, so the property should still be zero for the control group users (unless they enabled it themselves, ofc). Maybe we should do that sometime, but so far it didn't happen (and after all, if you survived despite no Homepage, it probably wouldn't give you too much usefulness).

We never enabled access to the Homepage for legacy users, so the property should still be zero for the control group users (unless they enabled it themselves, ofc). Maybe we should do that sometime, but so far it didn't happen (and after all, if you survived despite no Homepage, it probably wouldn't give you too much usefulness).

Makes sense and useful for this data analysis too. Then all looks good from my perspective!

Hi @Urbanecm_WMF ,

I'm wrapping up analysis on the released dataset and ran into a small pattern I'd like your help interpreting.

In all phases after homepage was rolled out to 100% (registrations from 2022-03-07 onward), I expected almost every user to have homepageState = '1'. In the data I see 1.3%–2.9% per phase that are not '1'. The breakdown of the residual:

Almost the entire residual is 'unset'. Across all five phases combined there are only 4 users with a literal '0'.

To get a sense of who these users are, I drew a random sample of 25 of them and queried the API. What I saw was unexpected:

• 14 of the 25 are currently autoconfirmed or extendedconfirmed, with editcounts ranging from a handful up to several thousand. For these users, the median time from registration to their first edit is 86 seconds, and nearly all of their first edits are to their own User: page. Some of these first edits are the Dashboard auto-generated User has enrolled in [[Wikipedia:Wiki_Ed/...]] template, but others are unrelated content (e.g., self-introductions, sandbox setup, or article edits).
• 5 of the 25 have made zero edits since registration.
• The remaining 6 have very low editcounts (1–5), with mixed first-edit type — again, a mix of User has enrolled in ... templates and ordinary first edits.

My first guess was that these are experienced users who disabled the homepage themselves at some point after registration. That fits the first group reasonably well. What I can't explain under the same story is the zero-edit group — if they never used the account, it's hard to see how they would have actively disabled anything.

With that as context, three questions.

1. Can a regular user change their homepageState through the normal Wikipedia UI, without using the API? You mentioned earlier (in T419002) that for the mentorship preference there is no UI-exposed toggle and a user would need to call the action=options API to change it. I want to confirm whether the homepage preference is the same, or different — for example, whether Special:Preferences exposes a toggle for it, or there is a link inside Special:Homepage that disables it.
2. When a user disables the homepage, what does the field show afterwards — is it written as '0', or does it disappear and show up as 'unset'?
3. Apart from a user disabling it, is there any other reason a user might not have homepageState = '1' in these 100% coverage phases? For example, something going wrong at signup — the property failing to be written, an error happening, or a brief window during a code update where some users weren't covered. I'm not assuming any of this — the zero-edit subgroup above is what makes me wonder whether something non-user-driven reason is also contributing.

Thanks

Thank you for the questions, @Bobocicada.

Can a regular user change their homepageState through the normal Wikipedia UI, without using the API? You mentioned earlier (in T419002) that for the mentorship preference there is no UI-exposed toggle and a user would need to call the action=options API to change it. I want to confirm whether the homepage preference is the same, or different — for example, whether Special:Preferences exposes a toggle for it, or there is a link inside Special:Homepage that disables it.

Special:Preferences does include a toggle for the Homepage, and I would not be surprised if users getting to a certain level of experience manipulate that property (but I don't have any hard data for it). Here is how that looks like:

When a user disables the homepage, what does the field show afterwards — is it written as '0', or does it disappear and show up as 'unset'?

It shows as unset. The user property defaults to 0, and we do not store explicit rows for users whose value matches the default one.

Apart from a user disabling it, is there any other reason a user might not have homepageState = '1' in these 100% coverage phases? For example, something going wrong at signup — the property failing to be written, an error happening, or a brief window during a code update where some users weren't covered. I'm not assuming any of this — the zero-edit subgroup above is what makes me wonder whether something non-user-driven reason is also contributing.

I would suggest taking a look at how the registration was made. Autogenerated edits like User has enrolled in [[Wikipedia:Wiki_Ed/...]] strongly suggest an external tool (such as the Wiki Ed Dashboard) might be creating those accounts. Such tools might influence how the Homepage provisioning code runs (for example, since accounts are global, a tool might create a tool against meta.wikimedia.org [where GrowthExperiments doesn't exist] and since Growth features are only automatically enabled at the user's primary wiki [NOT at wikis where the account is autocreated], they would be left with no Homepage). Hard to say with certainity without knowing specific usernames (and even then, I might not be able to pinpoint what happened precisely).

Hope those thoughts are helpful.