Page MenuHomePhabricator

Community updates module: audience targeting
Closed, ResolvedPublic5 Estimated Story Points

Description

User story & summary:

As a new editor visiting my Newcomer homepage, I want community updates to be targeted and relevant to me, because otherwise it's just a distraction.

Parent task: T360485: [EPIC] Newcomer homepage: Community updates module (FY23/24 WE1.3 / FY24/25 SDS2.1.3)

Designs:

Community updates: Figma designs for Homepage module.

Community updates: Figma designs for the configuration form.

NOTE: Figma designs show the end state of the MVP release. This task only covers the Community Configuration settings and logic for targeting related to the module.
Acceptance Criteria:

Community updates Configuration form:

  • Add Minimum number of edits a user must have to be shown this banner field
  • DRAFT: Add Maximum number of edits a user must have to be shown this banner field

Targeting:
Given I'm logged in and have Display newcomer homepage checked,
When I visit Special:Homepage,
AND the Community Updates module is turned on at my wiki (enabled in Community Configuration)
AND my edit count matches the edit count range specified in Community Configuration,
Then my homepage will include the Community Updates module.

Event Timeline

@JFernandez-WMF: reminder to consider if rather than a "Minimum number of edits..." field, we should consider having checkboxes for who to target (and allow for selecting multiple groups). We could roughly mirror the Recent Changes filters: Newly registered account holders (0 edits), Newcomers (1-9 edits), Learners (10-499 edits), Experienced editors (500+ edits). That limits flexibility, but might be less confusing for admins when they configure the form? We also discussed the option to have these defaults, but also add a free-form field.

Personally I would suggest excluding the free-form field idea until after the pilot experiment (to limit complexity). But I'm OK with whatever you suggest here.

KStoller-WMF set the point value for this task to 5.Jun 11 2024, 4:30 PM
KStoller-WMF lowered the priority of this task from High to Medium.Jun 25 2024, 4:56 PM

I see we're setting a minimum, which seems useful for setting a bottom threshold. However this setting alone only allows to filter out, "not experienced enough editors" for the relevant update. While I understand the intent, I think offering just this setting creates a bias towards non-experienced editors. With this reasoning, would it make sense to set a default of 0 for the min value (the most inclusive value) and maybe(?) another field for setting maximum value of edits (with a default of infinity?). That would allow update administrators to target "more newcomer" users.

Said it differently, I'm trying to understand what value brings a module shown on the "newcomer homepage", if then our community config allows to filter out some of these newcomers from the target audience.

cc @KStoller-WMF

Change #1052289 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[mediawiki/extensions/GrowthExperiments@master] [WIP] Community updates: add minimum number of edits to receive banner

https://gerrit.wikimedia.org/r/1052289

I used the change as a test for T357532 and found some issues. Waiting for that task to be resolved to move forward with this change. Adding a new top property for a feature only deployed in Beta does not strictly require a migration, @Urbanecm_WMF let me know if you think this work should be independent from the migration work and I will update the change.

I agree having a min and max makes sense.

@JFernandez-WMF - can you update designs to either add in a max field or the option to select one or more user groups to target (https://phabricator.wikimedia.org/T365886#9831585). Thanks!

Sgs updated the task description. (Show Details)

Thinking a bit more about min/max settings I see a couple of problems if we add a maximum setting as well:

  1. Min/max validation of each field is related to the other field, CC does not provide a way of validating a field in terms of the value of another field. This seems a nice feature request.
  2. Setting a default value for the maximum is tricky. CC does not allow default empty values (null) for some types, like the case of the number and integer. We could display a "very big" number as the default, but we cannot set an "empty" value for the field to signal the special case "don't apply the validation constraint if value is not set".

For now I'd recommend to post-pone the addition of a maximum until we figure out the questions above. A way solving (2) would be to do the work for T365145. What do you think? cc @KStoller-WMF @Urbanecm_WMF @Michael

  1. Setting a default value for the maximum is tricky. CC does not allow default empty values (null) for some types, like the case of the number and integer. We could display a "very big" number as the default, but we cannot set an "empty" value for the field to signal the special case "don't apply the validation constraint if value is not set".

Making that field nullable and making null the default would seem to be the correct thing to do. If that is currently not possible, then T365145 sounds like a blocker for this task. But could you explain what currently blocking nullable Number fields from just working?

Note that I'm seeing nullable fields also in other Schemas:
https://gerrit.wikimedia.org/r/c/mediawiki/extensions/GrowthExperiments/+/1052143/10/includes/Config/Schemas/MentorListSchema.php
But that Schema also has issues for me, maybe they are related, not sure.

That being said, Figma currently only shows a field for the minimum number of edits. So maybe that "maximum number of edits"-field is not that important anyway. (Which makes sense)

  1. Setting a default value for the maximum is tricky. CC does not allow default empty values (null) for some types, like the case of the number and integer. We could display a "very big" number as the default, but we cannot set an "empty" value for the field to signal the special case "don't apply the validation constraint if value is not set".

Making that field nullable and making null the default would seem to be the correct thing to do. If that is currently not possible, then T365145 sounds like a blocker for this task. But could you explain what currently blocking nullable Number fields from just working?

I think it blocks the addition of a maximum not the full task as originally defined, with only a minimum. I've updated the description with DRAFT for that AC

Note that I'm seeing nullable fields also in other Schemas:
https://gerrit.wikimedia.org/r/c/mediawiki/extensions/GrowthExperiments/+/1052143/10/includes/Config/Schemas/MentorListSchema.php
But that Schema also has issues for me, maybe they are related, not sure.

Since justinrainbow validator understands the full json-schema draft-04 specification, using arrays for types is ok from the CC validator/store pov but not from the editor. Message generation would fail and JsonForm testers would not pick up any renderer.

That being said, Figma currently only shows a field for the minimum number of edits. So maybe that "maximum number of edits"-field is not that important anyway. (Which makes sense)

Yes, the maximum requirement comes from a suggestion in this task, see T365886#9949201

I personally think it would be best to consider targeting using the "User registration and experience" groups (that are already familiar to Wikimedians from Recent Changes). Example:

  • Newly registered account holders (0 edits)
  • Newcomers (1-9 edits)
  • Learners (10-499 edits)
  • Experienced editors (500+ edits)

That would allow event organizers to clearly target the right users. For example, a campaign organized around article creation might exclude "Newly registered account holders" and "Newcomers", but select "Learners" and Experienced editors" (since the organizer might want to focus efforts on editors that have some initial experience).

I assume this also would limit the complexity of validating min and max numbers, since we aren't providing an open field.

But I'm fine with just adding a minimum for now if we want to keep this simple.

@Sgs - alternatively, we can remove this task from the MVP release requirements (& remove it from this sprint), if you think refined acceptance criteria and designs are needed to help us avoid making a decision now that might result in migration work & additional effort in the future.

Since the minimum cap is already implemented I think we can move forward with it for the MVP and seek feedback from campaign and event organizers.

  • Experienced editors (500+ edits)

Do we know how many of these visit the homepage?

I'm reluctant to hide the update based on edit count in general because I'm not sure it aligns with our values. I think a small informative text would be more appropriate, eg: Event recommended for Newcomers (1-9 edits) . It feels off to not show an event to user because they have -+1 edit but maybe I'm missing how organizers target their audiences.

Since the minimum cap is already implemented I think we can move forward with it for the MVP and seek feedback from campaign and event organizers.

Sounds good.

Do we know how many of these visit the homepage?

There are quite a few Newcomer Task edits completed by experienced editors. Many of the "newcomers" who had access to the Homepage when they first created an account, continue to use the homepage. For example, we can look at the breakdown of Newcomer tasks split by "User Edit Count Bucket".

Although Newcomer tasks are clearly most commonly completed by newcomers in the 1-99 edits "User Edit Count Bucket", in June 2024 we see that 14,089 edits were completed by the 100-999 "User Edit Count Bucket". And 7,420 edits were completed via editors in the 1,000-9,999 "User Edit Count Bucket".

Change #1052289 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] Community updates: add minimum number of edits to receive banner

https://gerrit.wikimedia.org/r/1052289

Etonkovidova updated the task description. (Show Details)
Etonkovidova subscribed.

The minimum value field is present on Special:CommunityConfiguration/CommunityUpdates and works as expected. The creation of max value field (marked as DRAFT in the task description) might be addressed in another task if needed.