Page MenuHomePhabricator

Update PrefUpdate schema to include edit count
Closed, ResolvedPublic3 Estimated Story Points

Description

Background

For the desktop improvements project, we would like to determine how opt-in and opt-out rates vary by the number of edits a user has. This will allow us to determine if there are differences between way more experienced editors and less experienced editors perceive the new vector interface. See also the previous investigation in T242419: [SPIKE] Instrument opt-in/opt-out mechanism for desktop improvements project

Timing

This is required by EOM March 2020.

Acceptance criteria

  • Add edit count bucket to the PrefUpdate schema

QA Steps

QA Results

ACStatusDetails
1T243071#5925455
2T243071#5925455

Event Timeline

ovasileva updated the task description. (Show Details)

The code for PrefUpdate lives in WikimediaEventsHooks::onUserSaveOptions file. To achieve this goal we need to:

  • edit the Schema:PrefUpdate wiki page and add edit count bucket
  • (SEE NOTE BELOW) edit the WikimediaEventsHooks and bump the schema revision
  • edit the $commonData array and add new editCount bucket

That should be all.

Note: usually we define Schema versions in skin.json or extension.json.. Those values are used on the JS side. Because the PrefUpdate schema is tracked on the backend side, there is no entry in the extension.json (only hardcoded in code), which IMHO is bad and it would be nice to update also the code to

  • store the schema revision in the extension.json
  • in the hooks file just use the version from extension.json instead of hardcoding one in hooks file

We (@Jdrewniak, @phuedx, and I) voted this as a medium today in part because there is chance to unify client side edit count bucket related code with the code that lives in onUserSaveOptions. @phuedx could you remind me again where the client side code lives that we would want to unify?

I think there's a solid case for migrating the user edit count bucketing code from the clients (see below) to the server. I think that there's a case to be made for putting User#getEditCountBucket in Core, WikimediaEvents, and EventLogging.

@phuedx could you remind me again where the client side code lives that we would want to unify?

WikimediaEvents seems like a reasonable place to house this code as 1) Is used "by Wikimedia Foundation sites for custom logging of events"; and 2) Popups and QuickSurveys both explicitly depend on it.

@phuedx I already tried to do that and there was lack of agreement on where to put the getEditCountBucket method. Please check T210106: Provide a reusable getEditCountBucket function for analytics purposes, there is a patch that adds this function on the JS side, but we abandoned that work.

Change 566791 had a related patch set uploaded (by Phuedx; owner: Phuedx):
[mediawiki/extensions/WikimediaEvents@master] [WIP] Add user edit count bucketing

https://gerrit.wikimedia.org/r/566791

I submitted the above change as a proof of concept for an approach that would allow us to centralise the user edit count bucketer in WikimediaEvents while not requiring that repositories bind to some API (requiring a hard dependency).

As discussed, this isn't blocked on T210106: Provide a reusable getEditCountBucket function for analytics purposes.

Implementing a simple bucketing function alongside the PrefUpdate instrumentation is cheap and what's required here. The larger conversation of lowering the cost of having this definition in one place can continue in parallel.

However, in the spirit of tidying up, I'd also recommend breaking out the PrefUpdate instrumentation into its own class, e.g.

PrefUpdateInstrumentation.php
namespace MediaWiki\Extensions\WikimediaEvents;

class PrefUpdateInstrumentation {
    public static function onUserSaveOptions( \User $user, array $options ) : void {
      /* ... */
    }

    public static function getBucketedUserEditCount( \User $user ) : string {
      /* ... */
    }

    // NOTE: This should accept an instance of Title.
    private static function isKnownSettingsPage( \OutputPage $out ) : boolean {
      /* ... */
    }
}

Change 571564 had a related patch set uploaded (by Phuedx; owner: Phuedx):
[mediawiki/extensions/WikimediaEvents@master] PrefUpdate: Extract instrumentation code

https://gerrit.wikimedia.org/r/571564

Change 571565 had a related patch set uploaded (by Phuedx; owner: Phuedx):
[mediawiki/extensions/WikimediaEvents@master] PrefUpdate: Tidy DocBlocks and add type hints

https://gerrit.wikimedia.org/r/571565

Change 571566 had a related patch set uploaded (by Phuedx; owner: Phuedx):
[mediawiki/extensions/WikimediaEvents@master] PrefUpdate: Added bucketedUserEditCount property

https://gerrit.wikimedia.org/r/571566

Change 571564 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] PrefUpdate: Extract instrumentation code

https://gerrit.wikimedia.org/r/571564

Change 571565 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] PrefUpdate: Tidy DocBlocks and add type hints

https://gerrit.wikimedia.org/r/571565

Change 571566 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] PrefUpdate: Added bucketedUserEditCount property

https://gerrit.wikimedia.org/r/571566

Change 566791 abandoned by Phuedx:
Add user edit count bucketing

Reason:
Per the above (I still actually need to respond…)

https://gerrit.wikimedia.org/r/566791

phuedx added a subscriber: Niedzielski.

@Niedzielski IIRC you said that you had experience with QAing EventLogging on the Beta Cluster and that you'd be willing to help @Edtadros QA this.

Noting that @Mayakp.wiki would be doing data QA following the QA from @Edtadros

Test Result

Status: ✅ PASS
OS: macOS Catalina
Browser: Chrome
Device: MBP

Test Artifact(s):

QA Steps

  1. Navigate to https://en.wikipedia.beta.wmflabs.org/wiki/Main_Page
  2. Log in
  3. Click "Preferences"
  4. Change any user preference that's whitelisted
  5. ✅ AC1: Observe that a PrefUpdate event is logged
    • Event was logged, see gif
  6. ✅ AC2: Observe that the event has a valid bucketedUserEditCount property
    • A valid bucketedUserEditCount was logged, see gif

T243071.gif (988×1 px, 3 MB)

Not able to check events in mysql because event logging on beta is turned off and will require a reboot. Will get back to this as soon as we get confirmation from Analytics that eventlogging is working as expected.

@Nuria got to the bottom of things (and thanks for the tip @Edtadros via @Etonkovidova)! The database _is_ read-only right now. However, @Nuria says that validating by looking at the all-events log is sufficient. @Mayakp.wiki, back to you :]

I will do the Data-QA for this on T246028 and update the task if I find any issues.

Ok to sign off. Completed data QA in T246028.