Page MenuHomePhabricator

Create growthexperiments_user_impact table in Wikimedia production
Closed, ResolvedPublic

Description

Table name: growthexperiments_user_impact
Cluster: I'm assuming it should go to x1 like all other GrowthExperiments tables.

Use case: cache secondary data related to a user's activity that is expensive to calculate (such as edited articles, thanks received, number of pageviews to the edited articles). See mw:Growth/Positive reinforcement for more information.

Table schema: json, sql

Expected usage patterns: basically a key-value store keyed by user ID, with some additional expiration mechanics. Web requests will get/set a single value corresponding to the current user when someone visits Special:Homepage (about 10K times a day); a maintenance script will batch-update values. (See store patch; maintenance script still WIP.)

Expected storage needs: about 5K per record (a JSON blob which we currently make no effort to compact), stored for 1) users who have registered in the last two weeks, 2) users who have registered within a year and made at least one edit, 3) users who have visited Special:Homepage recently. Using the guesstimate that enwiki is about half of all Wikimedia user accounts, that's about 150K acounts for 1), 1M accounts for 2), and 3) is going to be insignificant in comparison, so expected storage space is something like 6G. (The data is only actually needed when the user visits Special:Homepage, so we can almost arbitrarily reduce this if needed, at some UX cost for people who do visit their homepage but infrequently.)

Event Timeline

Change 831540 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/WikimediaMaintenance@master] createExtensionTables.php: Add growthexperiments_user_impact

https://gerrit.wikimedia.org/r/831540

Change 831542 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[operations/puppet@production] Add growthexperiments_user_impact to $private_tables

https://gerrit.wikimedia.org/r/831542

Change 831540 merged by jenkins-bot:

[mediawiki/extensions/WikimediaMaintenance@master] createExtensionTables.php: Add growthexperiments_user_impact

https://gerrit.wikimedia.org/r/831540

kostajh changed the task status from Open to In Progress.Sep 16 2022, 8:33 AM
kostajh triaged this task as Medium priority.
Tgr updated the task description. (Show Details)
Tgr updated the task description. (Show Details)
Tgr moved this task from In Progress to Code Review on the Growth-Team (Sprint 0 (Growth Team)) board.

This looks ok to me. I have a question though, will this table be cleaned up? Say a user registered and made an edit but never came back within a year, will that entry get deleted?

Yeah, the idea is to regularly refresh entries, and evict old ones.

@Tgr sounds good - we are ok with this table. Feel free to create it when needed!

Change 831542 merged by Ladsgroup:

[operations/puppet@production] Add growthexperiments_user_impact to $private_tables

https://gerrit.wikimedia.org/r/831542

I'm planning to run foreachwikiindblist growthexperiments.dblist extensions/WikimediaMaintenance/createExtensionTables.php growthexperiments on Monday.

Tgr changed the task status from In Progress to Open.Nov 8 2022, 12:45 AM
Tgr moved this task from In Progress to QA on the Growth-Team (Sprint 0 (Growth Team)) board.

I'm planning to run foreachwikiindblist growthexperiments.dblist extensions/WikimediaMaintenance/createExtensionTables.php growthexperiments on Monday.

This is done.