Beta testing of the ReadingLists extension
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Tgr
	Aug 31 2017, 8:08 AM

Description

Things that should be delayed until ReadingLists is deployed on the Beta cluster (to make it available for testing to app developers as soon as possible), but should be done before it is deployed in production:

deduplicate rle_project into its own table
EXPLAIN all queries and make sure they have sane indexes
review DBA recommendations in T164990#3264314
(except for T182052: Batch reading list operations which needs more discussion)

Details

	Subject	Repo	Branch	Lines +/-
	Add ReadingLists extension	mediawiki/tools/release	master	+1 -0
	Deploy ReadingLists to the beta cluster	operations/mediawiki-config	master	+12 -0

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Resolved	Tgr	T168985 Develop a Reading List Service
Resolved	Tgr	T168975 Develop a MediaWiki extension for managing Reading Lists
Resolved	Tgr	T181107 Deploy Reading Lists Service to production
Resolved	Tgr	T174651 Beta testing of the ReadingLists extension
Resolved	cooltey	T177853 Test integrating Android app with Reading List Service

Event Timeline

Tgr created this task.Aug 31 2017, 8:08 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 31 2017, 8:08 AM

Tgr added a parent task: T168975: Develop a MediaWiki extension for managing Reading Lists.Aug 31 2017, 8:08 AM

The main concern is the footprint per list item- it should be much lower to support large scale usage.

Tgr mentioned this in rERLS10b471b0a14c: Database schema and DB handling code.Aug 31 2017, 8:49 AM

Tgr updated the task description. (Show Details)Aug 31 2017, 4:54 PM

In T174651#3569124, @jcrespo wrote:

The main concern is the footprint per list item- it should be much lower to support large scale usage.

Would that be resolved by moving rle_project to its own table, or are you worried about titles as well? I guess we could deduplicate those as well. @Mholloway do we have any data from the Android app on the level of overlap between lists (ie. total number of reading list items over all users vs. number of unique pages which appear in the reading list of some user)?

Unfortunately we have no data on the specific articles in users' reading lists, only general data on how frequently users add or delete items or change metadata (e.g., the list description), along with the user's total number of lists, the lists' sizes, and from where they're being accessed in the app.

https://meta.wikimedia.org/wiki/Schema:MobileWikiAppReadingLists

Tgr mentioned this in T168974: Setup Database schema for Reading Lists.Sep 1 2017, 5:09 AM

It is ok to not do that now, and wait to see what is the regular usage in space per user and entry, and what the ways to optimize are. That would be reasonable course of action.

My only request is to not forget about this. I had instances in the past where a developer said "I will do that later" and then when things go wrong "don't have the bandwith to fix it". I do not have problems with how it is now, but as a warning, I will block further updates if the table grows over 300GB -or before, if it causes server problems-, so it is not forgotten. I just want someone to be responsible and reactive if problems arise (which normally don't until peak usage), I hope you understand my fears.

Tgr claimed this task.Sep 1 2017, 9:07 AM

• Fjalapeno moved this task from Needs triage to Kanban on the Product-Infrastructure-Team-Backlog-Deprecated board.Sep 14 2017, 3:24 PM

• Fjalapeno edited projects, added Product-Infrastructure-Team-Backlog-Deprecated (Kanban); removed Product-Infrastructure-Team-Backlog-Deprecated.

Tgr mentioned this in T174126: Security review for the ReadingLists extension.Sep 25 2017, 10:46 PM

Tgr updated the task description. (Show Details)Oct 5 2017, 8:56 AM

Change 384908 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[operations/mediawiki-config@master] Deploy ReadingLists to the beta cluster

https://gerrit.wikimedia.org/r/384908

gerritbot added a project: Patch-For-Review.Oct 17 2017, 11:50 PM

Change 384909 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[mediawiki/tools/release@master] Add ReadingLists extension

https://gerrit.wikimedia.org/r/384909

Tgr updated the task description. (Show Details)Oct 17 2017, 11:57 PM

Added a deploy window for tomorrow.

Change 384908 merged by jenkins-bot:
[operations/mediawiki-config@master] Deploy ReadingLists to the beta cluster

https://gerrit.wikimedia.org/r/384908

Mentioned in SAL (#wikimedia-operations) [2017-10-18T22:27:51Z] <tgr@tin> Synchronized wmf-config/: T174651 / gerrit 384908 - deploy ReadingLists to beta - noop for production (duration: 00m 53s)