Watchlist Expiry: Update Special:RecentChanges to support watchlist expiry [medium]
Closed, ResolvedPublicBUG REPORT
Actions

Assigned To

Authored By

	dom_walden
	May 7 2020, 5:22 PM

Description

As a Watchlist Expiry user, I only want to see pages that I'm currently watching when I filter by watched pages in RecentChanges, so that I can focus on the pages that I currently care about.

Background: As a Watchlist Expiry user, when I view recent changes and filter to only show my watched pages, I don't want to see changes for expired items. I am guessing that Special:RecentChanges with watched filter should show the same changes as Special:Watchlist.

Acceptance Criteria:

Update Special:RecentChanges to support watchlist expiry, so that any expired items are not displayed when filtering by watched items
There are at least a couple of parts to this change:
- Special:RecentChanges have a couple of filters related to Watched items ("On Watchlist", "New Watchlist changes", "Not on Watchlist"). These need to exclude (or I guess include for "Not on Watchlist") expired items.
- Work out what this part of the code is doing and whether it needs to be updated to exclude expired items.
- Check if this affects API:RecentChanges (I don't think it does...)

Steps to reproduce problem:

Login as a user with watched pages
Go to Special:RecentChanges?watchlist=watched

Expected behavior: You should only see changes for watched pages which have not expired
Observed behavior: You see changes for all watched pages, even if they have expired (unless they have been cleaned up from the database)

Visual Example:

Details

Subject	Repo	Branch	Lines +/-
Fix RecentChanges watchlist filters when WatchlistExpiry is off	mediawiki/core	REL1_35	+9 -5
Fix RecentChanges watchlist filters when WatchlistExpiry is off	mediawiki/core	master	+9 -5
Fix watchlist query and filters for RecentChanges	mediawiki/core	REL1_35	+64 -6
Fix watchlist query and filters for RecentChanges	mediawiki/core	master	+59 -6
Exclude expired watchlist items from RecentChanges and RecentChangesLinked	mediawiki/core	master	+53 -38

Customize query in gerrit

Related Objects

Mentioned In: T259026: Postgres CI jobs failing
T258310: Watchlist Expiry: display clock & remaining watch period in recentchanges [MEDIUM]
T256526: Items from watchlist no longer bolded at Special:RecentChangesLinked [[SMALL]] --> high priority
Mentioned Here: rMW72ac5e10bf99: Localisation updates from https://translatewiki.net.

Event Timeline

dom_walden created this task.May 7 2020, 5:22 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 7 2020, 5:22 PM

ifried moved this task from New & TBD Tickets to Needs Discussion on the Community-Tech board.May 7 2020, 6:27 PM

ifried updated the task description. (Show Details)May 12 2020, 10:31 PM

ifried renamed this task from Watchlist Expiry: Update Special:RecentChanges to support watchlist expiry to Watchlist Expiry: Update Special:RecentChanges to support watchlist expiry [medium].May 12 2020, 11:13 PM

ifried moved this task from Needs Discussion to Up Next (May 6-17) on the Community-Tech board.

ifried moved this task from Up Next (May 6-17) to Kanban-2019-20-Q4 on the Community-Tech board.May 13 2020, 4:59 PM

ifried edited projects, added Community-Tech (Kanban-2019-20-Q4); removed Community-Tech.

Change 602211 had a related patch set uploaded (by Samwilson; owner: Samwilson):
[mediawiki/core@master] Exclude expired watchlist items from RecentChanges

https://gerrit.wikimedia.org/r/602211

gerritbot added a project: Patch-For-Review.Jun 4 2020, 5:33 AM

Samwilson claimed this task.Jun 4 2020, 5:38 AM

Samwilson moved this task from Ready to In Development on the Community-Tech (Kanban-2019-20-Q4) board.

Samwilson moved this task from In Development to Needs Review/Feedback on the Community-Tech (Kanban-2019-20-Q4) board.Jun 9 2020, 11:14 PM

This is ready for review.

Change 602211 merged by jenkins-bot:
[mediawiki/core@master] Exclude expired watchlist items from RecentChanges and RecentChangesLinked

https://gerrit.wikimedia.org/r/602211

Ready for QA.

ReleaseTaggerBot added a project: MW-1.35-notes (1.35.0-wmf.38; 2020-06-23).Jun 17 2020, 12:00 AM

Maintenance_bot removed a project: Patch-For-Review.Jun 17 2020, 12:10 AM

I think we are always filtering out expired items which have yet to be purged, even if the "On Watchlist" filter is unchecked or "Not on Watchlist" is checked.

Looking at the SQL in the logs, with "On Watchlist" unchecked we have the condition:

... AND (we_expiry IS NULL OR we_expiry > '<current timestamp>')...

That condition is false for expired items that haven't been purged yet.

With "Not on Watchlist" checked we have the extra condition:

...AND (wl_user IS NULL)...

Which is also false if the expired item has yet to be purged.

@Samwilson Please double check my reasoning here.

(P.S. sorry for breaking Silent Friday, but if I waited for Monday you might not get this message until your Tuesday, which seems a bit unfair.)

Samwilson moved this task from QA to In Development on the Community-Tech (Kanban-2019-20-Q4) board.Jun 24 2020, 3:15 AM

Change 607413 had a related patch set uploaded (by Samwilson; owner: Samwilson):
[mediawiki/core@master] Fix watchlist query for RecentChanges

https://gerrit.wikimedia.org/r/607413

gerritbot added a project: Patch-For-Review.Jun 24 2020, 6:11 AM

DannyS712 mentioned this in T256526: Items from watchlist no longer bolded at Special:RecentChangesLinked [[SMALL]] --> high priority.Jun 27 2020, 9:39 AM

MBinder_WMF edited projects, added Community-Tech (Kanban-2020-21-Q1); removed Community-Tech (Kanban-2019-20-Q4).Jul 14 2020, 7:21 PM

Restricted Application edited projects, added Community-Tech; removed Community-Tech (Kanban-2020-21-Q1). · View Herald TranscriptJul 14 2020, 7:21 PM

MBinder_WMF edited projects, added Community-Tech (Kanban-2020-21-Q1); removed Community-Tech.Jul 14 2020, 7:23 PM

ARamirez_WMF moved this task from Ready 🎬 to In Development 💻 on the Community-Tech (Kanban-2020-21-Q1) board.Jul 14 2020, 7:24 PM

This is ready for review.

ifried mentioned this in T258310: Watchlist Expiry: display clock & remaining watch period in recentchanges [MEDIUM].Jul 23 2020, 7:29 PM

Change 607413 merged by Samwilson:
[mediawiki/core@master] Fix watchlist query and filters for RecentChanges

https://gerrit.wikimedia.org/r/607413

Samwilson moved this task from Review/Feedback 💬 to QA 🐛 on the Community-Tech (Kanban-2020-21-Q1) board.Jul 28 2020, 2:31 AM

ReleaseTaggerBot added a project: MW-1.36-notes (1.36.0-wmf.3; 2020-08-04).Jul 28 2020, 3:00 AM

Maintenance_bot removed a project: Patch-For-Review.Jul 28 2020, 3:10 AM

Reedy mentioned this in T259026: Postgres CI jobs failing.Jul 28 2020, 1:35 PM

@Samwilson With Watchlist Expiry disabled, if I select one of the watchlist filters on RecentChanges I get the error:

[f42a4c9a5de4f48ddc484228] /wiki/Special:RecentChanges?watchlist=notwatched&limit=500&days=3&enhanced=1&urlversion=2 Wikimedia\Rdbms\DBQueryError from line 1695 of /vagrant/mediawiki/includes/libs/rdbms/database/Database.php: A database query error has occurred. Did you forget to run your application's database schema updater after upgrading?

Error 1054: Unknown column 'we_expiry' in 'where clause' (127.0.0.1)
Function: SpecialRecentChanges::doMainQuery
Query: SELECT rc_id,rc_timestamp,rc_namespace,rc_title,rc_minor,rc_bot,rc_new,rc_cur_id,rc_this_oldid,rc_last_oldid,rc_type,rc_source,rc_patrolled,rc_ip,rc_old_len,rc_new_len,rc_deleted,rc_logid,rc_log_type,rc_log_action,rc_params,comment_rc_comment.comment_text AS `rc_comment_text`,comment_rc_comment.comment_data AS `rc_comment_data`,comment_rc_comment.comment_id AS `rc_comment_cid`,actor_rc_user.actor_user AS `rc_user`,actor_rc_user.actor_name AS `rc_user_text`,rc_actor,wl_user,wl_notificationtimestamp,page_latest,(SELECT GROUP_CONCAT(ctd_name SEPARATOR ',') FROM `change_tag` JOIN `change_tag_def` ON ((ct_tag_id=ctd_id)) WHERE ct_rc_id=rc_id ) AS `ts_tags` FROM `recentchanges` JOIN `comment` `comment_rc_comment` ON ((comment_rc_comment.comment_id = rc_comment_id)) JOIN `actor` `actor_rc_user` ON ((actor_rc_user.actor_id = rc_actor)) LEFT JOIN `watchlist` ON (wl_user = 4 AND (wl_title=rc_title) AND (wl_namespace=rc_namespace)) LEFT JOIN `page` ON ((rc_cur_id=page_id)) WHERE (wl_user IS NULL OR ( we_expiry IS NOT NULL AND we_expiry < '20200728152109' )) AND (rc_timestamp >= '20200725152109') AND rc_new IN (0,1) ORDER BY rc_timestamp DESC LIMIT 500

Backtrace:

#0 /vagrant/mediawiki/includes/libs/rdbms/database/Database.php(1679): Wikimedia\Rdbms\Database->getQueryException(string, integer, string, string)
#1 /vagrant/mediawiki/includes/libs/rdbms/database/Database.php(1654): Wikimedia\Rdbms\Database->getQueryExceptionAndLog(string, integer, string, string)
#2 /vagrant/mediawiki/includes/libs/rdbms/database/Database.php(1227): Wikimedia\Rdbms\Database->reportQueryError(string, integer, string, string, boolean)
#3 /vagrant/mediawiki/includes/libs/rdbms/database/Database.php(1903): Wikimedia\Rdbms\Database->query(string, string, integer)
#4 /vagrant/mediawiki/includes/libs/rdbms/database/DBConnRef.php(68): Wikimedia\Rdbms\Database->select(array, array, array, string, array, array)
#5 /vagrant/mediawiki/includes/libs/rdbms/database/DBConnRef.php(313): Wikimedia\Rdbms\DBConnRef->__call(string, array)
#6 /vagrant/mediawiki/includes/specials/SpecialRecentChanges.php(373): Wikimedia\Rdbms\DBConnRef->select(array, array, array, string, array, array)
#7 /vagrant/mediawiki/includes/specialpage/ChangesListSpecialPage.php(1025): SpecialRecentChanges->doMainQuery(array, array, array, array, array, FormOptions)
#8 /vagrant/mediawiki/includes/specialpage/ChangesListSpecialPage.php(630): ChangesListSpecialPage->getRows()
#9 /vagrant/mediawiki/includes/specials/SpecialRecentChanges.php(168): ChangesListSpecialPage->execute(NULL)
#10 /vagrant/mediawiki/includes/specialpage/SpecialPage.php(600): SpecialRecentChanges->execute(NULL)
#11 /vagrant/mediawiki/includes/specialpage/SpecialPageFactory.php(635): SpecialPage->run(NULL)
#12 /vagrant/mediawiki/includes/MediaWiki.php(307): MediaWiki\SpecialPage\SpecialPageFactory->executePath(Title, RequestContext)
#13 /vagrant/mediawiki/includes/MediaWiki.php(940): MediaWiki->performRequest()
#14 /vagrant/mediawiki/includes/MediaWiki.php(543): MediaWiki->main()
#15 /vagrant/mediawiki/index.php(53): MediaWiki->run()
#16 /vagrant/mediawiki/index.php(46): wfIndexMain()
#17 /var/www/w/index.php(5): require(string)
#18 {main}

I am guessing these queries need to be behind the feature flag.

Samwilson moved this task from QA 🐛 to In Development 💻 on the Community-Tech (Kanban-2020-21-Q1) board.Jul 29 2020, 12:22 AM

Change 616962 had a related patch set uploaded (by Samwilson; owner: Samwilson):
[mediawiki/core@master] Fix RecentChanges watchlist filters when WatchlistExpiry is off

https://gerrit.wikimedia.org/r/616962

gerritbot added a project: Patch-For-Review.Jul 29 2020, 1:29 AM

Thanks Dom, I should've caught that earlier! New patch in review now.

Change 616967 had a related patch set uploaded (by Samwilson; owner: Samwilson):
[mediawiki/core@REL1_35] Fix watchlist query and filters for RecentChanges

https://gerrit.wikimedia.org/r/616967

Change 616967 merged by jenkins-bot:
[mediawiki/core@REL1_35] Fix watchlist query and filters for RecentChanges

https://gerrit.wikimedia.org/r/616967

ReleaseTaggerBot edited projects, added MW-1.35-notes; removed MW-1.35-notes (1.35.0-wmf.38; 2020-06-23).Jul 29 2020, 7:00 PM

Change 616962 merged by jenkins-bot:
[mediawiki/core@master] Fix RecentChanges watchlist filters when WatchlistExpiry is off

https://gerrit.wikimedia.org/r/616962

Maintenance_bot removed a project: Patch-For-Review.Jul 30 2020, 10:10 PM

Change 617440 had a related patch set uploaded (by Samwilson; owner: Samwilson):
[mediawiki/core@REL1_35] Fix RecentChanges watchlist filters when WatchlistExpiry is off

https://gerrit.wikimedia.org/r/617440

gerritbot added a project: Patch-For-Review.Jul 30 2020, 10:41 PM

On beta, tested Special:RecentChanges and Special:RecentChangesLinked for permanently and temporarily watched and expired watched pages. Only tested the "On Watchlist" and "Not On Watchlist" filters.

On my local Vagrant, with Watchlist Expiry disabled, repeated the same tests. Results looked fine and no errors in the logs.

Test environment: https://en.wikipedia.beta.wmflabs.org MediaWiki 1.36.0-alpha (72ac5e1) 06:44, 31 July 2020; local vagrant MediaWiki 1.36.0-alpha (72ac5e1)

@dmaza I saw that T252136#6350203 had been merged and ready for QA, but I won't move it into QA in case you are not finished with it yet.

In T252136#6351609, @dom_walden wrote:

@dmaza I saw that T252136#6350203 had been merged and ready for QA, but I won't move it into QA in case you are not finished with it yet.

It was Sam's patch, I only reviewed and hit the +2. I should be ready for QA now.

Change 617440 merged by jenkins-bot:
[mediawiki/core@REL1_35] Fix RecentChanges watchlist filters when WatchlistExpiry is off

https://gerrit.wikimedia.org/r/617440

Maintenance_bot removed a project: Patch-For-Review.Jul 31 2020, 10:11 PM

Ready for QA, although that might all be done.

In T252136#6353284, @Samwilson wrote:

Ready for QA, although that might all be done.

Thanks! I have nothing more to do here.

This looks good, as per the notes provided by Dom. I'm marking this work as Done.

Watchlist Expiry: Update Special:RecentChanges to support watchlist expiry [medium]Closed, ResolvedPublicBUG REPORTActions

Description

Details

Related Objects

Event Timeline

Watchlist Expiry: Update Special:RecentChanges to support watchlist expiry [medium]
Closed, ResolvedPublicBUG REPORT
Actions