Page MenuHomePhabricator

Investigate: Curation toolbar not advancing [4 hours]
Closed, ResolvedPublicBUG REPORT

Description

Steps to reproduce:

  1. Click the next button the curation toolbar

Expected behavior
Get to the next article

Actual results:
The next button turns grey and does not advance

This does not happen on all articles has happened to multiple users (see https://en.wikipedia.org/wiki/Wikipedia_talk:New_pages_patrol/Reviewers#Curation_toolbar_not_advancing?). The exact trigger has not yet been found. It feels like it might have happened with one of the changes by @ifried and her team as part of recent improvements.

Details

Related Gerrit Patches:
mediawiki/extensions/PageTriage : masterFix curation next button becoming disabled
mediawiki/extensions/PageTriage : masterAdd debug output for 'next in queue' toolbar button
mediawiki/extensions/PageTriage : masterFix addition of extraneous parameters when advancing to next article.

Event Timeline

Barkeep49 created this task.Sep 5 2019, 1:22 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 5 2019, 1:22 PM
Restricted Application added a project: Growth-Team. · View Herald TranscriptSep 5 2019, 3:07 PM

Possibly the same as T221815 and T41419

Maybe? This has happened a few times, some of which were around when @MMiller_WMF and company were doing the AfC integration which were fixed. I do a fair amount of patrolling and this particular kind of not advancing behavior is new for me.

The only issue that I could see - if you have a limited set of articles returned in the feed and you click on the last one - then the advancing button won't work. Or you start in the middle of the list - then, when you get to the end of the list, the arrow button won't allow you to go any further.
I did not see that a sorting order plays any role in the issue. @Barkeep49 - can you confirm that the case described above is not what you observed?

@Etonkovidova I think sorting order does matter. An example of an article where I can produce the bug at this moment is, with the queue sorted to oldest, https://en.wikipedia.org/wiki/Chichester_District_Council

My settings are:
State: Unreviewed
Type: Nominated for deletion, all others
That: Show all
5591 results

Another experienced reviewer (User:Rosguil) says they only see it at the ends of the queue not the middle.

JJMC89 changed the subtype of this task from "Task" to "Bug Report".Sep 6 2019, 2:16 AM
kostajh moved this task from Inbox to FY 2019-20 on the Growth-Team board.Sep 6 2019, 9:17 AM
kostajh added a subscriber: kostajh.

If you look at the Network tab and inspect the API requests when interacting with the new pages feed, you can get some additional information. When I use the settings mentioned in T232093#5470186 I can see that there is a warning that page ID 24993733 is missing metadata. In the past, this caused problems with navigating the queue (both from Special:NewPagesFeed and from the toolbar).

I'm putting this on Growth-Team's radar but given we haven't touched PageCuration in a while, I think it would make more sense for Community-Tech to look into this first. Let us know, though.

Etonkovidova added a comment.EditedSep 6 2019, 8:03 PM

@Barkeep49 - thank you for providing the additional details. I was checking the scope of the issue in the testing env in case it's more wide-spread - it does not seem so.

As @kostajh mentioned in the comment above, the issue might be caused by some pages' issues; it's helpful to have the specific article that may be the cause.

By way of another example, when sorting by oldest https://en.wikipedia.org/wiki/Schoomaker doesn't let me advance. Nor does https://en.wikipedia.org/wiki/Chichester_District_Council but it seems to work OK from there. So far I haven't been able to find a more recent example from the front of the queue than the ones I posted in the NPP talk page.

The Community Tech team will take a look and see this bug is related to our recent changes.

ifried renamed this task from Curation toolbar not advancing to Investigate: Curation toolbar not advancing [4 hours].Sep 9 2019, 5:34 PM

Change 535754 had a related patch set uploaded (by Samwilson; owner: Samwilson):
[mediawiki/extensions/PageTriage@master] Add debug output for 'next in queue' toolbar button

https://gerrit.wikimedia.org/r/535754

Would it be worth adding some debug output for the next button? (See patch above.)

When the 'next' link is used to go to the next page in the queue, it makes a request to the pagetriagelist API endpoint with the below parameters. It overrides offset and pageoffset to match the current page's creation date and ID respectively, but all other parameters come from the current filter values.

date_range_from	2019-09-05T16:00:00.000Z
date_range_to	2019-09-06T15:59:59.000Z
namespace	0
showreviewed	1
showunreviewed	1
showdeleted	1
showredirs	1
showothers	1
nppDir		newestfirst
dir		newestfirst
limit		1
mode		npp
offset		20190901055329
pageoffset	352

Could this be related to sorting direction and dates of creation? Could it be being interfered with by the date_range_* parameters?

I'm not sure why it's failing in the reported situations, but there are three ways it can fail:

  1. the API call is failing (unlikely);
  2. the response from pagetriagelist is a error message (also unlikely, unless we're somehow sending invalid parameters or values and not just out-of-range ones);
  3. the response is successful and does contain a page's details, but those details don't have a page title.

When does (3) happen? I mean, why are we even checking for that? I'm confused. Doesn't every page have a title?

I think we should add debugging output for that situation, outputting both the request details and the response, in the hope that whoever next encounters this will be able to look at their browser console and tell us what's going on.

DannyS712 added a subscriber: DannyS712.EditedSep 16 2019, 6:08 AM

Okay, so I did some debugging:
Steps to reproduce: sort by oldest first in the feed, view the oldest, and try to advance

Request: https://en.wikipedia.org/w/api.php?action=pagetriagelist&format=json&namespace=0&showunreviewed=1&showothers=1&nppDir=oldestfirst&dir=oldestfirst&limit=1&mode=npp&offset=20060728232522&pageoffset=6163191

Query parameters:

action: pagetriagelist
format: json
namespace: 0
showunreviewed: 1
showothers: 1
nppDir: oldestfirst
dir: oldestfirst
limit: 1
mode: npp
offset: 20060728232522
pageoffset: 6163191

when viewing https://en.wikipedia.org/w/api.php?action=pagetriagelist&format=json&namespace=0&showunreviewed=1&showothers=1&nppDir=oldestfirst&dir=oldestfirst&limit=1&mode=npp&offset=20060728232522&pageoffset=6163191 via url, the result is (in readable json)

API result
{
    "warnings": {
        "main": {
            "*": "Unrecognized parameters: nppDir, mode."
        }
    },
    "pagetriagelist": {
        "result": "warning",
        "pages_missing_metadata": [
            "24993733"
        ],
        "pages": [
            {
                "pageid": "7455424",
                "linkcount": "36",
                "creation_date": "20061015153348",
                "patrol_status": "0",
                "is_redirect": "0",
                "ptrp_last_reviewed_by": "0",
                "ptrp_reviewed_updated": "20190916040716",
                "reviewer": null,
                "title": "Netrokona",
                "category_count": "5",
                "csd_status": "0",
                "prod_status": "0",
                "blp_prod_status": "0",
                "afd_status": "0",
                "rev_count": "5",
                "page_len": "3924",
                "snippet": "Netrokona () is a big town and District headquarter in Netrokona District in the division of Mymensingh. It is the largest town and urban centre of...",
                "user_name": "Reinyday",
                "user_editcount": "21044",
                "user_creation_date": "20040904215804",
                "user_autoconfirmed": "1",
                "user_bot": "0",
                "user_block_status": "0",
                "user_id": "100726",
                "reference": "1",
                "user_experience": "experienced",
                "afc_state": "1",
                "copyvio": "",
                "recreated": "",
                "creation_date_utc": "20061015153348",
                "creator_user_page": "User:Reinyday",
                "creator_user_page_url": "//en.wikipedia.org/wiki/User:Reinyday",
                "creator_user_page_exist": true,
                "creator_user_talk_page": "User talk:Reinyday",
                "creator_user_talk_page_url": "//en.wikipedia.org/wiki/User_talk:Reinyday",
                "creator_user_talk_page_exist": true,
                "creator_contribution_page": "Special:Contributions/Reinyday",
                "creator_contribution_page_url": "//en.wikipedia.org/wiki/Special:Contributions/Reinyday",
                "ores_articlequality": "Stub",
                "ores_draftquality": ""
            }
        ]
    }
}
`

Since the code to advance checks that result.pagetriagelist.result === 'success', this fails because the result is a warning. This could be because either the page is missing metadata, the unrecognized parameters, or both. The patch provided solves the unrecognized parameters.

Looking a bit deeper, the nppDir, mode parameters are set from ext.pageTriage.article.js, where mw.pageTriage.ArticleList is defined. There, there is a mechanism to remove them (getApiParams) but it is never called.

Change 536799 had a related patch set uploaded (by DannyS712; owner: DannyS712):
[mediawiki/extensions/PageTriage@master] Fix curation toolbar not advancing

https://gerrit.wikimedia.org/r/536799

This comment was removed by DannyS712.

Change 536799 merged by jenkins-bot:
[mediawiki/extensions/PageTriage@master] Fix addition of extraneous parameters when advancing to next article.

https://gerrit.wikimedia.org/r/536799

Samwilson reassigned this task from Samwilson to DannyS712.Sep 16 2019, 7:08 AM

Thanks @DannyS712 that looks like it fixes it.

Ready for QA.

Restricted Application added a project: User-DannyS712. · View Herald TranscriptSep 16 2019, 7:08 AM

Change 535754 abandoned by Samwilson:
Add debug output for 'next in queue' toolbar button

Reason:
Issue fixed.

https://gerrit.wikimedia.org/r/535754

Okay, so I did some debugging:
Steps to reproduce: sort by oldest first in the feed, view the oldest, and try to advance

@DannyS712 Using these steps, I cannot reproduce on https://test.wikipedia.org/wiki (which should not have your changes yet).

Unless, after going to the oldest in the field, I go back to Special:NewPagesFeed and change the filter to sort by Newest. Then, I refresh the oldest article and click next. I assume this is because it has got to the end of the list. This doesn't seem unreasonable to me (although it would be nice to tell the user they have reached the end).

This happens on both test and https://en.wikipedia.beta.wmflabs.org/wiki (the latter does have your change).

I have not been able to reproduce with an article "in the middle" of the page triage queue. But, see below.

Since the code to advance checks that result.pagetriagelist.result === 'success', this fails because the result is a warning. This could be because either the page is missing metadata, the unrecognized parameters, or both. The patch provided solves the unrecognized parameters.

I see that the Unrecognized parameters: nppDir, mode message appears even if the result of the request is success. See, for example, https://test.wikipedia.org/w/api.php?action=pagetriagelist&format=json&namespace=0&showunreviewed=1&showothers=1&nppDir=oldestfirst&dir=oldestfirst&limit=1&mode=npp&offset=20121015140227&pageoffset=44324

This suggests to me that perhaps the problem is the missing metadata. I have tried to find pages in the queue with missing metadata, but have yet to be able to reproduce the problem.

This suggests to me that perhaps the problem is the missing metadata. I have tried to find pages in the queue with missing metadata, but have yet to be able to reproduce the problem.

Still cannot find pages with missing metadata. I don't know under which circumstances this happens.

Nevertheless, I think this might be the problem. The API returns 'warning' when there is a page with missing metadata (see https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/PageTriage/+/master/includes/Api/ApiPageTriageList.php#101).

I can reproduce the problem if I force the code to consider each page as have missing metadata. Not sure how valid that is as a reproduction.

@DannyS712 @Samwilson @MusikAnimal Thoughts?

aezell added a subscriber: aezell.Oct 23 2019, 5:12 PM

@DannyS712 You've claimed this task so we've not been working on it further. Do you intend to come back to it? If so, we will wait but if you've moved on, we can jump in and finish up the work.

HMonroy claimed this task.Oct 24 2019, 7:28 PM
HMonroy added a subscriber: HMonroy.

@DannyS712 I'm going to take a look. Please let me know if you have any new findings.

@DannyS712 I'm going to take a look. Please let me know if you have any new findings.

Thanks; sorry, I didn't see the previous message. It was assigned to me when we thought the issue was just the parameters

Change 535754 restored by Samwilson:
Add debug output for 'next in queue' toolbar button

https://gerrit.wikimedia.org/r/535754

Change 535754 merged by jenkins-bot:
[mediawiki/extensions/PageTriage@master] Add debug output for 'next in queue' toolbar button

https://gerrit.wikimedia.org/r/535754

I also was not able to replicate this issue. I have merged https://gerrit.wikimedia.org/r/535754 to help isolate what is happening when the next button is not advancing.

@Barkeep49 can you please let us know what the console is outputting when you come across this issue again?

Good news. I was quickly able to find two instances of this bug.

With the queue set to display from oldest
Unreviewed
Nominated + All others
Show all

I was unable to advance from https://en.wikipedia.org/wiki/Nine_of_Coins to https://en.wikipedia.org/wiki/Ten_of_Coins
and from Ten of Coins to https://en.wikipedia.org/wiki/Let_Me_Down_Gently

Change 547661 had a related patch set uploaded (by HMonroy; owner: HMonroy):
[mediawiki/extensions/PageTriage@master] Fix curation next button becoming disabled

https://gerrit.wikimedia.org/r/547661

@Barkeep49 thank you for your help debugging this issue!

The API responds with result.pagetriagelist. result === 'warning' when there is missing metadata, but the respond also contains the data for the next page; therefore, the next button should not get disabled. I modified the logic to allow advancing as long as the API returns the data for the next page.

This will probably be very difficult if not impossible to QA, but we can try! See T232093#5624961 for the findings; seems like if there's a page in the queue that's missing metadata (even if it's not the next page), the API response returns warning instead of success. So we'll need to somehow ensure a page is missing metadata somewhere in the queue. Why there are pages missing metadata is a separate problem, apparently.

The patch won't merge due to an unrelated issue with ORES.

Change 547661 merged by jenkins-bot:
[mediawiki/extensions/PageTriage@master] Fix curation next button becoming disabled

https://gerrit.wikimedia.org/r/547661

I have found a way of reproducing this issue, although it is not terribly realistic. It requires write access to the database. Therefore, I can only reproduce on my local system.

For one of the articles in the PageTriage queue, if you change the name in the page table to an empty string then this will cause it to report missing metadata for that article.

Prior to the latest patch (T232093#5706150), doing this would cause the toolbar to not advance beyond the article before the article with missing metadata.

After the latest patch, the toolbar will advance to the next article after the one with missing metadata ("jumping over" the problem article). Unless the problem article is the last in the queue, in which case it will not advance.

Still might be worth monitoring this in production when it goes live. I don't know the best way to do this, other than relying on the users to report it.

@DannyS712 and @Barkeep49: Now that the changes have been released to production, we would like to check in and see how they look to you. Thank you!

I have been unable to reproduce this bug so it feels like this has been successfully patched.

ifried closed this task as Resolved.Dec 16 2019, 7:50 PM
ifried moved this task from Product sign-off to Done on the Community-Tech (Kanban-Q2-2019-20) board.

Great, thanks, @Barkeep49! I'm marking this work as Done.