An earlier attempt to fetch page {page title} failed. To limit server load, retries have been blocked for 30 minutes.
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Assigned To

Authored By

	kostajh
	Feb 6 2023, 5:49 PM

Description

Error

mwversion: 1.40.0-wmf.21
reqId: 9f37cb34-9073-4c43-9725-97412d678199
Find reqId in Logstash

normalized_message

An earlier attempt to fetch page "{page title}" failed. To limit server load, retries have been blocked for 30 minutes.

exception.trace

Impact

Logspam. We log error statuses from PageViewInfo unconditionally, which we don't want to do for pvi-cached-error-title. We can track how often that happens in statsd, though, and monitor via Grafana.

Notes

4,281,221 entries since wmf.21 got to group2.

Fallout from rEGREcd34408d3f83: Process more articles when fetching page view data

Details

Request URL: https://jobrunner.discovery.wmnet/rpc/RunSingleJob.php

Subject	Repo	Branch	Lines +/-
ComputedUserImpactLookup: Avoid infinite pageview lookup loop	mediawiki/extensions/GrowthExperiments	master	+19 -14
ComputedUserImpactLookup: Reduce logspam for page view rate limiting	mediawiki/extensions/GrowthExperiments	wmf/1.40.0-wmf.22	+11 -1
ComputedUserImpactLookup: Reduce logspam for page view rate limiting	mediawiki/extensions/GrowthExperiments	master	+11 -1

Customize query in gerrit

Related Objects

Mentioned In: T324675: UserImpact: Fetch information for more articles when calculating most-viewed-articles data point
Mentioned Here: rEGREcd34408d3f83: Process more articles when fetching page view data

Event Timeline

kostajh triaged this task as High priority.Feb 6 2023, 5:49 PM

kostajh created this task.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 6 2023, 5:49 PM

kostajh updated the task description. (Show Details)Feb 6 2023, 5:50 PM

kostajh edited projects, added GrowthExperiments-ImpactModule; removed GrowthExperiments.

kostajh moved this task from Incoming to Ready for Development on the Growth-Team (Sprint 0 (Growth Team)) board.

kostajh mentioned this in T324675: UserImpact: Fetch information for more articles when calculating most-viewed-articles data point.Feb 6 2023, 5:55 PM

Change 886955 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@master] ComputedUserImpactLookup: Reduce logspam for page view rate limiting

https://gerrit.wikimedia.org/r/886955

gerritbot added a project: Patch-For-Review.Feb 6 2023, 10:28 PM

kostajh claimed this task.Feb 6 2023, 10:28 PM

kostajh moved this task from Ready for Development to Code Review on the Growth-Team (Sprint 0 (Growth Team)) board.

kostajh updated the task description. (Show Details)

Change 887729 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/GrowthExperiments@master] ComputedUserImpactLookup: Avoid infinite pageview lookup loop

https://gerrit.wikimedia.org/r/887729

Change 886955 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] ComputedUserImpactLookup: Reduce logspam for page view rate limiting

https://gerrit.wikimedia.org/r/886955

ReleaseTaggerBot added a project: MW-1.40-notes (1.40.0-wmf.23; 2023-02-13).Feb 8 2023, 10:00 AM

Change 887848 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@wmf/1.40.0-wmf.22] ComputedUserImpactLookup: Reduce logspam for page view rate limiting

https://gerrit.wikimedia.org/r/887848

Change 887848 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@wmf/1.40.0-wmf.22] ComputedUserImpactLookup: Reduce logspam for page view rate limiting

https://gerrit.wikimedia.org/r/887848

Mentioned in SAL (#wikimedia-operations) [2023-02-09T09:49:59Z] <kharlan@deploy1002> Started scap: Backport for [[gerrit:887848|ComputedUserImpactLookup: Reduce logspam for page view rate limiting (T328945)]]

Mentioned in SAL (#wikimedia-operations) [2023-02-09T09:51:50Z] <kharlan@deploy1002> kharlan: Backport for [[gerrit:887848|ComputedUserImpactLookup: Reduce logspam for page view rate limiting (T328945)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet

Mentioned in SAL (#wikimedia-operations) [2023-02-09T09:59:06Z] <kharlan@deploy1002> Finished scap: Backport for [[gerrit:887848|ComputedUserImpactLookup: Reduce logspam for page view rate limiting (T328945)]] (duration: 09m 06s)

ReleaseTaggerBot edited projects, added MW-1.40-notes (1.40.0-wmf.22; 2023-02-06); removed MW-1.40-notes (1.40.0-wmf.23; 2023-02-13).Feb 9 2023, 10:00 AM

thcipriani moved this task from Untriaged to Feb 2023 on the Wikimedia-production-error board.Feb 9 2023, 4:45 PM

kostajh moved this task from Code Review to QA on the Growth-Team (Sprint 0 (Growth Team)) board.Feb 24 2023, 9:17 AM

Change 887729 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] ComputedUserImpactLookup: Avoid infinite pageview lookup loop

https://gerrit.wikimedia.org/r/887729

In T328945#8643614, @gerritbot wrote:

Change 887729 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] ComputedUserImpactLookup: Avoid infinite pageview lookup loop

https://gerrit.wikimedia.org/r/887729

See https://gerrit.wikimedia.org/r/c/mediawiki/extensions/GrowthExperiments/+/887729/4#message-1a56e75782848ed9b521c29dd2b4fa20291845d0 for testing instructions for that patch.

ReleaseTaggerBot edited projects, added MW-1.40-notes (1.40.0-wmf.25; 2023-02-27); removed MW-1.40-notes (1.40.0-wmf.22; 2023-02-06).Feb 24 2023, 10:00 AM

Maintenance_bot removed a project: Patch-For-Review.Feb 24 2023, 10:11 AM

kostajh closed this task as Resolved.Mar 1 2023, 8:28 PM