ElasticaWrite.php on line
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Description

Error

mwversion: 1.43.0-wmf.25
reqId: 16fe7390e587f686689b1351
Find reqId in Logstash

normalized_message

[{reqId}] {exception_url}   TypeError: Argument 3 passed to CirrusSearch\DataSender::sendWeightedTagsUpdate() must be of the type array, null given, called in /srv/mediawiki/php-1.43.0-wmf.25/extensions/CirrusSearch/includes/Job/ElasticaWrite.php on line

Frame	Location	Call
from	/srv/mediawiki/php-1.43.0-wmf.25/extensions/CirrusSearch/includes/DataSender.php(129)
#0	/srv/mediawiki/php-1.43.0-wmf.25/extensions/CirrusSearch/includes/Job/ElasticaWrite.php(181)	CirrusSearch\DataSender->sendWeightedTagsUpdate(string, string, null)
#1	/srv/mediawiki/php-1.43.0-wmf.25/extensions/CirrusSearch/includes/Job/JobTraits.php(139)	CirrusSearch\Job\ElasticaWrite->doJob()
#2	/srv/mediawiki/php-1.43.0-wmf.25/extensions/CirrusSearch/includes/Updater.php(467)	CirrusSearch\Job\CirrusGenericJob->run()
#3	/srv/mediawiki/php-1.43.0-wmf.25/extensions/CirrusSearch/includes/Updater.php(275)	CirrusSearch\Updater->pushElasticaWriteJobs(string, array, Closure)
#4	/srv/mediawiki/php-1.43.0-wmf.25/extensions/GrowthExperiments/includes/NewcomerTasks/AddLink/LinkRecommendationUpdater.php(139)	CirrusSearch\Updater->updateWeightedTags(MediaWiki\Page\PageIdentityValue, string)
#5	/srv/mediawiki/php-1.43.0-wmf.25/extensions/GrowthExperiments/maintenance/refreshLinkRecommendations.php(291)	GrowthExperiments\NewcomerTasks\AddLink\LinkRecommendationUpdater->processCandidate(MediaWiki\Title\Title, bool)
#6	/srv/mediawiki/php-1.43.0-wmf.25/extensions/GrowthExperiments/maintenance/refreshLinkRecommendations.php(160)	GrowthExperiments\Maintenance\RefreshLinkRecommendations->processCandidate(MediaWiki\Title\Title, bool)
#7	/srv/mediawiki/php-1.43.0-wmf.25/maintenance/includes/MaintenanceRunner.php(703)	GrowthExperiments\Maintenance\RefreshLinkRecommendations->execute()
#8	/srv/mediawiki/php-1.43.0-wmf.25/maintenance/run.php(51)	MediaWiki\Maintenance\MaintenanceRunner->run()
#9	/srv/mediawiki/multiversion/MWScript.php(158)	require_once(string)
#10	{main}

Impact

Significant logspam (372 errors over the past 24 hours). Impact on the maintenance script is unknown.

Notes

Only seen on mwmaint2002 (presumably refreshLinkRecommendations only runs there).

Details

Request URL: N/A (maintenance script)

	Subject	Repo	Branch	Lines +/-
	Do not pass null to DataSender::sendWeightedTagsUpdate $tagWeights	mediawiki/extensions/CirrusSearch	wmf/1.43.0-wmf.27	+1 -2
	Do not pass null to DataSender::sendWeightedTagsUpdate $tagWeights	mediawiki/extensions/CirrusSearch	master	+1 -2

Customize query in gerrit

Related Objects

Mentioned In: T377479: PHP Warning: array_merge(): Expected parameter 1 to be an array, null given
Mentioned Here: T377479: PHP Warning: array_merge(): Expected parameter 1 to be an array, null given
rECIR24bffeae1592: Extract WeightedTagsUpdater and add EventBusWeightedTagsUpdater implementation

Event Timeline

Lucas_Werkmeister_WMDE created this task.Oct 8 2024, 1:10 PM

Restricted Application added projects: Growth-Team, Discovery-Search. · View Herald TranscriptOct 8 2024, 1:10 PM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Impact on the maintenance script is unknown.

Looks like the error gets caught and logged in link RecommendationUpdater::processCandidate(), which is why the maintenance script continues running. Beyond that, I still don’t know what the impact of this is (presumably the pages that experience this error will be missing some kind of update).

Etonkovidova moved this task from Inbox to Needs Discussion on the Growth-Team board.Oct 10 2024, 12:01 AM

In T376715#10213965, @Lucas_Werkmeister_WMDE wrote:

Impact on the maintenance script is unknown.

Looks like the error gets caught and logged in link RecommendationUpdater::processCandidate(), which is why the maintenance script continues running. Beyond that, I still don’t know what the impact of this is (presumably the pages that experience this error will be missing some kind of update).

I didn't notice any user impact either. The rate of error is high >500 (see logstash filtered for the error for last 24 hours ).

Etonkovidova added subscribers: Sgs, Urbanecm_WMF, Michael.Oct 10 2024, 12:36 AM

Krinkle moved this task from Untriaged to Oct 2024 on the Wikimedia-production-error board.Oct 10 2024, 3:33 PM

Gehel triaged this task as High priority.Oct 14 2024, 2:48 PM

Gehel moved this task from needs triage to Current work on the Discovery-Search board.

Gehel edited projects, added Discovery-Search (Current work); removed Discovery-Search.

The type hint is from 24bffeae1592c4625ddebc451940d773b99b070b - in the old code the tagWeights was nullable, but not in the new created function.

Checked in wmf.27 - the rate is still high - logstash link: 605 hits for 24 hour.

The error seems to started on Oct 03/2024 - https://logstash.wikimedia.org/goto/49820d81ecc17c1321226d9006fa5d85

• dcausse claimed this task.Oct 17 2024, 9:30 AM

• dcausse moved this task from Incoming to In Progress on the Discovery-Search (Current work) board.

Change #1081096 had a related patch set uploaded (by DCausse; author: DCausse):

[mediawiki/extensions/CirrusSearch@master] Do not pass null to DataSender::sendWeightedTagsUpdate $tagWeights

https://gerrit.wikimedia.org/r/1081096

gerritbot added a project: Patch-For-Review.Oct 17 2024, 10:00 AM

• dcausse moved this task from In Progress to Needs review on the Discovery-Search (Current work) board.Oct 17 2024, 10:02 AM

Michael mentioned this in T377479: PHP Warning: array_merge(): Expected parameter 1 to be an array, null given.Oct 17 2024, 5:44 PM

Change #1081096 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@master] Do not pass null to DataSender::sendWeightedTagsUpdate $tagWeights

https://gerrit.wikimedia.org/r/1081096

ReleaseTaggerBot added a project: MW-1.43-notes (1.43.0-wmf.28; 2024-10-22).Oct 17 2024, 8:00 PM

Maintenance_bot removed a project: Patch-For-Review.Oct 17 2024, 8:32 PM

Urbanecm_WMF moved this task from Needs Discussion to Triaged on the Growth-Team board.Oct 18 2024, 8:52 AM

In T376715#10213965, @Lucas_Werkmeister_WMDE wrote:

Impact on the maintenance script is unknown.

Looks like the error gets caught and logged in link RecommendationUpdater::processCandidate(), which is why the maintenance script continues running. Beyond that, I still don’t know what the impact of this is (presumably the pages that experience this error will be missing some kind of update).

User impact is that no new link-recommendation tasks are being added to the task pool for any wiki. The big wikis should have a large-enough pool for that not being much of an issue, the impact on wikis with a smaller pool might be more dire. That being said, the change to fix this should roll out with the next train and then hopefully we will see new link-recommendation tasks again.

Michael added projects: Add-Link, Growth-Structured-Tasks.Oct 18 2024, 12:16 PM

@dcausse Is this something that can be back-ported on Monday, or should it better ride the train normally?

In T376715#10241780, @Michael wrote:

@dcausse Is this something that can be back-ported on Monday, or should it better ride the train normally?

Sure, I'll backport the patches on Monday esp. this particular issue, T377479 should be fine because my understanding is that it only happens in the error handling code path when elastic is unresponsive and the job should be retried, but I'll backport it anyways.

Change #1081396 had a related patch set uploaded (by DCausse; author: DCausse):

[mediawiki/extensions/CirrusSearch@wmf/1.43.0-wmf.27] Do not pass null to DataSender::sendWeightedTagsUpdate $tagWeights

https://gerrit.wikimedia.org/r/1081396

gerritbot added a project: Patch-For-Review.Oct 18 2024, 12:59 PM

Change #1081396 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@wmf/1.43.0-wmf.27] Do not pass null to DataSender::sendWeightedTagsUpdate $tagWeights

https://gerrit.wikimedia.org/r/1081396

Mentioned in SAL (#wikimedia-operations) [2024-10-21T09:16:23Z] <dcausse@deploy2002> Started scap sync-world: Backport for [[gerrit:1081402|Fix phan issue with getCounter returning NullMetric|CounterMetric]], [[gerrit:1081396|Do not pass null to DataSender::sendWeightedTagsUpdate $tagWeights (T376715)]]

Mentioned in SAL (#wikimedia-operations) [2024-10-21T09:27:08Z] <dcausse@deploy2002> dcausse: Backport for [[gerrit:1081402|Fix phan issue with getCounter returning NullMetric|CounterMetric]], [[gerrit:1081396|Do not pass null to DataSender::sendWeightedTagsUpdate $tagWeights (T376715)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Maintenance_bot removed a project: Patch-For-Review.Oct 21 2024, 9:30 AM

Mentioned in SAL (#wikimedia-operations) [2024-10-21T09:39:50Z] <dcausse@deploy2002> Finished scap sync-world: Backport for [[gerrit:1081402|Fix phan issue with getCounter returning NullMetric|CounterMetric]], [[gerrit:1081396|Do not pass null to DataSender::sendWeightedTagsUpdate $tagWeights (T376715)]] (duration: 23m 26s)

• dcausse moved this task from Needs review to Needs Reporting on the Discovery-Search (Current work) board.Oct 21 2024, 9:53 AM

The fix is backported to 1.43.0-wmf.27, did not see any new instance of this error since.
Sadly I could not backport the fix T377479 yet since the corresponding patch on the master branch has not been +2ed yet.

ReleaseTaggerBot edited projects, added MW-1.43-notes (1.43.0-wmf.27; 2024-10-15); removed MW-1.43-notes (1.43.0-wmf.28; 2024-10-22).Oct 21 2024, 10:00 AM

Thank you! This backport enabled our maintenance script to add back task recommendations! 🙏

The last timestamp for the error is Oct 21, 2024 @ 09:38:19.928 (logstash link) - can this tak be closed?

Gehel closed this task as Resolved.Nov 1 2024, 10:06 AM

TypeError: Argument 3 passed to CirrusSearch\DataSender::sendWeightedTagsUpdate() must be of the type array, null given, called in /srv/mediawiki/php-1.43.0-wmf.25/extensions/CirrusSearch/includes/Job/ElasticaWrite.php on line Closed, ResolvedPublicPRODUCTION ERRORActions