Page MenuHomePhabricator
Create Task
Maniphest T269315

Fatal error: Cannot pass parameter 1 by reference (Wikibase\DataModel\Entity)
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Description

Error

MediaWiki version: 1.36.0-wmf.20

message
Fatal error: Cannot pass parameter 1 by reference
exception.trace
#0 /srv/mediawiki/php-1.36.0-wmf.20/vendor/wikibase/data-model/src/Entity/EntityId.php(91): Wikibase\DataModel\Entity\EntityId::extractSerializationParts(string)
#1 /srv/mediawiki/php-1.36.0-wmf.20/vendor/wikibase/data-model/src/Entity/DispatchingEntityIdParser.php(45): Wikibase\DataModel\Entity\EntityId::splitSerialization(string)
#2 /srv/mediawiki/php-1.36.0-wmf.20/vendor/wikibase/data-model-serialization/src/Deserializers/SnakDeserializer.php(150): Wikibase\DataModel\Entity\DispatchingEntityIdParser->parse(string)
#3 /srv/mediawiki/php-1.36.0-wmf.20/vendor/wikibase/data-model-serialization/src/Deserializers/SnakDeserializer.php(116): Wikibase\DataModel\Deserializers\SnakDeserializer->deserializePropertyId(string)
#4 /srv/mediawiki/php-1.36.0-wmf.20/vendor/wikibase/data-model-serialization/src/Deserializers/SnakDeserializer.php(100): Wikibase\DataModel\Deserializers\SnakDeserializer->newValueSnak(array)
#5 /srv/mediawiki/php-1.36.0-wmf.20/vendor/wikibase/data-model-serialization/src/Deserializers/SnakDeserializer.php(82): Wikibase\DataModel\Deserializers\SnakDeserializer->getDeserialized(array)
#6 /srv/mediawiki/php-1.36.0-wmf.20/extensions/WikibaseQualityConstraints/src/ConstraintCheck/Helper/ConstraintParameterParser.php(222): Wikibase\DataModel\Deserializers\SnakDeserializer->deserialize(array)
#7 /srv/mediawiki/php-1.36.0-wmf.20/extensions/WikibaseQualityConstraints/src/ConstraintCheck/Helper/ConstraintParameterParser.php(257): WikibaseQuality\ConstraintReport\ConstraintCheck\Helper\ConstraintParameterParser->parsePropertyIdParameter(array, string)
#8 /srv/mediawiki/php-1.36.0-wmf.20/extensions/WikibaseQualityConstraints/src/ConstraintCheck/Checker/MandatoryQualifiersChecker.php(76): WikibaseQuality\ConstraintReport\ConstraintCheck\Helper\ConstraintParameterParser->parsePropertyParameter(array, string)
#9 /srv/mediawiki/php-1.36.0-wmf.20/extensions/WikibaseQualityConstraints/src/ConstraintCheck/DelegatingConstraintChecker.php(571): WikibaseQuality\ConstraintReport\ConstraintCheck\Checker\MandatoryQualifiersChecker->checkConstraint(WikibaseQuality\ConstraintReport\ConstraintCheck\Context\MainSnakContext, WikibaseQuality\ConstraintReport\Constraint)
#10 /srv/mediawiki/php-1.36.0-wmf.20/extensions/WikibaseQualityConstraints/src/ConstraintCheck/DelegatingConstraintChecker.php(465): WikibaseQuality\ConstraintReport\ConstraintCheck\DelegatingConstraintChecker->getCheckResultFor(WikibaseQuality\ConstraintReport\ConstraintCheck\Context\MainSnakContext, WikibaseQuality\ConstraintReport\Constraint)
#11 /srv/mediawiki/php-1.36.0-wmf.20/extensions/WikibaseQualityConstraints/src/ConstraintCheck/DelegatingConstraintChecker.php(374): WikibaseQuality\ConstraintReport\ConstraintCheck\DelegatingConstraintChecker->checkConstraintsForMainSnak(Wikibase\DataModel\Entity\Item, Wikibase\DataModel\Statement\Statement, NULL, array)
#12 /srv/mediawiki/php-1.36.0-wmf.20/extensions/WikibaseQualityConstraints/src/ConstraintCheck/DelegatingConstraintChecker.php(346): WikibaseQuality\ConstraintReport\ConstraintCheck\DelegatingConstraintChecker->checkStatement(Wikibase\DataModel\Entity\Item, Wikibase\DataModel\Statement\Statement, NULL, array)
#13 /srv/mediawiki/php-1.36.0-wmf.20/extensions/WikibaseQualityConstraints/src/ConstraintCheck/DelegatingConstraintChecker.php(154): WikibaseQuality\ConstraintReport\ConstraintCheck\DelegatingConstraintChecker->checkEveryStatement(Wikibase\DataModel\Entity\Item, NULL, array)
#14 /srv/mediawiki/php-1.36.0-wmf.20/extensions/WikibaseQualityConstraints/src/Api/CheckingResultsSource.php(53): WikibaseQuality\ConstraintReport\ConstraintCheck\DelegatingConstraintChecker->checkAgainstConstraintsOnEntityId(Wikibase\DataModel\Entity\ItemId, NULL, array, array)
#15 /srv/mediawiki/php-1.36.0-wmf.20/extensions/WikibaseQualityConstraints/src/Api/CachingResultsSource.php(234): WikibaseQuality\ConstraintReport\Api\CheckingResultsSource->getResults(array, array, NULL, array)
#16 /srv/mediawiki/php-1.36.0-wmf.20/extensions/WikibaseQualityConstraints/src/Api/CachingResultsSource.php(166): WikibaseQuality\ConstraintReport\Api\CachingResultsSource->getAndStoreResults(array, array, NULL, array)
#17 /srv/mediawiki/php-1.36.0-wmf.20/extensions/WikibaseQualityConstraints/src/Api/CheckConstraints.php(170): WikibaseQuality\ConstraintReport\Api\CachingResultsSource->getResults(array, array, NULL, array)
#18 /srv/mediawiki/php-1.36.0-wmf.20/includes/api/ApiMain.php(1607): WikibaseQuality\ConstraintReport\Api\CheckConstraints->execute()
#19 /srv/mediawiki/php-1.36.0-wmf.20/includes/api/ApiMain.php(587): ApiMain->executeAction()
#20 /srv/mediawiki/php-1.36.0-wmf.20/includes/api/ApiMain.php(558): ApiMain->executeActionWithErrorHandling()
#21 /srv/mediawiki/php-1.36.0-wmf.20/api.php(90): ApiMain->execute()
#22 /srv/mediawiki/php-1.36.0-wmf.20/api.php(45): wfApiMain()
#23 /srv/mediawiki/w/api.php(3): require(string)
#24 {main}

Impact

18 occurrences in the last 7 days (2020-11-26 to 2020-12-03), all on Wikidata.

Notes

Details

Request ID
X8gzrQpAEJoAA4gcXEsAAABQ
Request URL
https://www.wikidata.org/w/api.php?action=wbcheckconstraints&format=json&formatversion=2&uselang=en-ca&id=Q159&status=violation%7Cwarning%7Csuggestion%7Cbad-parameters

Related Objects

Event Timeline

Reedy created this task.Dec 3 2020, 1:27 AM
Restricted Application added a project: Wikidata. · View Herald TranscriptDec 3 2020, 1:27 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
• Krinkle renamed this task from Cannot pass parameter 1 by reference to Fatal error: Cannot pass parameter 1 by reference.Dec 3 2020, 3:25 AM
• Krinkle renamed this task from Fatal error: Cannot pass parameter 1 by reference to Fatal error: Cannot pass parameter 1 by reference (Wikibase\DataModel\Entity).
• Krinkle triaged this task as Unbreak Now! priority.
• Krinkle updated the task description. (Show Details)
• Krinkle edited Stack Trace. (Show Details)
• Krinkle added a project: Wikidata-Campsite.
• Krinkle moved this task from Untriaged to Dec 2020 on the Wikimedia-production-error board.
RhinosF1 subscribed.Dec 3 2020, 4:28 AM

Should this block the train?

• mmodell subscribed.EditedDec 3 2020, 6:31 AM
In T269315#6665738, @RhinosF1 wrote:

Should this block the train?

Given that it's unbreak now, then it probably should be a blocker.

I've added the parent task based on this assumption. If this is wrong, others who know better are free to remove the blocker

• mmodell added a parent task: T263186: 1.36.0-wmf.20 deployment blockers.Dec 3 2020, 6:32 AM
Lucas_Werkmeister_WMDE edited projects, added Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)); removed Wikidata-Campsite.Dec 3 2020, 12:21 PM
Ladsgroup subscribed.Dec 3 2020, 12:22 PM
Lucas_Werkmeister_WMDE claimed this task.Dec 3 2020, 12:25 PM
Lucas_Werkmeister_WMDE updated the task description. (Show Details)
Lucas_Werkmeister_WMDE moved this task from To Do (prioritised from top to bottom) to Doing on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.
Lucas_Werkmeister_WMDE added a comment.Dec 3 2020, 12:38 PM

Whatever this error is, it’s not new in wmf.20 – Logstash has occurrences from wmf.18 and wmf.16 as well. So I’m not convinced it should block the train.

Beyond that, I’m mystified at this so far. The offending line seems to be this:

$parts = explode( ':', $serialization );

But why would PHP expect the first explode() argument to be a reference? This feels like the kind of error that could be caused by opcache corruption, but the distribution of the error in Logstash doesn’t match the characteristics of diagnosing opcache corruption at all (it’s not limited to one server, nor does it happen in bursts), so it’s probably not that.

Ladsgroup removed a parent task: T263186: 1.36.0-wmf.20 deployment blockers.Dec 3 2020, 1:13 PM
Lucas_Werkmeister_WMDE added a comment.Dec 3 2020, 1:14 PM

It’s also curious that “Cannot pass parameter 1 by reference” is not the standard error message for calling a reference-parameter function with a non-referenceable argument, as far as I can tell. So far I’ve only managed to make PHP print that message with a lambda:

lucaswerkmeister-wmde@mw1281:~$ php --version
PHP 7.2.31-1+0~20200514.41+debian9~1.gbpe2a56b+wmf1+icu63 (cli) (built: Oct 12 2020 09:43:06) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies
    with Zend OPcache v7.2.31-1+0~20200514.41+debian9~1.gbpe2a56b+wmf1+icu63, Copyright (c) 1999-2018, by Zend Technologies
lucaswerkmeister-wmde@mw1281:~$ php -a
Interactive mode enabled

php > explode( ':', 'a:b:c' );
php > function f( &$ref ) {}
php > f( ':' );
Fatal error: Only variables can be passed by reference in php shell code on line 1
php > ('f')( ':' );
Fatal error: Only variables can be passed by reference in php shell code on line 1
php > $l = function ( &$ref ) {};
php > $l( ':' );
Warning: Uncaught Error: Cannot pass parameter 1 by reference in php shell code:1
Stack trace:
#0 {main}
  thrown in php shell code on line 1

I’m not sure if that’s significant.

Ladsgroup lowered the priority of this task from Unbreak Now! to High.Dec 3 2020, 1:14 PM

I think this should be addressed ASAP but doesn't need to be a train blocker or UBN.

Maintenance_bot moved this task from incoming to in progress on the Wikidata board.Dec 3 2020, 1:15 PM
Lucas_Werkmeister_WMDE removed Lucas_Werkmeister_WMDE as the assignee of this task.Dec 3 2020, 1:34 PM
Lucas_Werkmeister_WMDE moved this task from Doing to To Do (prioritised from top to bottom) on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.
Lucas_Werkmeister_WMDE subscribed.

I’m unassigning myself from this, since I have no idea how to continue working on it. As far as I can tell, PHP behaves nonsensically on this one particular line (EntityId.php, line 108), but with different stack traces leading to it, across multiple servers, but only very rarely (we probably parse entity IDs hundreds of times on most Wikidata page views).

Ladsgroup added a comment.Dec 3 2020, 2:06 PM

From what I'm seeing in the logs (here), the errors are either for wbcheckconstraints and only for checks that are so heavy that they all time out (on Q30, etc.) or for some random item and unreproducable. My initial thinking for the WBQS ones is that the timeout handler doesn't kill the process properly (and it shouldn't reach the point of timeout anyway but hopefully shellbox addresses that problem by migrating our regex checks to it.) and for the items, it's opcache corruption (specially since it's not reproducible). HTH.

hashar subscribed.Dec 3 2020, 2:37 PM

Thank you Lucas and Amir for the investigation.

Addshore subscribed.Dec 4 2020, 9:05 AM
Ladsgroup edited projects, added Wikidata-Campsite; removed Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)).Dec 4 2020, 1:55 PM
Umherirrender subscribed.Mar 2 2021, 10:23 PM

As opcache corruption is mention in on of the comments should this linked or mention on T245183: PHP7 corruption reports in 2020-2022 (Call on wrong object, etc.) for further debugging?

Restricted Application added a project: [DEPRECATED] wdwb-tech. · View Herald TranscriptMar 2 2021, 10:23 PM
Addshore moved this task from Inbox to External Realm on the [DEPRECATED] wdwb-tech board.Mar 5 2021, 11:51 AM
Lucas_Werkmeister_WMDE added a comment.Mar 5 2021, 11:55 AM

This seems to have mostly resolved itself – I found only one occurrence in the last 30 days now (on commonswiki, reqId YD7BKVeq10LaICkvzpGn1AAAAFE).

Daimona subscribed.Mar 8 2021, 4:07 PM

Even if it isn't an opcache corruption, it might still be a PHP bug. In principle it might make sense to report it upstream, but since we're using PHP 7.2 (no longer supported), I think we should first see if this still happens after upgrading to a supported version; that would be PHP 7.4 at the time of writing and until 28 Nov 2021. And we'd also need a smaller test case. As such, IMHO the best option is to just wait (at least until WMF servers run a PHP version that still has active support), see if it fixes itself, and then decide.

Addshore added a comment.Mar 9 2021, 11:46 PM

If we think its mostly resolved itself, and have no actions right now other than to wait, shall we go ahead and close this task?

Addshore closed this task as Resolved.Apr 22 2021, 4:03 PM
Addshore claimed this task.
Restricted Application added a project: User-Addshore. · View Herald TranscriptApr 22 2021, 4:03 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits