Page MenuHomePhabricator

Fatals and notices in FlowFixLinks.php
Closed, ResolvedPublic

Description

This was with:

Flow: 1650e76b25980112e9e5097c721d77e672501ae1 (from GuidedTour commit), which is on top of rEFLW55b5aa794fb4: Merge "Use the VirtualRESTService to configure Flow".

var_export( $wgFlowMigrateReferenceWiki );
false

I ran vagrant git-update (which includes update.php), and got the following errors.

Not sure if this would be an issue in production, or it's just because of the constant back and forth locally developing and testing this reference stuff.

wiki:  Removed 6 links to special pages.
wiki:  Rebuilt links for 300 workflows...
wiki:  Rebuilt links for 300 workflows...
wiki:  Rebuilt links for 300 workflows...
PHP Notice:  Undefined index: wiki:flow_ref:wiki:by-source:v3:0:Page_with_%25_2:wiki:4.7 in /vagrant/mediawiki/extensions/Flow/includes/Data/Compactor/FeatureCompactor.php on line 69
PHP Stack trace:
PHP   1. {main}() /var/www/w/MWScript.php:0
PHP   2. require_once() /var/www/w/MWScript.php:89
PHP   3. require_once() /vagrant/mediawiki/maintenance/update.php:214
PHP   4. UpdateMediaWiki->execute() /vagrant/mediawiki/maintenance/doMaintenance.php:103
PHP   5. LoggedUpdateMaintenance->execute() /vagrant/mediawiki/maintenance/update.php:181
PHP   6. FlowFixLinks->doDBUpdates() /vagrant/mediawiki/maintenance/Maintenance.php:1318
PHP   7. FlowFixLinks->rebuildCoreTables() /vagrant/mediawiki/extensions/Flow/maintenance/FlowFixLinks.php:38
PHP   8. Flow\LinksTableUpdater->doUpdate() /vagrant/mediawiki/extensions/Flow/maintenance/FlowFixLinks.php:83
PHP   9. AbstractContent->getSecondaryDataUpdates() /vagrant/mediawiki/extensions/Flow/includes/LinksTableUpdater.php:36
PHP  10. Flow\Content\BoardContent->getParserOutput() /vagrant/mediawiki/includes/content/AbstractContent.php:230
PHP  11. Flow\LinksTableUpdater->mutateParserOutput() /vagrant/mediawiki/extensions/Flow/includes/Content/BoardContent.php:192
PHP  12. Flow\LinksTableUpdater->getReferencesForTitle() /vagrant/mediawiki/extensions/Flow/includes/LinksTableUpdater.php:49
PHP  13. Flow\Data\ManagerGroup->find() /vagrant/mediawiki/extensions/Flow/includes/LinksTableUpdater.php:126
PHP  14. Flow\Data\ManagerGroup->call() /vagrant/mediawiki/extensions/Flow/includes/Data/ManagerGroup.php:141
PHP  15. call_user_func_array() /vagrant/mediawiki/extensions/Flow/includes/Data/ManagerGroup.php:129
PHP  16. Flow\Data\ObjectLocator->find() /vagrant/mediawiki/extensions/Flow/includes/Data/ManagerGroup.php:129
PHP  17. Flow\Data\ObjectLocator->findMulti() /vagrant/mediawiki/extensions/Flow/includes/Data/ObjectLocator.php:55
PHP  18. Flow\Data\Index\FeatureIndex->findMulti() /vagrant/mediawiki/extensions/Flow/includes/Data/ObjectLocator.php:80
wiki:  
PHP  19. Flow\Data\Compactor\FeatureCompactor->expandCacheResult() /vagrant/mediawiki/extensions/Flow/includes/Data/Index/FeatureIndex.php:484
wiki:  Notice: Undefined index: wiki:flow_ref:wiki:by-source:v3:0:Page_with_%25_2:wiki:4.7 in /vagrant/mediawiki/extensions/Flow/includes/Data/Compactor/FeatureCompactor.php on line 69
wiki:  
wiki:  Call Stack:
wiki:      0.0006     232368   1. {main}() /var/www/w/MWScript.php:0
wiki:      0.0059     326776   2. require_once('/vagrant/mediawiki/maintenance/update.php') /var/www/w/MWScript.php:89
wiki:      0.0093     664696   3. require_once('/vagrant/mediawiki/maintenance/doMaintenance.php') /vagrant/mediawiki/maintenance/update.php:214
wiki:      0.2565   14170920   4. UpdateMediaWiki->execute() /vagrant/mediawiki/maintenance/doMaintenance.php:103
wiki:      3.0572   19163752   5. LoggedUpdateMaintenance->execute() /vagrant/mediawiki/maintenance/update.php:181
wiki:      3.0581   19163928   6. FlowFixLinks->doDBUpdates() /vagrant/mediawiki/maintenance/Maintenance.php:1318
wiki:      3.1518   21035656   7. FlowFixLinks->rebuildCoreTables() /vagrant/mediawiki/extensions/Flow/maintenance/FlowFixLinks.php:38
PHP Warning:  Invalid argument supplied for foreach() in /vagrant/mediawiki/extensions/Flow/includes/Data/Compactor/FeatureCompactor.php on line 70
PHP Stack trace:
PHP   1. {main}() /var/www/w/MWScript.php:0
PHP   2. require_once() /var/www/w/MWScript.php:89
PHP   3. require_once() /vagrant/mediawiki/maintenance/update.php:214
PHP   4. UpdateMediaWiki->execute() /vagrant/mediawiki/maintenance/doMaintenance.php:103
PHP   5. LoggedUpdateMaintenance->execute() /vagrant/mediawiki/maintenance/update.php:181
PHP   6. FlowFixLinks->doDBUpdates() /vagrant/mediawiki/maintenance/Maintenance.php:1318
PHP   7. FlowFixLinks->rebuildCoreTables() /vagrant/mediawiki/extensions/Flow/maintenance/FlowFixLinks.php:38
PHP   8. Flow\LinksTableUpdater->doUpdate() /vagrant/mediawiki/extensions/Flow/maintenance/FlowFixLinks.php:83
PHP   9. AbstractContent->getSecondaryDataUpdates() /vagrant/mediawiki/extensions/Flow/includes/LinksTableUpdater.php:36
PHP  10. Flow\Content\BoardContent->getParserOutput() /vagrant/mediawiki/includes/content/AbstractContent.php:230
PHP  11. Flow\LinksTableUpdater->mutateParserOutput() /vagrant/mediawiki/extensions/Flow/includes/Content/BoardContent.php:192
PHP  12. Flow\LinksTableUpdater->getReferencesForTitle() /vagrant/mediawiki/extensions/Flow/includes/LinksTableUpdater.php:49
PHP  13. Flow\Data\ManagerGroup->find() /vagrant/mediawiki/extensions/Flow/includes/LinksTableUpdater.php:126
PHP  14. Flow\Data\ManagerGroup->call() /vagrant/mediawiki/extensions/Flow/includes/Data/ManagerGroup.php:141
wiki:     54.9285   49811960   8. Flow\LinksTableUpdater->doUpdate() /vagrant/mediawiki/extensions/Flow/maintenance/FlowFixLinks.php:83
wiki:     54.9365   49820792   9. AbstractContent->getSecondaryDataUpdates() /vagrant/mediawiki/extensions/Flow/includes/LinksTableUpdater.php:36
wiki:     54.9365   49821080  10. Flow\Content\BoardContent->getParserOutput() /vagrant/mediawiki/includes/content/AbstractContent.php:230
wiki:     54.9402   49829168  11. Flow\LinksTableUpdater->mutateParserOutput() /vagrant/mediawiki/extensions/Flow/includes/Content/BoardContent.php:192
wiki:     54.9402   49829216  12. Flow\LinksTableUpdater->getReferencesForTitle() /vagrant/mediawiki/extensions/Flow/includes/LinksTableUpdater.php:49
wiki:     54.9402   49830248  13. Flow\Data\ManagerGroup->find() /vagrant/mediawiki/extensions/Flow/includes/LinksTableUpdater.php:126
wiki:     54.9402   49831408  14. Flow\Data\ManagerGroup->call() /vagrant/mediawiki/extensions/Flow/includes/Data/ManagerGroup.php:141
wiki:     54.9403   49832296  15. call_user_func_array() /vagrant/mediawiki/extensions/Flow/includes/Data/ManagerGroup.php:129
wiki:     54.9403   49832632  16. Flow\Data\ObjectLocator->find() /vagrant/mediawiki/extensions/Flow/includes/Data/ManagerGroup.php:129
PHP  15. call_user_func_array() /vagrant/mediawiki/extensions/Flow/includes/Data/ManagerGroup.php:129
PHP  16. Flow\Data\ObjectLocator->find() /vagrant/mediawiki/extensions/Flow/includes/Data/ManagerGroup.php:129
PHP  17. Flow\Data\ObjectLocator->findMulti() /vagrant/mediawiki/extensions/Flow/includes/Data/ObjectLocator.php:55
PHP  18. Flow\Data\Index\FeatureIndex->findMulti() /vagrant/mediawiki/extensions/Flow/includes/Data/ObjectLocator.php:80
PHP  19. Flow\Data\Compactor\FeatureCompactor->expandCacheResult() /vagrant/mediawiki/extensions/Flow/includes/Data/Index/FeatureIndex.php:484
PHP Fatal error:  Unsupported operand types in /vagrant/mediawiki/extensions/Flow/includes/Data/Compactor/FeatureCompactor.php on line 81
PHP Stack trace:
PHP   1. {main}() /var/www/w/MWScript.php:0
PHP   2. require_once() /var/www/w/MWScript.php:89
PHP   3. require_once() /vagrant/mediawiki/maintenance/update.php:214
PHP   4. UpdateMediaWiki->execute() /vagrant/mediawiki/maintenance/doMaintenance.php:103
PHP   5. LoggedUpdateMaintenance->execute() /vagrant/mediawiki/maintenance/update.php:181
PHP   6. FlowFixLinks->doDBUpdates() /vagrant/mediawiki/maintenance/Maintenance.php:1318
PHP   7. FlowFixLinks->rebuildCoreTables() /vagrant/mediawiki/extensions/Flow/maintenance/FlowFixLinks.php:38
PHP   8. Flow\LinksTableUpdater->doUpdate() /vagrant/mediawiki/extensions/Flow/maintenance/FlowFixLinks.php:83
PHP   9. AbstractContent->getSecondaryDataUpdates() /vagrant/mediawiki/extensions/Flow/includes/LinksTableUpdater.php:36
PHP  10. Flow\Content\BoardContent->getParserOutput() /vagrant/mediawiki/includes/content/AbstractContent.php:230
PHP  11. Flow\LinksTableUpdater->mutateParserOutput() /vagrant/mediawiki/extensions/Flow/includes/Content/BoardContent.php:192
PHP  12. Flow\LinksTableUpdater->getReferencesForTitle() /vagrant/mediawiki/extensions/Flow/includes/LinksTableUpdater.php:49
PHP  13. Flow\Data\ManagerGroup->find() /vagrant/mediawiki/extensions/Flow/includes/LinksTableUpdater.php:126
PHP  14. Flow\Data\ManagerGroup->call() /vagrant/mediawiki/extensions/Flow/includes/Data/ManagerGroup.php:141
PHP  15. call_user_func_array() /vagrant/mediawiki/extensions/Flow/includes/Data/ManagerGroup.php:129
PHP  16. Flow\Data\ObjectLocator->find() /vagrant/mediawiki/extensions/Flow/includes/Data/ManagerGroup.php:129
PHP  17. Flow\Data\ObjectLocator->findMulti() /vagrant/mediawiki/extensions/Flow/includes/Data/ObjectLocator.php:55
PHP  18. Flow\Data\Index\FeatureIndex->findMulti() /vagrant/mediawiki/extensions/Flow/includes/Data/ObjectLocator.php:80
PHP  19. Flow\Data\Compactor\FeatureCompactor->expandCacheResult() /vagrant/mediawiki/extensions/Flow/includes/Data/Index/FeatureIndex.php:484
wiki:     54.9403   49833072  17. Flow\Data\ObjectLocator->findMulti() /vagrant/mediawiki/extensions/Flow/includes/Data/ObjectLocator.php:55
wiki:     54.9403   49834104  18. Flow\Data\Index\FeatureIndex->findMulti() /vagrant/mediawiki/extensions/Flow/includes/Data/ObjectLocator.php:80
wiki:     54.9455   49858560  19. Flow\Data\Compactor\FeatureCompactor->expandCacheResult() /vagrant/mediawiki/extensions/Flow/includes/Data/Index/FeatureIndex.php:484
wiki:  
wiki:  
wiki:  Warning: Invalid argument supplied for foreach() in /vagrant/mediawiki/extensions/Flow/includes/Data/Compactor/FeatureCompactor.php on line 70
wiki:  
wiki:  Call Stack:
wiki:      0.0006     232368   1. {main}() /var/www/w/MWScript.php:0
wiki:      0.0059     326776   2. require_once('/vagrant/mediawiki/maintenance/update.php') /var/www/w/MWScript.php:89
wiki:      0.0093     664696   3. require_once('/vagrant/mediawiki/maintenance/doMaintenance.php') /vagrant/mediawiki/maintenance/update.php:214
wiki:      0.2565   14170920   4. UpdateMediaWiki->execute() /vagrant/mediawiki/maintenance/doMaintenance.php:103
wiki:      3.0572   19163752   5. LoggedUpdateMaintenance->execute() /vagrant/mediawiki/maintenance/update.php:181
wiki:      3.0581   19163928   6. FlowFixLinks->doDBUpdates() /vagrant/mediawiki/maintenance/Maintenance.php:1318
wiki:      3.1518   21035656   7. FlowFixLinks->rebuildCoreTables() /vagrant/mediawiki/extensions/Flow/maintenance/FlowFixLinks.php:38
wiki:     54.9285   49811960   8. Flow\LinksTableUpdater->doUpdate() /vagrant/mediawiki/extensions/Flow/maintenance/FlowFixLinks.php:83
wiki:     54.9365   49820792   9. AbstractContent->getSecondaryDataUpdates() /vagrant/mediawiki/extensions/Flow/includes/LinksTableUpdater.php:36
wiki:     54.9365   49821080  10. Flow\Content\BoardContent->getParserOutput() /vagrant/mediawiki/includes/content/AbstractContent.php:230
wiki:     54.9402   49829168  11. Flow\LinksTableUpdater->mutateParserOutput() /vagrant/mediawiki/extensions/Flow/includes/Content/BoardContent.php:192
wiki:     54.9402   49829216  12. Flow\LinksTableUpdater->getReferencesForTitle() /vagrant/mediawiki/extensions/Flow/includes/LinksTableUpdater.php:49
wiki:     54.9402   49830248  13. Flow\Data\ManagerGroup->find() /vagrant/mediawiki/extensions/Flow/includes/LinksTableUpdater.php:126
wiki:     54.9402   49831408  14. Flow\Data\ManagerGroup->call() /vagrant/mediawiki/extensions/Flow/includes/Data/ManagerGroup.php:141
wiki:     54.9403   49832296  15. call_user_func_array() /vagrant/mediawiki/extensions/Flow/includes/Data/ManagerGroup.php:129
wiki:     54.9403   49832632  16. Flow\Data\ObjectLocator->find() /vagrant/mediawiki/extensions/Flow/includes/Data/ManagerGroup.php:129
wiki:     54.9403   49833072  17. Flow\Data\ObjectLocator->findMulti() /vagrant/mediawiki/extensions/Flow/includes/Data/ObjectLocator.php:55
wiki:     54.9403   49834104  18. Flow\Data\Index\FeatureIndex->findMulti() /vagrant/mediawiki/extensions/Flow/includes/Data/ObjectLocator.php:80
wiki:     54.9455   49858560  19. Flow\Data\Compactor\FeatureCompactor->expandCacheResult() /vagrant/mediawiki/extensions/Flow/includes/Data/Index/FeatureIndex.php:484
wiki:  
wiki:  
wiki:  Fatal error: Unsupported operand types in /vagrant/mediawiki/extensions/Flow/includes/Data/Compactor/FeatureCompactor.php on line 81
wiki:  
wiki:  Call Stack:
wiki:      0.0006     232368   1. {main}() /var/www/w/MWScript.php:0
wiki:      0.0059     326776   2. require_once('/vagrant/mediawiki/maintenance/update.php') /var/www/w/MWScript.php:89
wiki:      0.0093     664696   3. require_once('/vagrant/mediawiki/maintenance/doMaintenance.php') /vagrant/mediawiki/maintenance/update.php:214
wiki:      0.2565   14170920   4. UpdateMediaWiki->execute() /vagrant/mediawiki/maintenance/doMaintenance.php:103
wiki:      3.0572   19163752   5. LoggedUpdateMaintenance->execute() /vagrant/mediawiki/maintenance/update.php:181
wiki:      3.0581   19163928   6. FlowFixLinks->doDBUpdates() /vagrant/mediawiki/maintenance/Maintenance.php:1318
wiki:      3.1518   21035656   7. FlowFixLinks->rebuildCoreTables() /vagrant/mediawiki/extensions/Flow/maintenance/FlowFixLinks.php:38
wiki:     54.9285   49811960   8. Flow\LinksTableUpdater->doUpdate() /vagrant/mediawiki/extensions/Flow/maintenance/FlowFixLinks.php:83
wiki:     54.9365   49820792   9. AbstractContent->getSecondaryDataUpdates() /vagrant/mediawiki/extensions/Flow/includes/LinksTableUpdater.php:36
wiki:     54.9365   49821080  10. Flow\Content\BoardContent->getParserOutput() /vagrant/mediawiki/includes/content/AbstractContent.php:230
wiki:     54.9402   49829168  11. Flow\LinksTableUpdater->mutateParserOutput() /vagrant/mediawiki/extensions/Flow/includes/Content/BoardContent.php:192
wiki:     54.9402   49829216  12. Flow\LinksTableUpdater->getReferencesForTitle() /vagrant/mediawiki/extensions/Flow/includes/LinksTableUpdater.php:49
wiki:     54.9402   49830248  13. Flow\Data\ManagerGroup->find() /vagrant/mediawiki/extensions/Flow/includes/LinksTableUpdater.php:126
wiki:     54.9402   49831408  14. Flow\Data\ManagerGroup->call() /vagrant/mediawiki/extensions/Flow/includes/Data/ManagerGroup.php:141
wiki:     54.9403   49832296  15. call_user_func_array() /vagrant/mediawiki/extensions/Flow/includes/Data/ManagerGroup.php:129
wiki:     54.9403   49832632  16. Flow\Data\ObjectLocator->find() /vagrant/mediawiki/extensions/Flow/includes/Data/ManagerGroup.php:129
wiki:     54.9403   49833072  17. Flow\Data\ObjectLocator->findMulti() /vagrant/mediawiki/extensions/Flow/includes/Data/ObjectLocator.php:55
wiki:     54.9403   49834104  18. Flow\Data\Index\FeatureIndex->findMulti() /vagrant/mediawiki/extensions/Flow/includes/Data/ObjectLocator.php:80
wiki:     54.9455   49858560  19. Flow\Data\Compactor\FeatureCompactor->expandCacheResult() /vagrant/mediawiki/extensions/Flow/includes/Data/Index/FeatureIndex.php:484
wiki:

Event Timeline

Mattflaschen-WMF renamed this task from Fatals in FlowFixLinks.php to Fatals and notices in FlowFixLinks.php.Sep 2 2015, 3:10 AM
Mattflaschen-WMF updated the task description. (Show Details)
Mattflaschen-WMF set Security to None.

I'm going to guess/hope it may be related to testing the reference stuff & having some invalid data in cache somewhere.

However, I can't figure out any way to trigger this.
The first notice (indefined index) should never be triggered AFAIK. FeatureIndex::findMulti passes $fromCache & $keyToQuery to FeatureCompactor::expandCacheResult, which fails to locate one of $fromCache's keys in $keyToQuery.
I'm obviously overlooking something since this notice did occur, but I can not figure out how $fromCache can have different data than $keyToQuery - both originate from the same $fromCache result early on in FeatureIndex::findMulti and none of the manipulations that happen in that function seem invalid.

Best I can think of right now is that somehow $fromCache = $this->cache->getMulti( array_unique( $cacheKeys ) ); (FeatureIndex.php L389) returns the (in-memory or remote) cached data with some unexpected key (= not the cache key, not in $cacheKeys), so the array_intersect on the next line starts with invalid data.

Turns out MemcachedBagOStuffs are unreliable.
It'll rawurlencode keys when fetching data from cache, but won't decode them when responding.
In this specific case: the original key must've been wiki:flow_ref:wiki:by-source:v3:0:Page_with_%_2:wiki:4.7 before MemcachedBagOStuff::encodeKey turned it into wiki:flow_ref:wiki:by-source:v3:0:Page_with_%25_2:wiki:4.7. It wasn't decoded before responding in key => value format, so the key wasn't recognized.

Change 235458 had a related patch set uploaded (by Matthias Mullie):
Decode Memcached keys before returning getMulti result

https://gerrit.wikimedia.org/r/235458

I'm not finding an open task, but i've seen ori mention a couple times in the last week that memcache is having issues in prod. randomly guessing, but might be related?

Change 235458 merged by Matthias Mullie:
Decode Memcached keys before returning getMulti result

https://gerrit.wikimedia.org/r/235458

Backport if necessary depending when we run the script.

Checked /a/mw-log - no related errors to FeatureCompactor.php.

DannyH closed this task as Resolved.Sep 17 2015, 4:49 PM