Mentioned in SAL (#wikimedia-operations) [2017-03-15T20:30:53Z] <twentyafterfour> T160569 blocks the train until I can figure out what is causing it. The frequency is low so I haven't reverted to wmf.15, group 1 remains on wmf.16 refs T158997

This happens as part of interwiki-search based on language identification, we ask $wgConf for the all the config we know about that starts with wgCirrusSearch for the other wiki. I suppose the temp fix would be to just set this in CirrusSearch-common.php? Longer term we need to move away from wgConf to an expanded version of the cirrus-config-dump api call

Change 342907 had a related patch set uploaded (by EBernhardson):
[operations/mediawiki-config] Keep variable around that was removed in wmf.16

https://gerrit.wikimedia.org/r/342907

Ah so the problem is mismatched versions across wikis due to multiversion. Maybe just catch the exception?

the problem with catching the exception is it happens in a different process, wgConf implements a complete hack that shells out to get this configuration from the other wiki (part of why we need to get it moved over to an api call instead...)

Change 342907 merged by jenkins-bot:
[operations/mediawiki-config] Keep variable around that was removed in wmf.16

https://gerrit.wikimedia.org/r/342907

Change 342914 had a related patch set uploaded (by 20after4):
[operations/mediawiki-config] Temp fix for wmf.16: define wgCirrusSearchCacheWarmers

https://gerrit.wikimedia.org/r/342914

Now I'm getting the same error for $wgCirrusSearchCacheWarmers so I submitted https://gerrit.wikimedia.org/r/342914

Change 342914 merged by jenkins-bot:
[operations/mediawiki-config] Temp fix for wmf.16: define wgCirrusSearchCacheWarmers

https://gerrit.wikimedia.org/r/342914

Change 342944 had a related patch set uploaded (by 20after4):
[operations/mediawiki-config] Undefined global wgCirrusSearchCompletionGeoContextProfiles

https://gerrit.wikimedia.org/r/342944

Change 342944 merged by jenkins-bot:
[operations/mediawiki-config] Undefined global wgCirrusSearchCompletionGeoContextProfiles

https://gerrit.wikimedia.org/r/342944

Mentioned in SAL (#wikimedia-operations) [2017-03-15T21:54:50Z] <twentyafterfour@tin> Synchronized wmf-config/CirrusSearch-common.php: Deploy I67d71217d16cd75a2646cf658cead2c5fd740cd9 refs T160569 and T158997 (duration: 00m 42s)

not sure if this is related or separate issue:

Fatal error: Couldn't find constant Elastica\Search::OPTION_SEARCH_TYPE_COUNT in /srv/mediawiki/php-1.29.0-wmf.16/extensions/ApiFeatureUsage/ApiFeatureUsageQueryEngineElastica.php on line 88

submitted separate task for T160578: Fatal error: Couldn't find constant Elastica\Search::OPTION_SEARCH_TYPE_COUNT in ApiFeatureUsageQueryEngineElastica.php on line 88

But now I'm getting this same error for wgCirrusSearchCompletionSuggesterGeoContext

This feels like I'm playing a game of whack-a-mole. @EBernhardson can you point me at a list of removed globals? I'd rather not submit+deploy 10 more patches today.

lemme poke at the code and see

I think this is the full set:

$wgCirrusSearchMainPageCacheWarmer = false;
$wgCirrusSearchCacheWarmers = [];
$wgCirrusSearchCompletionSuggesterGeoContext = ['build' => false];
$wgCirrusSearchCompletionGeoContextSettings = [];
$wgCirrusSearchCompletionGeoContextProfiles = [];

Change 342948 had a related patch set uploaded (by EBernhardson):
[operations/mediawiki-config] Undefined globals from CirrusSearch

https://gerrit.wikimedia.org/r/342948

Change 342948 merged by jenkins-bot:
[operations/mediawiki-config] Undefined globals from CirrusSearch

https://gerrit.wikimedia.org/r/342948

Mentioned in SAL (#wikimedia-operations) [2017-03-15T22:51:07Z] <twentyafterfour@tin> Synchronized wmf-config/CirrusSearch-common.php: Deploy I4980daf73d5a98afedfae4a254a88ea078912296 refs T160569 and T158997 (duration: 00m 42s)

After this deploy the interwiki search from wmf.15 to wmf.16 works as expected and doesn't log errors: https://en.wikipedia.org/w/index.php?search=ערך+מומלץ&title=Special:Search

Thanks @EBernhardson!