All Wikivoyage requests serve Fatal Error: "invalid magic word 'noexternallanglinks' from line 315 of common/php-1.22wmf11/includes/MagicWord.php"
Closed, ResolvedPublic

Description

https://he.wikivoyage.org/wiki/Tarragona

Gives a MWException. Only occurs over HTTPS, and sometimes only occurs for certain languages.


Version: wmf-deployment
Severity: normal
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=52040

bzimport added a subscriber: Unknown Object (MLST).
bzimport set Reference to bz52038.
Parent5446 created this task.Via LegacyJul 25 2013, 4:54 PM
Krinkle added a comment.Via ConduitJul 25 2013, 4:55 PM

krinkle@fluorine:/a/mw-log$ ack-grep c6a790a7 exception.log -A 20
2013-07-25 16:51:01 mw1177 enwikivoyage: [c6a790a7] /wiki/Tarragona Exception from line 315 of /usr/local/apache/common-local/php-1.22wmf11/includes/MagicWord.php: Error: invalid magic word 'noexternallanglinks'
#0 /usr/local/apache/common-local/php-1.22wmf11/includes/MagicWord.php(238): MagicWord->load('noexternallangl...')
#1 /usr/local/apache/common-local/php-1.22wmf11/includes/parser/Parser.php(4904): MagicWord::get('noexternallangl...')
#2 /usr/local/apache/common-local/php-1.22wmf11/extensions/Wikibase/client/WikibaseClient.hooks.php(673): Parser->setFunctionHook('noexternallangl...', '\Wikibase\NoLan...', 1)
#3 [internal function]: Wikibase\ClientHooks::onParserFirstCallInit(Object(Parser))
#4 /usr/local/apache/common-local/php-1.22wmf11/includes/Hooks.php(199): call_user_func_array('\Wikibase\Clien...', Array)
#5 /usr/local/apache/common-local/php-1.22wmf11/includes/GlobalFunctions.php(3835): Hooks::run('ParserFirstCall...', Array)
#6 /usr/local/apache/common-local/php-1.22wmf11/includes/parser/Parser.php(270): wfRunHooks('ParserFirstCall...', Array)
#7 [internal function]: Parser->firstCallInit()
#8 /usr/local/apache/common-local/php-1.22wmf11/includes/StubObject.php(79): call_user_func_array(Array, Array)
#9 /usr/local/apache/common-local/php-1.22wmf11/includes/StubObject.php(99): StubObject->_call('firstCallInit', Array)
#10 /usr/local/apache/common-local/php-1.22wmf11/includes/cache/MessageCache.php(996): StubObject->__call('firstCallInit', Array)
#11 /usr/local/apache/common-local/php-1.22wmf11/includes/cache/MessageCache.php(996): StubObject->firstCallInit()
#12 /usr/local/apache/common-local/php-1.22wmf11/includes/cache/MessageCache.php(974): MessageCache->getParser()
#13 /usr/local/apache/common-local/php-1.22wmf11/includes/Message.php(696): MessageCache->transform('$1 ??? Travel g...', true, Object(Language), Object(Title))
#14 /usr/local/apache/common-local/php-1.22wmf11/includes/Message.php(496): Message->transformText('$1 ??? Travel g...')
#15 /usr/local/apache/common-local/php-1.22wmf11/includes/Message.php(553): Message->toString()
#16 /usr/local/apache/common-local/php-1.22wmf11/includes/OutputPage.php(846): Message->text()
#17 /usr/local/apache/common-local/php-1.22wmf11/includes/OutputPage.php(889): OutputPage->setHTMLTitle(Object(Message))
#18 /usr/local/apache/common-local/php-1.22wmf11/includes/Article.php(555): OutputPage->setPageTitle('Tarragona')
#19 /usr/local/apache/common-local/php-1.22wmf11/includes/actions/ViewAction.php(44): Article->view()

Krinkle added a comment.Via ConduitJul 25 2013, 5:01 PM

Not limited to HTTPS. That url just happened to be cached on HTTP still and not over HTTPS. Any url that is not cached (or invalidated by purge/edit) will result in fatal error.

e.g.

greg added a comment.Via ConduitJul 25 2013, 5:45 PM

We're back up, but I'm leaving this bug open for now to document what happened (for Aude and Reedy).

aude added a comment.Via ConduitJul 25 2013, 5:55 PM
  • the mergeMessageFiles script failed(?) due to wikibase bad initialization order
  • we tried to define the i18n / namespace /magic word / alias fields explicitly in extension-list, to try and allow merge message files to work. Somehow it does not work for magic words (?) and wikibase repo messages were missing.
  • would have loved to try rebuild and check it is okay for wmf12 only (test2 and test wikidata); instead it apparently also rebuilded and scapped the cache for wmf11 (wikidata + wikivoyage)
  • Wikibase Client (wikivoyage) was missing a magic word and that causes exception. (would be awesome if a missing magic word does not do that or fails more gracefully)
  • I removed the enforcement of initialization order and reverted change to extension-list, which fixes the issue for now.

PS, would be great to have a backup copy of the cache, so we have to wait for everything to rebuild again.

gerritbot added a comment.Via ConduitJul 25 2013, 8:16 PM

Change 75997 had a related patch set uploaded by Aude:
(bug 52038) catch exception when magic word not found and warn

https://gerrit.wikimedia.org/r/75997

Reedy added a comment.Via ConduitJul 26 2013, 12:32 AM
  • Bug 52040 has been marked as a duplicate of this bug. ***
Aklapper added a comment.Via ConduitJul 26 2013, 12:42 PM

(In reply to comment #3)

We're back up

I assume via reverting the change to extension-list?

Patch in comment 5 seems to require rework.

Reedy added a comment.Via ConduitJul 26 2013, 12:48 PM

Site is up and functioning. Nothing blocking of highest priority about this bug now

greg added a comment.Via ConduitJul 26 2013, 4:06 PM

Reedy: can you do a quick write up of this under https://wikitech.wikimedia.org/wiki/Incident_documentation ? Explaining what caused it and what to do/not to do next time would be great.

aude added a comment.Via ConduitJul 26 2013, 4:28 PM

@andre my patch would only help mitigate the situation in Wikibase.

our magic words are very important but imho not so essential to take the site down vs. having wikidata links appear where they shouldn't, etc.

Not sure logging the warning is sufficient action though, and the problem with missing magic words will occur again with magic words outside of wikibase. These sort of exceptions and outages have happened before and guarantee such incident *will* happen again with some other extension or magic word in core.

Open to other suggestions :)

Parent5446 added a comment.Via ConduitJul 26 2013, 4:29 PM

I mentioned this on the patch, but why exactly would magic words not be found?

gerritbot added a comment.Via ConduitJul 26 2013, 4:32 PM

Change 75997 abandoned by Aude:
(bug 52038) catch exception when magic word not found and warn

Reason:
nice try, but problems with missing magic words then appear in other places.

https://gerrit.wikimedia.org/r/75997

aude added a comment.Via ConduitJul 26 2013, 4:35 PM

note, when i try catching the exception, then errors occur elsewhere.

The better option would be to have a script or some sort of check that the mediawiki codebase is fine, not throwing such exceptions, etc. before scapping.

I'm not quite sure how we'd do that ....

aude added a comment.Via ConduitJul 26 2013, 4:36 PM

@Tyler it happens with localisation cache build is incomplete or something and then is scapped.

Add Comment