This is borderline UBN.

The traceback doesn't involve abusefilter but it's still possible. I'm also worried it might be flagged revs (in some weird combo config) but fawiki is really similar to enwiki so I think it's not that.

The stacktrace shows that the calls are coming from the api

With https://gerrit.wikimedia.org/r/c/mediawiki/core/+/664935 I have added a safe guard for this situation some weeks ago, but that seems not to work here

See https://meta.wikimedia.org/w/index.php?diff=21326013#MediaWiki\Revision\RevisionAccessException, reported there as well.

In T279832#6988779, @Ladsgroup wrote:

The traceback doesn't involve abusefilter but it's still possible. I'm also worried it might be flagged revs (in some weird combo config) but fawiki is really similar to enwiki so I think it's not that.

I just tried to move https://ms.wikipedia.org/wiki/Bones to Bones (siri TV), and that caused the error as well. It's definitely not FlaggedRevs related, since that extension is not even installed in mswiki.

Can someone with access to logstash provide an update on which wikis are affected and since when? We know fawiki, dewiki and mswiki are affected but who else? Also, as far as fawiki, we know this was reported as early as April 9th (one day after 1.36.0-wmf.38 was deployed); do we have logs from before April 8th?

Of note, this is not impacting every move (e.g. [https://fa.wikipedia.org/wiki/%D9%88%DB%8C%DA%98%D9%87:%D8%B3%DB%8C%D8%A7%D9%87%D9%87%E2%80%8C%D9%87%D8%A7/move we have had tons of successful moves on fawiki in nearly every namespace]). I got this exact error recently when I tried to move an article from one title to another (same namespace). This selectivity is why I think AbuseFilter is involved.

Tagging relevant core tags, as it looks like it was caused by something in core.

FTR, three exceptions caused by my move attempt in mswiki (Bones -> Bones (siri TV); reqid b27894e2-1e49-42c6-bd43-f0f8df10f1d6):

2021-04-11 17:08:19 [b27894e2-1e49-42c6-bd43-f0f8df10f1d6] mw1333 mswiki 1.36.0-wmf.38 exception ERROR: [b27894e2-1e49-42c6-bd43-f0f8df10f1d6] /w/index.php?title=Khas:Pindah_laman&action=submit   MediaWiki\Revision\RevisionAccessException: Revision 4632993 belongs to page ID 187769, the provided Title object belongs to page ID 1051303 {"exception_url":"/w/index.php?title=Khas:Pindah_laman&action=submit","reqId":"b27894e2-1e49-42c6-bd43-f0f8df10f1d6","caught_by":"entrypoint"}
[Exception MediaWiki\Revision\RevisionAccessException] (/srv/mediawiki/php-1.36.0-wmf.38/includes/Revision/RevisionStore.php:1803) Revision 4632993 belongs to page ID 187769, the provided Title object belongs to page ID 1051303
  #0 /srv/mediawiki/php-1.36.0-wmf.38/includes/Revision/RevisionStore.php(3215): MediaWiki\Revision\RevisionStore->ensureRevisionRowMatchesTitle(stdClass, Title, array)
  #1 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(3535): MediaWiki\Revision\RevisionStore->getKnownCurrentRevision(Title)
  #2 [internal function]: Parser::statelessFetchRevisionRecord(Title, Parser)
  #3 /srv/mediawiki/php-1.36.0-wmf.38/includes/Revision/RenderedRevision.php(393): call_user_func(array, Title, Parser)
  #4 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(3480): MediaWiki\Revision\RenderedRevision->MediaWiki\Revision\{closure}(Title, Parser)
  #5 /srv/mediawiki/php-1.36.0-wmf.38/extensions/Scribunto/includes/engines/LuaCommon/TitleLibrary.php(294): Parser->fetchCurrentRevisionRecordOfTitle(Title)
  #6 /srv/mediawiki/php-1.36.0-wmf.38/extensions/Scribunto/includes/engines/LuaCommon/TitleLibrary.php(328): Scribunto_LuaTitleLibrary->getContentInternal(string)
  #7 /srv/mediawiki/php-1.36.0-wmf.38/extensions/Scribunto/includes/engines/LuaSandbox/LuaSandboxCallback.php(26): Scribunto_LuaTitleLibrary->getContent(string)
  #8 [internal function]: Scribunto_LuaSandboxCallback->__call(string, array)
  #9 /srv/mediawiki/php-1.36.0-wmf.38/extensions/Scribunto/includes/engines/LuaSandbox/LuaSandboxInterpreter.php(113): LuaSandboxFunction->call(LuaSandboxFunction)
  #10 /srv/mediawiki/php-1.36.0-wmf.38/extensions/Scribunto/includes/engines/LuaCommon/LuaEngine.php(296): Scribunto_LuaSandboxInterpreter->callFunction(LuaSandboxFunction, LuaSandboxFunction)
  #11 /srv/mediawiki/php-1.36.0-wmf.38/extensions/Scribunto/includes/engines/LuaCommon/LuaModule.php(68): Scribunto_LuaEngine->executeFunctionChunk(LuaSandboxFunction, PPTemplateFrame_Hash)
  #12 /srv/mediawiki/php-1.36.0-wmf.38/extensions/Scribunto/includes/common/Hooks.php(128): Scribunto_LuaModule->invoke(string, PPTemplateFrame_Hash)
  #13 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(3356): ScribuntoHooks::invokeHook(Parser, PPTemplateFrame_Hash, array)
  #14 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(3041): Parser->callParserFunction(PPTemplateFrame_Hash, string, array)
  #15 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/PPFrame_Hash.php(263): Parser->braceSubstitution(array, PPTemplateFrame_Hash)
  #16 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(2955): PPFrame_Hash->expand(PPNode_Hash_Tree)
  #17 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/PPFrame_Hash.php(263): Parser->braceSubstitution(array, PPTemplateFrame_Hash)
  #18 /srv/mediawiki/php-1.36.0-wmf.38/extensions/ParserFunctions/includes/ParserFunctions.php(919): PPFrame_Hash->expand(PPNode_Hash_Tree)
  #19 /srv/mediawiki/php-1.36.0-wmf.38/extensions/ParserFunctions/includes/ParserFunctions.php(139): MediaWiki\Extensions\ParserFunctions\ParserFunctions::decodeTrimExpand(PPNode_Hash_Tree, PPTemplateFrame_Hash)  #20 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(3356): MediaWiki\Extensions\ParserFunctions\ParserFunctions::ifeq(Parser, PPTemplateFrame_Hash, array)
  #21 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(3041): Parser->callParserFunction(PPTemplateFrame_Hash, string, array)
  #22 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/PPFrame_Hash.php(263): Parser->braceSubstitution(array, PPTemplateFrame_Hash)
  #23 /srv/mediawiki/php-1.36.0-wmf.38/extensions/ParserFunctions/includes/ParserFunctions.php(144): PPFrame_Hash->expand(PPNode_Hash_Tree)
  #24 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(3356): MediaWiki\Extensions\ParserFunctions\ParserFunctions::ifeq(Parser, PPTemplateFrame_Hash, array)
  #25 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(3041): Parser->callParserFunction(PPTemplateFrame_Hash, string, array)
  #26 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/PPFrame_Hash.php(263): Parser->braceSubstitution(array, PPTemplateFrame_Hash)
  #27 /srv/mediawiki/php-1.36.0-wmf.38/extensions/ParserFunctions/includes/ParserFunctions.php(144): PPFrame_Hash->expand(PPNode_Hash_Tree)
  #28 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(3356): MediaWiki\Extensions\ParserFunctions\ParserFunctions::ifeq(Parser, PPTemplateFrame_Hash, array)
  #29 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(3041): Parser->callParserFunction(PPTemplateFrame_Hash, string, array)
  #30 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/PPFrame_Hash.php(263): Parser->braceSubstitution(array, PPTemplateFrame_Hash)
  #31 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/PPTemplateFrame_Hash.php(97): PPFrame_Hash->expand(PPNode_Hash_Tree, integer)
  #32 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(3227): PPTemplateFrame_Hash->cachedExpand(string, PPNode_Hash_Tree)
  #33 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/PPFrame_Hash.php(263): Parser->braceSubstitution(array, PPFrame_Hash)
  #34 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(2879): PPFrame_Hash->expand(PPNode_Hash_Tree, integer)
  #35 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(1549): Parser->replaceVariables(string)
  #36 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(639): Parser->internalParse(string)
  #37 /srv/mediawiki/php-1.36.0-wmf.38/includes/content/WikitextContent.php(375): Parser->parse(string, Title, ParserOptions, boolean, boolean, integer)
  #38 /srv/mediawiki/php-1.36.0-wmf.38/includes/content/AbstractContent.php(591): WikitextContent->fillParserOutput(Title, integer, ParserOptions, boolean, ParserOutput)
  #39 /srv/mediawiki/php-1.36.0-wmf.38/includes/Revision/RenderedRevision.php(266): AbstractContent->getParserOutput(Title, integer, ParserOptions, boolean)
  #40 /srv/mediawiki/php-1.36.0-wmf.38/includes/Revision/RenderedRevision.php(235): MediaWiki\Revision\RenderedRevision->getSlotParserOutputUncached(WikitextContent, boolean)
  #41 /srv/mediawiki/php-1.36.0-wmf.38/includes/Revision/RevisionRenderer.php(217): MediaWiki\Revision\RenderedRevision->getSlotParserOutput(string, array)
  #42 /srv/mediawiki/php-1.36.0-wmf.38/includes/Revision/RevisionRenderer.php(154): MediaWiki\Revision\RevisionRenderer->combineSlotOutput(MediaWiki\Revision\RenderedRevision, array)
  #43 [internal function]: MediaWiki\Revision\RevisionRenderer->MediaWiki\Revision\{closure}(MediaWiki\Revision\RenderedRevision, array)
  #44 /srv/mediawiki/php-1.36.0-wmf.38/includes/Revision/RenderedRevision.php(197): call_user_func(Closure, MediaWiki\Revision\RenderedRevision, array)
  #45 /srv/mediawiki/php-1.36.0-wmf.38/includes/Storage/DerivedPageDataUpdater.php(1348): MediaWiki\Revision\RenderedRevision->getRevisionParserOutput()
  #46 [internal function]: MediaWiki\Storage\DerivedPageDataUpdater->getCanonicalParserOutput()
  #47 /srv/mediawiki/php-1.36.0-wmf.38/includes/edit/PreparedEdit.php(104): call_user_func(array)
  #48 /srv/mediawiki/php-1.36.0-wmf.38/includes/edit/PreparedEdit.php(119): MediaWiki\Edit\PreparedEdit->getOutput()
  #49 /srv/mediawiki/php-1.36.0-wmf.38/includes/Storage/DerivedPageDataUpdater.php(1326): MediaWiki\Edit\PreparedEdit->__get(string)
  #50 /srv/mediawiki/php-1.36.0-wmf.38/includes/Storage/DerivedPageDataUpdater.php(1528): MediaWiki\Storage\DerivedPageDataUpdater->getPreparedEdit()
  #51 /srv/mediawiki/php-1.36.0-wmf.38/includes/page/WikiPage.php(2381): MediaWiki\Storage\DerivedPageDataUpdater->doUpdates()
  #52 /srv/mediawiki/php-1.36.0-wmf.38/includes/MovePage.php(1096): WikiPage->doEditUpdates(MediaWiki\Revision\RevisionStoreRecord, User, array)
  #53 /srv/mediawiki/php-1.36.0-wmf.38/includes/MovePage.php(658): MovePage->moveToInternal(User, Title, string, boolean, array)
  #54 /srv/mediawiki/php-1.36.0-wmf.38/includes/MovePage.php(504): MovePage->moveUnsafe(User, string, boolean, array)
  #55 /srv/mediawiki/php-1.36.0-wmf.38/includes/specials/SpecialMovepage.php(861): MovePage->moveIfAllowed(User, string, boolean)
  #56 /srv/mediawiki/php-1.36.0-wmf.38/includes/specials/SpecialMovepage.php(204): MovePageForm->doSubmit()
  #57 /srv/mediawiki/php-1.36.0-wmf.38/includes/specialpage/SpecialPage.php(646): MovePageForm->execute(NULL)
  #58 /srv/mediawiki/php-1.36.0-wmf.38/includes/specialpage/SpecialPageFactory.php(1382): SpecialPage->run(NULL)
  #59 /srv/mediawiki/php-1.36.0-wmf.38/includes/MediaWiki.php(309): MediaWiki\SpecialPage\SpecialPageFactory->executePath(Title, RequestContext)
  #60 /srv/mediawiki/php-1.36.0-wmf.38/includes/MediaWiki.php(913): MediaWiki->performRequest()
  #61 /srv/mediawiki/php-1.36.0-wmf.38/includes/MediaWiki.php(546): MediaWiki->main()
  #62 /srv/mediawiki/php-1.36.0-wmf.38/index.php(53): MediaWiki->run()
  #63 /srv/mediawiki/php-1.36.0-wmf.38/index.php(46): wfIndexMain()
  #64 /srv/mediawiki/w/index.php(3): require(string)
  #65 {main}
2021-04-11 17:08:20 [b27894e2-1e49-42c6-bd43-f0f8df10f1d6] mw1333 mswiki 1.36.0-wmf.38 exception ERROR: [b27894e2-1e49-42c6-bd43-f0f8df10f1d6] /w/index.php?title=Khas:Pindah_laman&action=submit   InvalidArgumentException: The Title object yields no ID. Perhaps the page [[Bones_(siri_TV)]] doesn't exist? {"exception_url":"/w/index.php?title=Khas:Pindah_laman&action=submit","reqId":"b27894e2-1e49-42c6-bd43-f0f8df10f1d6","caught_by":"other"}
[Exception InvalidArgumentException] (/srv/mediawiki/php-1.36.0-wmf.38/includes/deferred/LinksUpdate.php:140) The Title object yields no ID. Perhaps the page [[Bones_(siri_TV)]] doesn't exist?
  #0 /srv/mediawiki/php-1.36.0-wmf.38/includes/Storage/DerivedPageDataUpdater.php(1384): LinksUpdate->__construct(Title, ParserOutput, boolean)
  #1 /srv/mediawiki/php-1.36.0-wmf.38/includes/deferred/RefreshSecondaryDataUpdate.php(83): MediaWiki\Storage\DerivedPageDataUpdater->getSecondaryDataUpdates(boolean)
  #2 /srv/mediawiki/php-1.36.0-wmf.38/includes/deferred/DeferredUpdates.php(513): RefreshSecondaryDataUpdate->doUpdate()
  #3 /srv/mediawiki/php-1.36.0-wmf.38/includes/deferred/DeferredUpdates.php(390): DeferredUpdates::attemptUpdate(RefreshSecondaryDataUpdate, Wikimedia\Rdbms\LBFactoryMulti)
  #4 /srv/mediawiki/php-1.36.0-wmf.38/includes/deferred/DeferredUpdates.php(221): DeferredUpdates::run(RefreshSecondaryDataUpdate, Wikimedia\Rdbms\LBFactoryMulti, Monolog\Logger, BufferingStatsdDataFactory, string)
  #5 /srv/mediawiki/php-1.36.0-wmf.38/includes/deferred/DeferredUpdatesScope.php(264): DeferredUpdates::{closure}(RefreshSecondaryDataUpdate, integer)
  #6 /srv/mediawiki/php-1.36.0-wmf.38/includes/deferred/DeferredUpdatesScope.php(196): DeferredUpdatesScope->processStageQueue(integer, integer, Closure)
  #7 /srv/mediawiki/php-1.36.0-wmf.38/includes/deferred/DeferredUpdates.php(242): DeferredUpdatesScope->processUpdates(integer, Closure)
  #8 /srv/mediawiki/php-1.36.0-wmf.38/includes/MediaWiki.php(1093): DeferredUpdates::doUpdates(string)
  #9 /srv/mediawiki/php-1.36.0-wmf.38/includes/MediaWiki.php(821): MediaWiki->restInPeace()
  #10 /srv/mediawiki/php-1.36.0-wmf.38/includes/MediaWiki.php(833): MediaWiki->{closure}()
  #11 /srv/mediawiki/php-1.36.0-wmf.38/includes/MediaWiki.php(571): MediaWiki->doPostOutputShutdown()
  #12 /srv/mediawiki/php-1.36.0-wmf.38/index.php(53): MediaWiki->run()
  #13 /srv/mediawiki/php-1.36.0-wmf.38/index.php(46): wfIndexMain()
  #14 /srv/mediawiki/w/index.php(3): require(string)
  #15 {main}
2021-04-11 17:08:20 [b27894e2-1e49-42c6-bd43-f0f8df10f1d6] mw1333 mswiki 1.36.0-wmf.38 exception ERROR: [b27894e2-1e49-42c6-bd43-f0f8df10f1d6] /w/index.php?title=Khas:Pindah_laman&action=submit   InvalidArgumentException: The Title object yields no ID. Perhaps the page [[Perbincangan:Bones_(siri_TV)]] doesn't exist? {"exception_url":"/w/index.php?title=Khas:Pindah_laman&action=submit","reqId":"b27894e2-1e49-42c6-bd43-f0f8df10f1d6","caught_by":"other"}
[Exception InvalidArgumentException] (/srv/mediawiki/php-1.36.0-wmf.38/includes/deferred/LinksUpdate.php:140) The Title object yields no ID. Perhaps the page [[Perbincangan:Bones_(siri_TV)]] doesn't exist?
  #0 /srv/mediawiki/php-1.36.0-wmf.38/includes/Storage/DerivedPageDataUpdater.php(1384): LinksUpdate->__construct(Title, ParserOutput, boolean)
  #1 /srv/mediawiki/php-1.36.0-wmf.38/includes/deferred/RefreshSecondaryDataUpdate.php(83): MediaWiki\Storage\DerivedPageDataUpdater->getSecondaryDataUpdates(boolean)
  #2 /srv/mediawiki/php-1.36.0-wmf.38/includes/deferred/DeferredUpdates.php(513): RefreshSecondaryDataUpdate->doUpdate()
  #3 /srv/mediawiki/php-1.36.0-wmf.38/includes/deferred/DeferredUpdates.php(390): DeferredUpdates::attemptUpdate(RefreshSecondaryDataUpdate, Wikimedia\Rdbms\LBFactoryMulti)
  #4 /srv/mediawiki/php-1.36.0-wmf.38/includes/deferred/DeferredUpdates.php(221): DeferredUpdates::run(RefreshSecondaryDataUpdate, Wikimedia\Rdbms\LBFactoryMulti, Monolog\Logger, BufferingStatsdDataFactory, string)
  #5 /srv/mediawiki/php-1.36.0-wmf.38/includes/deferred/DeferredUpdatesScope.php(264): DeferredUpdates::{closure}(RefreshSecondaryDataUpdate, integer)
  #6 /srv/mediawiki/php-1.36.0-wmf.38/includes/deferred/DeferredUpdatesScope.php(196): DeferredUpdatesScope->processStageQueue(integer, integer, Closure)
  #7 /srv/mediawiki/php-1.36.0-wmf.38/includes/deferred/DeferredUpdates.php(242): DeferredUpdatesScope->processUpdates(integer, Closure)
  #8 /srv/mediawiki/php-1.36.0-wmf.38/includes/MediaWiki.php(1093): DeferredUpdates::doUpdates(string)
  #9 /srv/mediawiki/php-1.36.0-wmf.38/includes/MediaWiki.php(821): MediaWiki->restInPeace()
  #10 /srv/mediawiki/php-1.36.0-wmf.38/includes/MediaWiki.php(833): MediaWiki->{closure}()
  #11 /srv/mediawiki/php-1.36.0-wmf.38/includes/MediaWiki.php(571): MediaWiki->doPostOutputShutdown()
  #12 /srv/mediawiki/php-1.36.0-wmf.38/index.php(53): MediaWiki->run()
  #13 /srv/mediawiki/php-1.36.0-wmf.38/index.php(46): wfIndexMain()
  #14 /srv/mediawiki/w/index.php(3): require(string)
  #15 {main}

In T279832#6989667, @Urbanecm wrote:

...
  #15 {main}
2021-04-11 17:08:20 [b27894e2-1e49-42c6-bd43-f0f8df10f1d6] mw1333 mswiki 1.36.0-wmf.38 exception ERROR: [b27894e2-1e49-42c6-bd43-f0f8df10f1d6] /w/index.php?title=Khas:Pindah_laman&action=submit   InvalidArgumentException: The Title object yields no ID. Perhaps the page [[Perbincangan:Bones_(siri_TV)]] doesn't exist? {"exception_url":"/w/index.php?title=Khas:Pindah_laman&action=submit","reqId":"b27894e2-1e49-42c6-bd43-f0f8df10f1d6","caught_by":"other"}
[Exception InvalidArgumentException] (/srv/mediawiki/php-1.36.0-wmf.38/includes/deferred/LinksUpdate.php:140) The Title object yields no ID. Perhaps the page [[Perbincangan:Bones_(siri_TV)]] doesn't exist?
...

Why is the existence of the target of your move being checked? Of course it doesn't exist.

In T279832#6989666, @Huji wrote:

Can someone with access to logstash provide an update on which wikis are affected and since when? We know fawiki, dewiki and mswiki are affected but who else?

Sure. https://logstash.wikimedia.org/goto/3b038c75fbae60f5e5f4777ed393b1d0 is my query, FTR.

Apparently this is not a recent error, as I can see errors triggered in wmf.34 and earlier. For affected wikis, it clearly affects mostly ruwiki, fawiki and cewiki. It doesn't appear to affect every wiki.

Also, as far as fawiki, we know this was reported as early as April 9th (one day after 1.36.0-wmf.38 was deployed); do we have logs from before April 8th?

Yes, logs are kept for ~90 days.

Of note, this is not impacting every move (e.g. [https://fa.wikipedia.org/wiki/%D9%88%DB%8C%DA%98%D9%87:%D8%B3%DB%8C%D8%A7%D9%87%D9%87%E2%80%8C%D9%87%D8%A7/move we have had tons of successful moves on fawiki in nearly every namespace]). I got this exact error recently when I tried to move an article from one title to another (same namespace). This selectivity is why I think AbuseFilter is involved.

Fortunately, that's easy to test when there's a stable way to reproduce. I deactivated AbuseFilter on a debug server, and tried to move a page again, and it still didn't get moved.

In T279832#6989698, @Urbanecm wrote:

[...]
Fortunately, that's easy to test when there's a stable way to reproduce. I deactivated AbuseFilter on a debug server, and tried to move a page again, and it still didn't get moved.

Uninstalling Scribunto and ParserFunctions (two extensions listed in the stacktrace) helped.

In T279832#6989698, @Urbanecm wrote:

I deactivated AbuseFilter on a debug server, and tried to move a page again, and it still didn't get moved.

That is what I wanted to suggest but you beat me to it.

Do we have a standard script to take in an error log as input, scrape off the PHP file names, and return their latest change timestamp from git? This can help generate an initial list of target files to look at.

In T279832#6989699, @Urbanecm wrote:

In T279832#6989698, @Urbanecm wrote:

[...]
Fortunately, that's easy to test when there's a stable way to reproduce. I deactivated AbuseFilter on a debug server, and tried to move a page again, and it still didn't get moved.

Uninstalling Scribunto and ParserFunctions (two extensions listed in the stacktrace) helped.

Does uninstalling Scribunto alone help?

@Urbanecm and I just test this on mwdebug1001 with this page on fawiki: فهرست نامداران بابل. An attempt to move it to فهرست اهالی بابل failed on PROD, but succeeded on mwdebug1001 when Scribunto was disabled. The target page did not exist before the move.

After the move, I tried moving it back to the original title; it failed again on PROD.

Conclusions:

• It has nothing to do with whether the target of the move exists or not.
• It most likely has something to do with Scribunto.

Next, we removed all of the templates from the page; we assumed that without templates and Lua Modules used by the page, Scribunto should not be engaged. Yet the move failed again in PROD and the trace back was still the same, indicating that includes/parser/Parser.php was triggering Scribunto through the hooks.

I even went as far as completely emptying the page, saving it, and attempting the move and it failed.

Stack trace for when I attempted to move the page while it was empty is shown below. I don't know the MW Parser well, but is surprising to me that PPTemplateFrame_Hash is instantiated when the page has absolutely no content.

2021-04-11 18:19:24 [5e0866de-ab49-4d60-9eb1-dc47902584c9] mw1395 fawiki 1.36.0-wmf.38 exception ERROR: [5e0866de-ab49-4d60-9eb1-dc47902584c9] /w/index.php?title=%D9%88%DB%8C%DA%98%D9%87:%D8%A7%D9%86%D8%AA%D9%82%D8%A7%D9%84_%D8%B5%D9%81%D8%AD%D9%87&action=submit   MediaWiki\Revision\RevisionAccessException: Revision 31758637 belongs to page ID 1869134, the provided Title object belongs to page ID 5602792 {"exception_url":"/w/index.php?title=%D9%88%DB%8C%DA%98%D9%87:%D8%A7%D9%86%D8%AA%D9%82%D8%A7%D9%84_%D8%B5%D9%81%D8%AD%D9%87&action=submit","reqId":"5e0866de-ab49-4d60-9eb1-dc47902584c9","caught_by":"entrypoint"}
[Exception MediaWiki\Revision\RevisionAccessException] (/srv/mediawiki/php-1.36.0-wmf.38/includes/Revision/RevisionStore.php:1803) Revision 31758637 belongs to page ID 1869134, the provided Title object belongs to page ID 5602792
  #0 /srv/mediawiki/php-1.36.0-wmf.38/includes/Revision/RevisionStore.php(3215): MediaWiki\Revision\RevisionStore->ensureRevisionRowMatchesTitle(stdClass, Title, array)
  #1 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(3535): MediaWiki\Revision\RevisionStore->getKnownCurrentRevision(Title)
  #2 [internal function]: Parser::statelessFetchRevisionRecord(Title, Parser)
  #3 /srv/mediawiki/php-1.36.0-wmf.38/includes/Revision/RenderedRevision.php(393): call_user_func(array, Title, Parser)
  #4 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(3480): MediaWiki\Revision\RenderedRevision->MediaWiki\Revision\{closure}(Title, Parser)
  #5 /srv/mediawiki/php-1.36.0-wmf.38/extensions/Scribunto/includes/engines/LuaCommon/TitleLibrary.php(294): Parser->fetchCurrentRevisionRecordOfTitle(Title)
  #6 /srv/mediawiki/php-1.36.0-wmf.38/extensions/Scribunto/includes/engines/LuaCommon/TitleLibrary.php(328): Scribunto_LuaTitleLibrary->getContentInternal(string)
  #7 /srv/mediawiki/php-1.36.0-wmf.38/extensions/Scribunto/includes/engines/LuaSandbox/LuaSandboxCallback.php(26): Scribunto_LuaTitleLibrary->getContent(string)
  #8 [internal function]: Scribunto_LuaSandboxCallback->__call(string, array)
  #9 /srv/mediawiki/php-1.36.0-wmf.38/extensions/Scribunto/includes/engines/LuaSandbox/LuaSandboxInterpreter.php(113): LuaSandboxFunction->call(LuaSandboxFunction)
  #10 /srv/mediawiki/php-1.36.0-wmf.38/extensions/Scribunto/includes/engines/LuaCommon/LuaEngine.php(296): Scribunto_LuaSandboxInterpreter->callFunction(LuaSandboxFunction, LuaSandboxFunction)
  #11 /srv/mediawiki/php-1.36.0-wmf.38/extensions/Scribunto/includes/engines/LuaCommon/LuaModule.php(68): Scribunto_LuaEngine->executeFunctionChunk(LuaSandboxFunction, PPTemplateFrame_Hash)
  #12 /srv/mediawiki/php-1.36.0-wmf.38/extensions/Scribunto/includes/common/Hooks.php(128): Scribunto_LuaModule->invoke(string, PPTemplateFrame_Hash)
  #13 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(3356): ScribuntoHooks::invokeHook(Parser, PPTemplateFrame_Hash, array)
  #14 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(3041): Parser->callParserFunction(PPTemplateFrame_Hash, string, array)
  #15 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/PPFrame_Hash.php(263): Parser->braceSubstitution(array, PPTemplateFrame_Hash)
  #16 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(2955): PPFrame_Hash->expand(PPNode_Hash_Tree)
  #17 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/PPFrame_Hash.php(263): Parser->braceSubstitution(array, PPTemplateFrame_Hash)
  #18 /srv/mediawiki/php-1.36.0-wmf.38/extensions/ParserFunctions/includes/ParserFunctions.php(919): PPFrame_Hash->expand(PPNode_Hash_Tree)
  #19 /srv/mediawiki/php-1.36.0-wmf.38/extensions/ParserFunctions/includes/ParserFunctions.php(139): MediaWiki\Extensions\ParserFunctions\ParserFunctions::decodeTrimExpand(PPNode_Hash_Tree, PPTemplateFrame_Hash)  #20 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(3356): MediaWiki\Extensions\ParserFunctions\ParserFunctions::ifeq(Parser, PPTemplateFrame_Hash, array)
  #21 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(3041): Parser->callParserFunction(PPTemplateFrame_Hash, string, array)
  #22 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/PPFrame_Hash.php(263): Parser->braceSubstitution(array, PPTemplateFrame_Hash)
  #23 /srv/mediawiki/php-1.36.0-wmf.38/extensions/ParserFunctions/includes/ParserFunctions.php(144): PPFrame_Hash->expand(PPNode_Hash_Tree)
  #24 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(3356): MediaWiki\Extensions\ParserFunctions\ParserFunctions::ifeq(Parser, PPTemplateFrame_Hash, array)
  #25 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(3041): Parser->callParserFunction(PPTemplateFrame_Hash, string, array)
  #26 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/PPFrame_Hash.php(263): Parser->braceSubstitution(array, PPTemplateFrame_Hash)
  #27 /srv/mediawiki/php-1.36.0-wmf.38/extensions/ParserFunctions/includes/ParserFunctions.php(144): PPFrame_Hash->expand(PPNode_Hash_Tree)
  #28 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(3356): MediaWiki\Extensions\ParserFunctions\ParserFunctions::ifeq(Parser, PPTemplateFrame_Hash, array)
  #29 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(3041): Parser->callParserFunction(PPTemplateFrame_Hash, string, array)
  #30 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/PPFrame_Hash.php(263): Parser->braceSubstitution(array, PPTemplateFrame_Hash)
  #31 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/PPTemplateFrame_Hash.php(97): PPFrame_Hash->expand(PPNode_Hash_Tree, integer)
  #32 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(3227): PPTemplateFrame_Hash->cachedExpand(string, PPNode_Hash_Tree)
  #33 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/PPFrame_Hash.php(263): Parser->braceSubstitution(array, PPFrame_Hash)
  #34 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(2879): PPFrame_Hash->expand(PPNode_Hash_Tree, integer)
  #35 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(1549): Parser->replaceVariables(string)
  #36 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(639): Parser->internalParse(string)
  #37 /srv/mediawiki/php-1.36.0-wmf.38/includes/content/WikitextContent.php(375): Parser->parse(string, Title, ParserOptions, boolean, boolean, integer)
  #38 /srv/mediawiki/php-1.36.0-wmf.38/includes/content/AbstractContent.php(591): WikitextContent->fillParserOutput(Title, integer, ParserOptions, boolean, ParserOutput)
  #39 /srv/mediawiki/php-1.36.0-wmf.38/includes/Revision/RenderedRevision.php(266): AbstractContent->getParserOutput(Title, integer, ParserOptions, boolean)
  #40 /srv/mediawiki/php-1.36.0-wmf.38/includes/Revision/RenderedRevision.php(235): MediaWiki\Revision\RenderedRevision->getSlotParserOutputUncached(WikitextContent, boolean)
  #41 /srv/mediawiki/php-1.36.0-wmf.38/includes/Revision/RevisionRenderer.php(217): MediaWiki\Revision\RenderedRevision->getSlotParserOutput(string, array)
  #42 /srv/mediawiki/php-1.36.0-wmf.38/includes/Revision/RevisionRenderer.php(154): MediaWiki\Revision\RevisionRenderer->combineSlotOutput(MediaWiki\Revision\RenderedRevision, array)
  #43 [internal function]: MediaWiki\Revision\RevisionRenderer->MediaWiki\Revision\{closure}(MediaWiki\Revision\RenderedRevision, array)
  #44 /srv/mediawiki/php-1.36.0-wmf.38/includes/Revision/RenderedRevision.php(197): call_user_func(Closure, MediaWiki\Revision\RenderedRevision, array)
  #45 /srv/mediawiki/php-1.36.0-wmf.38/includes/Storage/DerivedPageDataUpdater.php(1348): MediaWiki\Revision\RenderedRevision->getRevisionParserOutput()
  #46 /srv/mediawiki/php-1.36.0-wmf.38/includes/Storage/DerivedPageDataUpdater.php(1740): MediaWiki\Storage\DerivedPageDataUpdater->getCanonicalParserOutput()
  #47 /srv/mediawiki/php-1.36.0-wmf.38/includes/Storage/DerivedPageDataUpdater.php(1491): MediaWiki\Storage\DerivedPageDataUpdater->doParserCacheUpdate()
  #48 /srv/mediawiki/php-1.36.0-wmf.38/includes/page/WikiPage.php(2381): MediaWiki\Storage\DerivedPageDataUpdater->doUpdates()
  #49 /srv/mediawiki/php-1.36.0-wmf.38/includes/MovePage.php(1096): WikiPage->doEditUpdates(MediaWiki\Revision\RevisionStoreRecord, User, array)
  #50 /srv/mediawiki/php-1.36.0-wmf.38/includes/MovePage.php(658): MovePage->moveToInternal(User, Title, string, boolean, array)
  #51 /srv/mediawiki/php-1.36.0-wmf.38/includes/MovePage.php(504): MovePage->moveUnsafe(User, string, boolean, array)
  #52 /srv/mediawiki/php-1.36.0-wmf.38/includes/specials/SpecialMovepage.php(861): MovePage->moveIfAllowed(User, string, boolean)
  #53 /srv/mediawiki/php-1.36.0-wmf.38/includes/specials/SpecialMovepage.php(204): MovePageForm->doSubmit()
  #54 /srv/mediawiki/php-1.36.0-wmf.38/includes/specialpage/SpecialPage.php(646): MovePageForm->execute(NULL)
  #55 /srv/mediawiki/php-1.36.0-wmf.38/includes/specialpage/SpecialPageFactory.php(1382): SpecialPage->run(NULL)
  #56 /srv/mediawiki/php-1.36.0-wmf.38/includes/MediaWiki.php(309): MediaWiki\SpecialPage\SpecialPageFactory->executePath(Title, RequestContext)
  #57 /srv/mediawiki/php-1.36.0-wmf.38/includes/MediaWiki.php(913): MediaWiki->performRequest()
  #58 /srv/mediawiki/php-1.36.0-wmf.38/includes/MediaWiki.php(546): MediaWiki->main()
  #59 /srv/mediawiki/php-1.36.0-wmf.38/index.php(53): MediaWiki->run()
  #60 /srv/mediawiki/php-1.36.0-wmf.38/index.php(46): wfIndexMain()
  #61 /srv/mediawiki/w/index.php(3): require(string)
  #62 {main}

Update: this line is supposed to stop replaceVariables() when text length is zero (page is empty) yet it is somehow being missed and code reaches $frame->expand(..) on line 2879 as shown in the stack trace. This is really bizarre.

@Urbanecm see my update on my last comment right above. Could we modify mwdebug1001 through a patch that would log the value of $text on line 2862 of Parser.php so we can look into it and see why Parser does not find it to have a length of zero?

In T279832#6989748, @Huji wrote:

@Urbanecm see my update on my last comment right above. Could we modify mwdebug1001 through a patch that would log the value of $text on line 2862 of Parser.php so we can look into it and see why Parser does not find it to have a length of zero?

Bingo :). It's good to note that parser does not parse only the page, but also a lot of messages. I added a wfDebug() call there, and the last thing that gets parsed before the exception is {{R from move}}. MediaWiki:Move-redirect-text is the interface message that has this template. Emptying that interface message allowed me to move a page on fawiki.

Okay that was really helpful. I did not know whether it is the page content of the "source" page or the "destination" page that is triggering the issue. Evidently, it is the content of the source page (which is autoreplaced with {{R from move}}). And that template does use a module (Module:Redirect). Let's see where I can go from here.

Here is another observation. After going into Scribunto's code and coming out of it, the execution path described by the stack trace ends with this exception:

[Exception MediaWiki\Revision\RevisionAccessException] (/srv/mediawiki/php-1.36.0-wmf.38/includes/Revision/RevisionStore.php:1803) Revision 31758637 belongs to page ID 1869134, the provided Title object belongs to page ID 5602792

Page ID 1869134 belongs to the source of that attempted move, i.e., article titled فهرست_اهالی_بابل in main namespace. But the other page id, 5602792, is nowhere to be found; it is neither in page table nor in archive table. At least that is what my query results on Wikireplicas indicate.

Another piece of interesting information shared by a user on fawiki: the move they attempted failed, then they unchecked the "Move associated talk page" option and the move worked. They could subsequently move the talk page separate with success.

So we are looking for something that has to do with the relationship of the page and its talk page. Looking at the source of fa:Template:R from move on fawiki, I notice this piece that could be relevant:

{{#ifeq: {{PAGENAME:{{#invoke:redirect|main|{{TALKPAGENAME}}}}}}|{{PAGENAME:{{#invoke:redirect|main|{{SUBJECTPAGENAME}}}}}}||{{#ifeq: {{FULLPAGENAME}}|{{SUBJECTPAGENAME}}||[[رده:صفحه و بحثش تغییرمسیر به دو جای مختلف]]}}}}}}

To help non-Persian readers, [[رده:صفحه و بحثش تغییرمسیر به دو جای مختلف]] translates to [[Category:Unsynchronized talk page redirects]]. Clearly relevant.

Looking at the source of fa:Module:Redirect I can see that this template fetches a MW title; I am guessing that is the part that maps to the first few lines of the stack trace, which I have copied below again. Specifically, I think the above template code invokes the getTitle() method of the Lua module, which eventually calls fetchCurrentRevisionRecordOfTitle() here.

[Exception MediaWiki\Revision\RevisionAccessException] (/srv/mediawiki/php-1.36.0-wmf.38/includes/Revision/RevisionStore.php:1803) Revision 31758637 belongs to page ID 1869134, the provided Title object belongs to page ID 5602792
  #0 /srv/mediawiki/php-1.36.0-wmf.38/includes/Revision/RevisionStore.php(3215): MediaWiki\Revision\RevisionStore->ensureRevisionRowMatchesTitle(stdClass, Title, array)
  #1 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(3535): MediaWiki\Revision\RevisionStore->getKnownCurrentRevision(Title)
  #2 [internal function]: Parser::statelessFetchRevisionRecord(Title, Parser)
  #3 /srv/mediawiki/php-1.36.0-wmf.38/includes/Revision/RenderedRevision.php(393): call_user_func(array, Title, Parser)
  #4 /srv/mediawiki/php-1.36.0-wmf.38/includes/parser/Parser.php(3480): MediaWiki\Revision\RenderedRevision->MediaWiki\Revision\{closure}(Title, Parser)
  #5 /srv/mediawiki/php-1.36.0-wmf.38/extensions/Scribunto/includes/engines/LuaCommon/TitleLibrary.php(294): Parser->fetchCurrentRevisionRecordOfTitle(Title)
  #6 /srv/mediawiki/php-1.36.0-wmf.38/extensions/Scribunto/includes/engines/LuaCommon/TitleLibrary.php(328): Scribunto_LuaTitleLibrary->getContentInternal(string)
  #7 /srv/mediawiki/php-1.36.0-wmf.38/extensions/Scribunto/includes/engines/LuaSandbox/LuaSandboxCallback.php(26): Scribunto_LuaTitleLibrary->getContent(string)
  #8 [internal function]: Scribunto_LuaSandboxCallback->__call(string, array)

Aside from being challenged in my understanding of both the parser and Lua, I am also challenged by yet being to find a related file in the source that was edited recently.

Having established that this has to do with the page AND the talk page, I copied the current contents of the page and the talk page to another page and talk page in the same namespace on fawiki. To my surprise, this new pair was possible to move without an error.

In T279832#6989770, @Urbanecm wrote:

In T279832#6989748, @Huji wrote:

@Urbanecm see my update on my last comment right above. Could we modify mwdebug1001 through a patch that would log the value of $text on line 2862 of Parser.php so we can look into it and see why Parser does not find it to have a length of zero?

Bingo :). It's good to note that parser does not parse only the page, but also a lot of messages. I added a wfDebug() call there, and the last thing that gets parsed before the exception is {{R from move}}. MediaWiki:Move-redirect-text is the interface message that has this template. Emptying that interface message allowed me to move a page on fawiki.

For the record, https://people.wikimedia.org/~urbanecm/nda/XWikimediaDebug.log-20210411-first-move-redacted.gz [NDA-only link; @Huji should have access; my IP address/cookies redacted, as well as all logs following the error] should be full debug logs captured for my yesterday experiment. Maybe something else attracts your attention there.

In T279832#6989980, @Huji wrote:

Here is another observation. After going into Scribunto's code and coming out of it, the execution path described by the stack trace ends with this exception:

[Exception MediaWiki\Revision\RevisionAccessException] (/srv/mediawiki/php-1.36.0-wmf.38/includes/Revision/RevisionStore.php:1803) Revision 31758637 belongs to page ID 1869134, the provided Title object belongs to page ID 5602792

Page ID 1869134 belongs to the source of that attempted move, i.e., article titled فهرست_اهالی_بابل in main namespace. But the other page id, 5602792, is nowhere to be found; it is neither in page table nor in archive table. At least that is what my query results on Wikireplicas indicate.

Confirmed via a query on the unredacted internal replicas. I bet the nonexistent ID is happening because the move failed, and the DB transaction got rollbacked.

To further debug what's doing on, I enabled SQL queries log on mwdebug1001, and tried to move a page again. https://people.wikimedia.org/~urbanecm/nda/XWikimediaDebug.log-20210412-sql-debug-enabled-redacted.gz [NDA-only link] is the result. In my case, the exception message reads (posting to make the IDs clear):

2021-04-12 15:15:26 [05cae92c-56f9-4199-aca4-b5856a6c8070] mwdebug1001 fawiki 1.36.0-wmf.38 exception ERROR: [05cae92c-56f9-4199-aca4-b5856a6c8070] /w/index.php?title=%D9%88%DB%8C%DA%98%D9%87:%D8%A7%D9%86%D8%AA%D9%82%D8%A7%D9%84_%D8%B5%D9%81%D8%AD%D9%87&action=submit   MediaWiki\Revision\RevisionAccessException: Revision 31759990 belongs to page ID 1869134, the provided Title object belongs to page ID 5604330

One of the many queries that get logged is:

2021-04-12 15:15:26 [05cae92c-56f9-4199-aca4-b5856a6c8070] mwdebug1001 fawiki 1.36.0-wmf.38 DBQuery DEBUG: WikiPage::insertOn [0.001s] 10.64.48.109: INSERT IGNORE INTO `page` (page_namespace,page_title,page_restrictions,page_is_redirect,page_is_new,page_random,page_touched,page_latest,page_len) VALUES (0,'فهرست_اهالی_بابل','',0,1,'0.383871357595','20210412151526',0,0) {"method":"WikiPage::insertOn","db_host":"10.64.48.109","sql":"INSERT IGNORE INTO `page` (page_namespace,page_title,page_restrictions,page_is_redirect,page_is_new,page_random,page_touched,page_latest,page_len) VALUES (0,'فهرست_اهالی_بابل','',0,1,'0.383871357595','20210412151526',0,0)","domain":"fawiki","runtime":0.001}
2021-04-12 15:15:26 [05cae92c-56f9-4199-aca4-b5856a6c8070] mwdebug1001 fawiki 1.36.0-wmf.38 DBQuery DEBUG: MediaWiki\Revision\RevisionStore::getPreviousRevisionId [0.001s] 10.64.48.109: SELECT  page_latest  FROM `page`    WHERE page_id = 5604330  LIMIT 1   {"method":"MediaWiki\\Revision\\RevisionStore::getPreviousRevisionId","db_host":"10.64.48.109","sql":"SELECT  page_latest  FROM `page`    WHERE page_id = 5604330  LIMIT 1  ","domain":"fawiki","runtime":0.001}

I assume that this is where page_id 5604330 gets into play.

In T279832#6991608, @Urbanecm wrote:

For the record, https://people.wikimedia.org/~urbanecm/nda/XWikimediaDebug.log-20210411-first-move-redacted.gz [NDA-only link; @Huji should have access; my IP address/cookies redacted, as well as all logs following the error] should be full debug logs captured for my yesterday experiment. Maybe something else attracts your attention there.

It says I am not authorized :(

... I bet the nonexistent ID is happening because the move failed, and the DB transaction got rollbacked.
...
I assume that this is where page_id 5604330 gets into play.

It actually makes sense; the page id is close to that of other pages that were created among the time the move failed (at the time of this writing, the largest page_id for fawiki is 5605532).

Let's think through this together: you try to move a page that has a talk page, the source page is replaced with {{R from move}} and this template kicks Scribunto into action, the module tries to get a title object for the .. source page? destination page? source/destination talk page? .. and it fails because the INSERT command that was supposed to create that page was already rolled back.

Interesting observation: the function that invokes that last SQL query is getPreviousRevisionId(). This function is rarely used in core and extensions. I think the specific call is here inside the insertRevisionOn() method. The page has been moved and the new redirect that is created in its place should have only one revision, so $rev->getParentId() for it should be null which is what line 474 checks, and in line 475 getPreviousRevisionId is called.

Of note, getPreviousRevisionId() is deprecated.

Anyway, the prior step in the trace should be here (may be you can verify that using the trace log that you tried to give me access to above?) but I cannot figure out where the problem is yet.

In T279832#6992405, @Huji wrote:

In T279832#6991608, @Urbanecm wrote:

For the record, https://people.wikimedia.org/~urbanecm/nda/XWikimediaDebug.log-20210411-first-move-redacted.gz [NDA-only link; @Huji should have access; my IP address/cookies redacted, as well as all logs following the error] should be full debug logs captured for my yesterday experiment. Maybe something else attracts your attention there.

It says I am not authorized :(

Interesting. Which credentials are you trying to use? It's supposed to accept LDAP/developer/wikitech credentials. I just added Huji (in addition to huji) to the list of individually authorized users, hopefully that would fix it. If it still doesn't work, PM me at IRC, so I can figure out how to properly give you access there.

In T279832#6992425, @Urbanecm wrote:

... I just added Huji (in addition to huji) to the list of individually authorized users, hopefully that would fix it.

It did fix it. I am not familiar with this platform or where to find the source code for it. Can I ask you to figure out where to file a task/ticket/bug so that uppercasing of the first letter of usernames would be handled gracefully by the people.wikimedia.org platform itself?

In T279832#6992539, @Huji wrote:

In T279832#6992425, @Urbanecm wrote:

... I just added Huji (in addition to huji) to the list of individually authorized users, hopefully that would fix it.

It did fix it. I am not familiar with this platform or where to find the source code for it. Can I ask you to figure out where to file a task/ticket/bug so that uppercasing of the first letter of usernames would be handled gracefully by the people.wikimedia.org platform itself?

Cool, I'm happy to hear it – let me know what you think about the logs.

The login form you saw comes from standard IDP that covers various services, and people with access to people.wikimedia.org can use it to restrict access to certain files (see https://wikitech.wikimedia.org/wiki/People.wikimedia.org#How_To_add_SSO_Authentication). The right tag to use would be CAS-SSO.

I have not made any further progress on this, and I am still unable to replicate this on my local wiki.

Tickets that look like they may be related:

• T259022: Revision 105215021 belongs to page ID 2442048, the provided Title object belongs to page ID 8437055
• T263340: Fatal InvalidArgumentException "the provided Title object belongs to page" when viewing pages with history broken by Special:MergeHistory
• T268910: Page can't be moved: InvalidArgumentException: Revision belongs to different page ID than Title object

Pinging @tstarling to have a look.

I'm unclear on the current impact of this issue.

If this is still affecting a significant number of pages on some wiki, it should be bumped back to UBN.

It is hard to quantify the impact (only those with logstash access can quantify it really). But I can tell you that in the last few weeks, at least 6 or 7 users have reported it on fawiki for various pages. I can also tell you that in that same period, hundreds of pages have been successfully moved on fawiki. So your original assessment of "it only impacts some pages on some wikis" is correct, but because no solid workaround exists, I still prefer this to be UBN and attract more attention from the developer community and the WMF engineers.

We could just remove the exception. Log a warning instead.

I've poked at this a bit -- it's more difficult to reproduce than I assumed. We certainly do have an old pre-move Title object stuck in the Title cache, but loading revisions with it does not appear to trigger the exception, because RevisionStore just loads by page_id, it doesn't actually care if the title text is wrong. If you do Title::newFromText('Foo'), then overwrite Title::$mLatestID and Title::$mArticleID to point to [[Bar]], RevisionLookup will happily load the revision from Bar. If $mLatestID and $mArticleID are inconsistent with each other, that will trigger the exception, but merely being stale should not be enough to do that.

In T279832#7007553, @Huji wrote:

It is hard to quantify the impact (only those with logstash access can quantify it really). But I can tell you that in the last few weeks, at least 6 or 7 users have reported it on fawiki for various pages. I can also tell you that in that same period, hundreds of pages have been successfully moved on fawiki. So your original assessment of "it only impacts some pages on some wikis" is correct, but because no solid workaround exists, I still prefer this to be UBN and attract more attention from the developer community and the WMF engineers.

UBN generally means "If you know anything about this area, drop whatever you're doing and work on this, things are on fire". For attracting attention for a bug not in this category, "High" is best.

In T279832#7008056, @tstarling wrote:

We could just remove the exception. Log a warning instead.

What would the side effects be, if the error is silently ignored and the move continues?

In T279832#7008102, @ArielGlenn wrote:

In T279832#7008056, @tstarling wrote:

We could just remove the exception. Log a warning instead.

What would the side effects be, if the error is silently ignored and the move continues?

There is two errors mentioned in the task description:

1. The "provided Title object belongs to" error could be ignored, but it should at least cause tests to fail. This should not be taken lightly, since the inconsistency may introduce data corruption.
2. The "The Title object yields no ID" cannot be ignored - LinksUpdate cannot function without the page ID. We could make it so that the request doesn't fail, but the database state would be incomplete. On the other hand, if the page really doesn't exist in the DB, not running LinksUpdate for it would be fine I guess.

I suspect that the first error may be causing a rollback that triggers the second error - the rollback causes the page row with the new page ID to disappear from the database, so LinksUpdate fails. We should not schedule LinksUpdate until the db transaction has been successfully committed. Perhaps that is something that should be built into DeferredUpdate.

In above comments, we found some evidence that both the page being moved and its talk page played a role in this issue raising. We also found that users can work around this bug by moving the page and the talk page separately.

Today, when trying to move a page that was again impacted by this bug (from fa:فیچیت to fa:پیچیت) I realized that the talk page can be speedy deleted. Once the talk page was deleted, the move worked. This, again, confirms that the situation in which this bug is manifested has to do with both the page and the talk page (and the fact that {{R for move}} needs to access the talk page through its underlying Lua module).

In T279832#7013970, @Huji wrote:

We also found that users can work around this bug by moving the page and the talk page separately.

Can we use it as a workaround? I've seen many complains about this error at pt.wiki.

In T279832#7013970, @Huji wrote:

In above comments, we found some evidence that both the page being moved and its talk page played a role in this issue raising. We also found that users can work around this bug by moving the page and the talk page separately.

That is very interesting - do you have a way to reliably reproduce the error? Can you set up some "immovable" sandbox page that can be used to investigate this? We have so far been unable to reproduce the issue in a testing environment. Being able to reproduce it on the life wiki, somewhere were it's not bothering anyone, would perhaps allow us to find the issue.

In T279832#7027441, @daniel wrote:

In T279832#7013970, @Huji wrote:

In above comments, we found some evidence that both the page being moved and its talk page played a role in this issue raising. We also found that users can work around this bug by moving the page and the talk page separately.

That is very interesting - do you have a way to reliably reproduce the error? Can you set up some "immovable" sandbox page that can be used to investigate this? We have so far been unable to reproduce the issue in a testing environment. Being able to reproduce it on the life wiki, somewhere were it's not bothering anyone, would perhaps allow us to find the issue.

No, I have not been able to. I even went so far as copying the exact page and talk page into another page and talk page (same namespace, so article to article) and it still did not trigger the error for the second page. It is so weird.

I could get you a list of a few pages on fawiki that are impacted, and make you a local admin temporarily (or you may be able to assign yourself a higher-admin right like @Urbanecm did). I know production testing is a last resort, but in absence of reproducibility, we may need to rely on this last resort.

In T279832#7027441, @daniel wrote:

In T279832#7013970, @Huji wrote:

In above comments, we found some evidence that both the page being moved and its talk page played a role in this issue raising. We also found that users can work around this bug by moving the page and the talk page separately.

That is very interesting - do you have a way to reliably reproduce the error? Can you set up some "immovable" sandbox page that can be used to investigate this? We have so far been unable to reproduce the issue in a testing environment. Being able to reproduce it on the life wiki, somewhere were it's not bothering anyone, would perhaps allow us to find the issue.

Moving https://fa.wikipedia.org/w/index.php?title=%D9%81%D9%87%D8%B1%D8%B3%D8%AA_%D8%A7%D9%87%D8%A7%D9%84%DB%8C_%D8%A8%D8%A7%D8%A8%D9%84&redirect=no should fail reliably (unless you do some livehacking; so far, disabling scirbunto on a debug server overruled the error, see above), but be warned it is a real fawiki page. I failed to reproduce it locally or intentionally break a page on fawiki so far :/. I just tried to XML import the page, and moving that new page works as I would expect :/.

I spent a bit of time on it and people can move pages in fawiki now. If your wiki is affected please do the similar thing I did. This bug looks like an onion. So many layers.

TLDR: this edit fixed the issue in fawiki. It's not a great fix but at least enables people to move pages.

• First of all, this failure is causing tiny data corruption (or major cache pollution, depends on how you see it). Going back to the failed move, the templates used in the page are:

• The traceback says getContent lua function is the call that breaks the whole thing (in Module:Redirect). I need to spend some time to figure out why Module:Redirect works in enwiki but not in fawiki (and some other wikis).
• The error seems to come from a huge misconception. Reading from master doesn't mean you'll be reading the latest data. Our databases are on REPEATABLE_READ lock, meaning if something changes and you have queried master before, you're going to get stale data. Specially since this is happening in a deferred update, outside of context of the main commit, at the same time with other racing deferred update. Unfortunately, the way DAO flags (IDBAccessObject interface) work, doesn't let me to set connection flag to CONN_TRX_AUTOCOMMIT so it'll look at it outside of the commit context (and possibly get fresh data).
• I'm slightly hesitant on why we even have a double check on this (RevisionStore::ensureRevisionRowMatchesTitle) unless we have had major data corruption incident that this was made to protect against. This seems to be doing a lot of extra query in a wrong way and has caused issues before too: T246720: Unable to save edit due to "The given Title does not belong to page" error (Wikipedias with FlaggedRevs enabled). It has some assumptions that I find questionable.

HTH

@Ladsgroup we already knew that portion of the {{R from move}} template which you removed in your edit was the cause for this issue to manifest. But that is not the root cause.

By editing the template in the way you did, you have created a workaround for fawiki moves to not fail, but you have not solved the problem. Also, you have made it impossible to reproduce this problem (we had some known cases which now will not cause a failure and cannot be studied). Finally, you have taken away a useful functionality of that template.

Therefore, respectfully, I am going to revert your edit on that template. There is a bigger underlying issue here that needs to be found and fixed.

I explained the underlying problem in depth below, we know what's causing this issue and letting users move pages is more important than some tracking categories. You can revert it once the underlying issue I described is resolved.

Let me put it this way: if you can replicate this in another wiki, I am good with your modification to the template. Otherwise, your edit makes the issue non-testable and will significantly delay its fix, and also takes away useful features on wiki. Both of those are bad things.

I understand but:

• it's still possible to test it in other wikis. Devs can see the list of fatals in logstash which are constantly happening.
• harming a wiki for sake of reproducibility is not nice. I see at least hundreds of attempts leading to fatal in fawiki in logstash every day. This is putting devs convenience front of our editors.
• this also spams our error logs causing noise on stuff that might sneak in and cause damage, like data corruption. It can also trigger alerts for SRE. We don't keep error logs for the sake of reproducibility. I can point you to some documentation on why we need to keep our error logs clean as possible.
• you very very likely can reproduce it in beta cluster or test.wikipedia.org by copying the Lua modules I mentioned.
• there's not much need to reproduce the error. I explained the underlying problem. The REPEATABLE_READ lock.

In T279832#7031513, @Ladsgroup wrote:

I understand but:

• it's still possible to test it in other wikis. Devs can see the list of fatals in logstash which are constantly happening.

You suggested a similar change should be performed to template on those wikis. Quote: "If your wiki is affected please do the similar thing I did." Are you saying fawiki should work around this ASAP, and other wikis should assume the burden of keeping this testable?

• harming a wiki for sake of reproducibility is not nice. I see at least hundreds of attempts leading to fatal in fawiki in logstash every day. This is putting devs convenience front of our editors.

Totally agree. Which is why the first order of work should be to reproduce this on a non-production wiki. As soon as that happens, I am in full agreement to implement temporary workarounds on the production wikis until we fix the issue.

• this also spams our error logs causing noise on stuff that might sneak in and cause damage, like data corruption. It can also trigger alerts for SRE. We don't keep error logs for the sake of reproducibility. I can point you to some documentation on why we need to keep our error logs clean as possible.

If it has so many negative impacts, then why on earth did we reduce it from UBN to high?

• you very very likely can reproduce it in beta cluster or test.wikipedia.org by copying the Lua modules I mentioned.

I tried it on my local wiki and was not successful.

Even more: I copied a page and its talk page to a new title on fawiki, and despite having the exact same content and using the exact same templates and modules, the new page did not fail to move while the original one did.

If you are "very very" confident it is reproducible on test or beta, do us a favor and reproduce it please :)

• there's not much need to reproduce the error. I explained the underlying problem. The REPEATABLE_READ lock.

If you are so confident that is the issue, then please offer a fix. If not able to, then let others do so. Do not sweep this under the rug by a template edit on fawiki though. The issue is still there, isn't it?

I summary: I think your intention was right, and you have provided valuable information here through your analysis, but I take that you have not been able to reproduce this, and while you have a theory about the root cause, you have not been able to verify it or offer a fix for it. Unless any of those circumstances change, I think we should let someone else who is more capable than me and you try and take us to the next step.

Random thought on the side: could this be related to the fact that LinkCache puts meta-data about pages in certain namespaces (e.g. NS_TEMPLATE and NS_MODULE ) into WANObject cache? The way that cache is kept up to date seems manual and error prone, so perhaps we hit a stale entry in that cache, which has a bad title/page association?

It seems like MovePage doesn't directly invalidate that cache, the way we do on article create, edit, and delete.

In T279832#7031214, @Ladsgroup wrote:

• The error seems to come from a huge misconception. Reading from master doesn't mean you'll be reading the latest data. Our databases are on REPEATABLE_READ lock, meaning if something changes and you have queried master before, you're going to get stale data.

My understanding of REPEATABLE_READ is that you may get stale data if something changed concurrently via another DB connection. You should get the latest info according to all changes made by the same request via the same connection.

[EDIT: I confirmed this by playing with two mysql clients in a test database: a connection always sees its own writes, but may not see another connection's writes until both have committed the current transaction]

Specially since this is happening in a deferred update, outside of context of the main commit, at the same time with other racing deferred update.

Deferred updates from the same request don't execute concurrently, the execute in the order they were enqueued, inside the same request. If the main transaction was committed successfully, I don't see how the deferred update could get stale data.

What might be happening though is that the main transaction is failing (due to "RevisionAccessException: Revision NNNN belongs to page ID MMMM"), but the deferred update was already scheduled. When the deferred update tries to run on the page we just tried (but failed) to create, it dies because it can't find that page.

Change 683242 had a related patch set uploaded (by Daniel Kinzler; author: Daniel Kinzler):

[mediawiki/core@master] RevisionStore: don't die on mismatching Titles

https://gerrit.wikimedia.org/r/683242

The patch above doesn't fix the underlying problem, but it should allow pages moves to function.

The failure in LinksUpdate can probably be fixed by T281340: DeferredUpdates should not be enqueued while inside a database transaction..

Change 683242 merged by jenkins-bot:

[mediawiki/core@master] RevisionStore: don't die on mismatching Titles

https://gerrit.wikimedia.org/r/683242

I get a very similar error message when suppress deleting a page. Should I raise a separate ticket?

2021-04-30 14:19:41 [YIwR9I-Al3g08XtZ2MEmsAAAAAY] deployment-jobrunner04 enwiki 1.37.0-alpha exception ERROR: [YIwR9I-Al3g08XtZ2MEmsAAAAAY] /rpc/RunSingleJob.php   InvalidArgumentException: The Title object yields no ID. Perhaps the page [[Asian_black_bear]] doesn't exist? {"exception_url":"/rpc/RunSingleJob.php","reqId":"YIwR9I-Al3g08XtZ2MEmsAAAAAY","caught_by":"other"} 
[Exception InvalidArgumentException] (/srv/mediawiki/php-master/includes/deferred/LinksUpdate.php:140) The Title object yields no ID. Perhaps the page [[Asian_black_bear]] doesn't exist?
  #0 /srv/mediawiki/php-master/includes/Storage/DerivedPageDataUpdater.php(1390): LinksUpdate->__construct(Title, ParserOutput, boolean)
  #1 /srv/mediawiki/php-master/includes/deferred/RefreshSecondaryDataUpdate.php(83): MediaWiki\Storage\DerivedPageDataUpdater->getSecondaryDataUpdates(boolean)
  #2 /srv/mediawiki/php-master/includes/deferred/DeferredUpdates.php(513): RefreshSecondaryDataUpdate->doUpdate()
  #3 /srv/mediawiki/php-master/includes/Storage/DerivedPageDataUpdater.php(1722): DeferredUpdates::attemptUpdate(RefreshSecondaryDataUpdate, Wikimedia\Rdbms\LBFactoryMulti)
  #4 /srv/mediawiki/php-master/includes/page/WikiPage.php(2435): MediaWiki\Storage\DerivedPageDataUpdater->doSecondaryDataUpdates(array)
  #5 /srv/mediawiki/php-master/includes/jobqueue/jobs/RefreshLinksJob.php(190): WikiPage->doSecondaryDataUpdates(array)
  #6 /srv/mediawiki/php-master/includes/jobqueue/jobs/RefreshLinksJob.php(126): RefreshLinksJob->runForTitle(Title)
  #7 /srv/mediawiki/php-master/extensions/EventBus/includes/JobExecutor.php(79): RefreshLinksJob->run()
  #8 /srv/mediawiki/rpc/RunSingleJob.php(76): MediaWiki\Extension\EventBus\JobExecutor->execute(array)
  #9 {main}

I would make a separate task, because this is a completely separate workflow.

I would consider filing it as a child for T281340. Perhaps this task (T279832) should also be made a child of T281340.

In T279832#7027557, @Urbanecm wrote:

Moving https://fa.wikipedia.org/w/index.php?title=%D9%81%D9%87%D8%B1%D8%B3%D8%AA_%D8%A7%D9%87%D8%A7%D9%84%DB%8C_%D8%A8%D8%A7%D8%A8%D9%84&redirect=no should fail reliably (unless you do some livehacking; so far, disabling scirbunto on a debug server overruled the error, see above), but be warned it is a real fawiki page. I failed to reproduce it locally or intentionally break a page on fawiki so far :/. I just tried to XML import the page, and moving that new page works as I would expect :/.

I just confirmed that it is still impossible to move that page, see reqId 60f88a1e-fa26-4bb7-9899-8e6d113f5847 at May 4, 2021 @ 09:59:52.

I plan to investigate this on the live system today, with the help of @Ladsgroup. Let's hope we can find out what's going on.

Change 684967 had a related patch set uploaded (by Daniel Kinzler; author: Daniel Kinzler):

[mediawiki/core@master] MovePage: clear title cache before parsing new redirect.

https://gerrit.wikimedia.org/r/684967

Change 684967 merged by jenkins-bot:

[mediawiki/core@master] MovePage: clear title cache before parsing new redirect.

https://gerrit.wikimedia.org/r/684967

@daniel the patch you proposed was merged on 5/4 so I assume it will hit production on Monday; is that correct? Just want to make sure when we can test it more broadly in production and (hopefully) close this task.