Page MenuHomePhabricator

Flow: InvalidDataException "Flow workflow is for different page" from WorkflowLoaderFactory.php
Open, NormalPublic

Description

Matches for "WorkflowLoaderFactory" in the last 30 days (as of 13 July 2018) in type:mediawiki channel:exception

/srv/mediawiki/php-1.32.0-wmf.8/extensions/Flow/includes/WorkflowLoaderFactory.php:13019
/srv/mediawiki/php-1.32.0-wmf.10/extensions/Flow/includes/WorkflowLoaderFactory.php:13063
/srv/mediawiki/php-1.32.0-wmf.12/extensions/Flow/includes/WorkflowLoaderFactory.php:13011

Errors were seen on a wide range of web servers and wikis, including during JobQueue execution.

Two samples:

server:  ar.wikipedia.org
reqId:   W0lMLwpAICoAABkaQ60AAACF
method:  GET
url:     /wiki/<a page in the main namespace>

exception.message: Flow workflow is for different page
exception.class:   Flow\Exception\InvalidDataException
exception.code:    different-page
exception.file:    /srv/mediawiki/php-1.32.0-wmf.12/extensions/Flow/includes/WorkflowLoaderFactory.php:130
exception.trace:
#0 /srv/mediawiki/php-1.32.0-wmf.12/extensions/Flow/includes/WorkflowLoaderFactory.php(103): Flow\WorkflowLoaderFactory->loadWorkflowById(Title, Flow\Model\UUID)
#1 /srv/mediawiki/php-1.32.0-wmf.12/extensions/Flow/includes/Actions/Action.php(105): Flow\WorkflowLoaderFactory->createWorkflowLoader(Title)
#2 /srv/mediawiki/php-1.32.0-wmf.12/extensions/Flow/includes/Actions/ViewAction.php(20): Flow\Actions\FlowAction->showForAction(string, OutputPage)
#3 /srv/mediawiki/php-1.32.0-wmf.12/extensions/Flow/includes/Actions/Action.php(50): Flow\Actions\ViewAction->showForAction(string)
#4 /srv/mediawiki/php-1.32.0-wmf.12/includes/MediaWiki.php(500): Flow\Actions\FlowAction->show()
#5 /srv/mediawiki/php-1.32.0-wmf.12/includes/MediaWiki.php(294): MediaWiki->performAction(Article, Title)
#6 /srv/mediawiki/php-1.32.0-wmf.12/includes/MediaWiki.php(867): MediaWiki->performRequest()
#7 /srv/mediawiki/php-1.32.0-wmf.12/includes/MediaWiki.php(524): MediaWiki->main()
#8 /srv/mediawiki/php-1.32.0-wmf.12/index.php(42): MediaWiki->run()
#9 /srv/mediawiki/w/index.php(3): include(string)
#10 {main}
server: jobrunner.discovery.wmnet
host: mw1301
reqId: WzrM@wpAIDQAAAVN3v4AAADR
url: /rpc/RunSingleJob.php
wiki: frwiki

exception.class	       	Flow\Exception\InvalidDataException
exception.code	       	different-page
exception.file	       	/srv/mediawiki/php-1.32.0-wmf.10/extensions/Flow/includes/WorkflowLoaderFactory.php:130
exception.message	Flow workflow is for different page
exception.trace:
#0 /srv/mediawiki/php-1.32.0-wmf.10/extensions/Flow/includes/WorkflowLoaderFactory.php(103): Flow\WorkflowLoaderFactory->loadWorkflowById(Title, Flow\Model\UUID)
#1 /srv/mediawiki/php-1.32.0-wmf.10/extensions/Flow/includes/Content/BoardContent.php(245): Flow\WorkflowLoaderFactory->createWorkflowLoader(Title, Flow\Model\UUID)
#2 /srv/mediawiki/php-1.32.0-wmf.10/extensions/Flow/includes/Content/BoardContent.php(223): Flow\Content\BoardContent->getWorkflowLoader(Title)
#3 /srv/mediawiki/php-1.32.0-wmf.10/extensions/Flow/includes/Content/BoardContent.php(173): Flow\Content\BoardContent->generateHtml(Title, User)
#4 /srv/mediawiki/php-1.32.0-wmf.10/includes/content/ContentHandler.php(1324): Flow\Content\BoardContent->getParserOutput(Title, integer, ParserOptions)
#5 /srv/mediawiki/php-1.32.0-wmf.10/extensions/CirrusSearch/includes/Updater.php(340): ContentHandler->getParserOutputForIndexing(WikiPage, ParserCache)
#6 /srv/mediawiki/php-1.32.0-wmf.10/extensions/CirrusSearch/includes/Updater.php(396): CirrusSearch\Updater::buildDocument(CirrusSearch, WikiPage, CirrusSearch\Connection, integer, integer, integer)
#7 /srv/mediawiki/php-1.32.0-wmf.10/extensions/CirrusSearch/includes/Updater.php(204): CirrusSearch\Updater->buildDocumentsForPages(array, integer)
#8 /srv/mediawiki/php-1.32.0-wmf.10/extensions/CirrusSearch/includes/Updater.php(83): CirrusSearch\Updater->updatePages(array, integer)
#9 /srv/mediawiki/php-1.32.0-wmf.10/extensions/CirrusSearch/includes/Job/LinksUpdate.php(52): CirrusSearch\Updater->updateFromTitle(Title)
#10 /srv/mediawiki/php-1.32.0-wmf.10/extensions/CirrusSearch/includes/Job/Job.php(98): CirrusSearch\Job\LinksUpdate->doJob()
#11 /srv/mediawiki/php-1.32.0-wmf.10/extensions/EventBus/includes/JobExecutor.php(67): CirrusSearch\Job\Job->run()
#12 /srv/mediawiki/rpc/RunSingleJob.php(80): JobExecutor->execute(array)
#13 {main}

Details

Reference
bz68526

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Quiddity created this task.Jul 24 2014, 7:15 PM

I pressed "Reply" on a new post, and received this pink error box:
Exception Caught: Flow workflow is for different page

webconsole says:

"Exception Caught: Flow workflow is for different page" load.php:164
console.trace(): load.php:164
mw.log</log.warn() load.php:164
FlowBoardComponent.UI.showError() load.php:42
FlowBoardComponent.UI.events.interactiveHandlers.apiRequest/<() load.php:36
jQuery.Callbacks/fire() load.php:44
jQuery.Callbacks/self.fireWith() load.php:46
.Deferred/</deferred[tuple[0]]() load.php:47
jQuery.Callbacks/fire() load.php:44
jQuery.Callbacks/self.fireWith() load.php:46
.Deferred/</deferred[tuple[0]]() load.php:47
jQuery.Callbacks/fire() load.php:44
jQuery.Callbacks/self.fireWith() load.php:46
.Deferred/</deferred[tuple[0]]() load.php:47
mw.Api.prototype.ajax/xhr<() load.php:28
jQuery.Callbacks/fire() load.php:44
jQuery.Callbacks/self.fireWith() load.php:46
done() load.php:135
.send/callback() load.php:141

(Sorry, a misclick created an empty bug)

Created attachment 16034
screenshot

Attached:

bsitu wrote:

Is this reproducable? You may have some old browser cache, the topic link in the screenshot has old format: ?workflow=xxxxxxxx

I just got the same error, whilst trying to edit the board-header at https://en.wikipedia.org/wiki/Wikipedia_talk:Flow/Developer_test_page

Reproducible, I can't edit the header.

Identical output in webconsole, except for line numbers.

.....

I then tried editing the header at https://www.mediawiki.org/wiki/Talk:Sandbox
and got the same error.

On the IRC recentchanges feed, this popped up:
<rc-pmtpa> [[Talk:Sandbox]] ! http://www.mediawiki.org/w/index.php?title=Talk:Sandbox&action=compare-header-revisions&header_newRevision=rzjscc80i0rwkggx * Quiddity (WMF) * (+6) edited the header

but that link leads to an error:
Could not find a revision to load flow content.

Return to MediaWiki.
[43d717ac] 2014-07-31 18:08:42: Fatal exception of type Flow\Exception\InvalidInputException

and there is nothing to match it in the onwiki Board-history, or the onwiki recentchanges feed.

I tried again, and the edit was saved successfully.

....

But, I cannot do the same at enwiki.

I see these entries in the IRC feed:

<rc-pmtpa> [[Wikipedia talk:Flow/Developer test page]] http://en.wikipedia.org/w/index.php?title=Wikipedia_talk:Flow/Developer_test_page&action=compare-header-revisions&header_newRevision=rzjt42xhnqzlhdz5 * Quiddity (WMF) * (-10) edited the header

and clicking them leads to an equivalent error page:

Could not find a revision to load flow content.

Return to Main Page.
[e4d9a9fa] 2014-07-31 18:27:24: Fatal exception of type Flow\Exception\InvalidInputException

The referenced page headers are both editable now.

As for the IRC feed, it looks like we have a bug where the IRC lines go out before the database transaction is commited. I've put up https://gerrit.wikimedia.org/r/155438 to deal with that issue.

The other issue i noticed here was bad data in a related WikiReference from before we correctly handled relative titles from parsoid. It was a single reference so I updated it from eval.php and everything seems to be working again.

Neither of address the original issue, but I can't seem to reproduce it after fixing the above issue.

(In reply to Erik Bernhardson from comment #6)

Neither of address the original issue, but I can't seem to reproduce it
after fixing the above issue.

ok, I can't reproduce currently either, so will reduce importance, and mark unconfirmed.
I'm glad some orthogonal fixes came out of it, at least! :)

Quiddity removed a subscriber: Maryana.Dec 19 2014, 1:42 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 28 2015, 1:38 AM
Krinkle renamed this task from Flow: Exception Caught: Flow workflow is for different page to Flow: InvalidDataException "Flow workflow is for different page" from WorkflowLoaderFactory.php.Jul 14 2018, 4:32 AM
Krinkle updated the task description. (Show Details)
Krinkle removed a subscriber: wikibugs-l-list.
Restricted Application added a project: Growth-Team. · View Herald TranscriptJul 14 2018, 4:32 AM
Krinkle updated the task description. (Show Details)
JTannerWMF moved this task from Inbox to To Triage on the Growth-Team board.Jul 18 2018, 6:32 PM

I've updated the task description with more recent data from Logstash. This caused 96 fatal errors in last 30 days.

I've updated the task description with more recent data from Logstash. This caused 96 fatal errors in last 30 days.

I've been causing a certain amount of them while troubleshooting T198771, which is one of many materialization of this problem.

JTannerWMF moved this task from To Triage to Q1 2018-19 on the Growth-Team board.Aug 1 2018, 6:03 PM
JTannerWMF edited projects, added Growth-Team (Current Sprint); removed Growth-Team.
JTannerWMF assigned this task to kostajh.

Here's the query I'm looking at. From this I see two manifestations of the exception:

  1. The exception occurs temporarily. This is because the Flow database is external from Mediawiki's, so they can temporarily be out of sync. To reproduce this, you can 1) go to test.wikipedia.org or el.wikipedia.org, go to your preferences -> beta and enable Flow, 2) visit your talk page in another tab, 3) in the tab with preferences disable Flow and click save, 4) quickly reload the tab with your talk page open. User talk page example, non user talk page example
  2. The exception occurs in the context of a user ban. From reviewing the logs for both users (zhwiki, elwiki) we can see that the exception pops up after the ban is instated. My guess is that the order of operations is something like 1) user is banned, 2) user goes to their preferences to turn off their Flow enabled talk page, 3) a bug in Flow/core relating to the user being banned results in the mediawiki DB not getting updated properly, so that their talk page is still considered by core to be a Flow page when it shouldn't, and the Flow DB has been updated so that the workflow can't be loaded.

While both manifestations are similar in that database synchronization plays a role, the first manifestation corrects itself without any intervention, whereas the second manifestation has resulted in two broken user talk pages. I'll start investigating the second manifestation and see if I have any ideas about how to fix the DB synchronization challenge as well.

Quiddity edited subscribers, added: Trizek-WMF; removed: Quiddity, Spage.Aug 23 2018, 8:09 PM
Etonkovidova added a subscriber: Etonkovidova.EditedAug 24 2018, 7:40 PM

@kostajh
I did some more testing on testwiki - the first manifestation, as you mention in your above comment, is still quite easily reproducible when enabling/disabling 'Structured discussion...' beta feature.
e.g. the error in logstash:

 2018-08-24T19:14:11	/wiki/User_talk:Uma17
exception.trace	       	#0 /srv/mediawiki/php-1.32.0-wmf.18/extensions/Flow/includes/WorkflowLoaderFactory.php(103): Flow\WorkflowLoaderFactory->loadWorkflowById(Title, Flow\Model\UUID)
#1 /srv/mediawiki/php-1.32.0-wmf.18/extensions/Flow/includes/Actions/Action.php(105): Flow\WorkflowLoaderFactory->createWorkflowLoader(Title)
#2 /srv/mediawiki/php-1.32.0-wmf.18/extensions/Flow/includes/Actions/ViewAction.php(20): Flow\Actions\FlowAction->showForAction(string, OutputPage)
#3 /srv/mediawiki/php-1.32.0-wmf.18/extensions/Flow/includes/Actions/Action.php(50): Flow\Actions\ViewAction->showForAction(string)
#4 /srv/mediawiki/php-1.32.0-wmf.18/includes/MediaWiki.php(501): Flow\Actions\FlowAction->show()
#5 /srv/mediawiki/php-1.32.0-wmf.18/includes/MediaWiki.php(294): MediaWiki->performAction(Article, Title)
#6 /srv/mediawiki/php-1.32.0-wmf.18/includes/MediaWiki.php(868): MediaWiki->performRequest()
#7 /srv/mediawiki/php-1.32.0-wmf.18/includes/MediaWiki.php(525): MediaWiki->main()
#8 /srv/mediawiki/php-1.32.0-wmf.18/index.php(42): MediaWiki->run()
#9 /srv/mediawiki/w/index.php(3): include(string)
#10 {main}

exception_id	       	W4BZAwpAIEIAAFGcR8EAAABC

There is a small annoying bug - the state of the checkbox for 'Structured discussion...' beta feature is very often out of sync with which format the User talk page is displayed - the page might be in Flow format, but the checkbox is displayed unchecked.

There is a small annoying bug - the state of the checkbox for 'Structured discussion...' beta feature is very often out of sync with which format the User talk page is displayed - the page might be in Flow format, but the checkbox is displayed unchecked.

Interesting, I hadn't noticed that but I'll keep an eye out for it!

Two additional things to note:

  1. Some revisions on the el wiki user's talk page are still broken, so noting that the script can fix the main talk page entry point but not necessarily all revisions
  2. More importantly, I figured out how to reproduce the second manifestation listed above. If (1) an admin marks a user's talk page as protected and editable only by admins, (2) blocks that user [you don't need to specify the block to disallow editing the talk page, just the "account creation" block], (3) the blocked user goes to preferences and switches off Flow, then you get: https://test.wikipedia.org/wiki/User_talk:Test_talk_flow_enabled

This is not exactly the same scenario as what occurred with the users on el and zh wiki, but at least it gives us an idea of how to reproduce it locally which should lead to some clarity on how to fix the problem in various scenarios.

kostajh added a subscriber: aaron.Sep 21 2018, 1:31 AM

Some more notes on this. There are two issues co-occurring here.

The first issue is that disabling a Flow board as a blocked user erroneously results in an ImportException getting thrown due to insufficient permissions: handleSubmit() returns an array of errors, and then OptInController.php#L550 throws the exception. This shouldn't happen because the context passed to handleSubmit is supposed to be the "Flow talk page manager" user, not the user who is enabling/disable preferences.

The second issue is that once the import exception is thrown, we go into the catch block of OptInController#L136 – this is what I see locally anyway. (One oddity is that I can't find instances of "Failed to disable Flow on" in the logs, and that should be there.) The Flow database is supposed to be rolled back, then the master database. What happens in practice is that the master DB is reverted, but the Flow DB is not, leading to the exception noted in this task's description. When I observe this in my development environment, $flowDbw->rollback( $outerMethod ) throws a DBUnexpectedError exception

Looking at the Mediawiki docs on use of transaction rollback, it seems like we should not be calling ->rollback() at all but instead throwing an exception, which should handle rolling back all databases. When I do so locally (throw new \MWException('some message', 0, $t) in the context of the try/catch block in OptInController#L136, that seems to work.

As noted in my comment above, I've been able to reproduce a variation on this error, so one idea would be to try changing the rollback code in the try/catch block for testwiki only, and see if that works.

cc @aaron

Change 461980 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/Flow@master] Use DBLoadBalancerFactory to rollback all DBs, add more logging

https://gerrit.wikimedia.org/r/461980

Change 461980 merged by jenkins-bot:
[mediawiki/extensions/Flow@master] Use DBLoadBalancerFactory to rollback all DBs, add more logging

https://gerrit.wikimedia.org/r/461980

Change 462837 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/Flow@master] Catch Exceptions only for HHVM compatibility

https://gerrit.wikimedia.org/r/462837

There's a separate but related issue about the logs not populating when gathering errors for blocks, will continue looking into that.

Change 462837 merged by jenkins-bot:
[mediawiki/extensions/Flow@master] Catch Exceptions only for HHVM compatibility

https://gerrit.wikimedia.org/r/462837

Summary of where we are at:

  1. The merged patch at https://gerrit.wikimedia.org/r/462837 should correctly handle catching exceptions in HHVM
  2. Locally, I've verified with the hhvm role that exceptions thrown during the OccupationController->disable() method are caught, and also that the flow db and the mediawiki db are rolled back
  3. Locally, I've verified that adding the protectedpagetext and editprotected rights to the Flow talk page manager (I tested by adding this user to the sysop group) would allow users to disable Flow on their talk page even if it's marked protected. However, it seems like that isn't exactly desired behavior – presumably if an admin has marked someone's talk page as protected, they don't want that user to be able to change it at all.
  4. What I'd like to do next is inspect $errors = $workflow->getPermissionErrors( 'edit', $context->getUser(), 'secure' ); via mwrepl on zhwiki or elwiki for the problem pages that we know about.
  5. It still seems like a good idea to check permissions before initiating the opt-in/opt-out process, and provide some information to the user about why they can't opt-in/opt-out. This patch has started on that work although there is more to be done to improve the UX https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Flow/+/455897

Change 463791 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/Flow@master] WIP: Add job for fixing boards when InvalidDataException is thrown

https://gerrit.wikimedia.org/r/463791

Will come back to this on Friday or next week. Once 6394b76d67d57b9708ae7ede1d388152f67c284b is in production, we'll have better insight into the exception that's thrown as well as some more information in the logs. We can also see if rolling back the DBs then works as intended.

Also, here's a debug script to print out the errors (we need to fill in the user ID for users we want to look at): P7615. But let's wait until after the train is in production so that exceptions will be caught properly.

Change 464871 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/Flow@master] Log block errors in OptIn/OptOut process

https://gerrit.wikimedia.org/r/464871

Change 463791 abandoned by Kosta Harlan:
WIP: Add job for fixing boards when InvalidDataException is thrown

Reason:
We can handle these errors on a case by case basis.

https://gerrit.wikimedia.org/r/463791

We have some additional logging in place now, and are properly catching exceptions in the OptInController, so we will come back to this if/when fresh instances of the problem are reported.

Change 464871 merged by jenkins-bot:
[mediawiki/extensions/Flow@master] Log block errors in OptIn/OptOut process

https://gerrit.wikimedia.org/r/464871

Change 455897 had a related patch set uploaded (by Krinkle; owner: Kosta Harlan):
[mediawiki/extensions/Flow@master] WIP: Check permissions before attempting to move talk page

https://gerrit.wikimedia.org/r/455897

Still seen in recent logs. The latest repro case is https://www.wikidata.org/w/index.php?title=User_talk:Seojeongah80&action=history.

[W@JlZApAICwAAFTvFcQAAAAX] 2018-11-07 04:09:08: Fatal exception of type "Flow\Exception\InvalidDataException"

@Krinkle I triggered that one while looking at T201654. I could be wrong but I think that this user's talk page was broken before the various fixes we implemented around this bug were deployed. The user is blocked, so they probably tried to convert their talk page to Flow, an exception was thrown and not caught (since HHVM didn't recognize the catch Error $error syntax.

I can run the script to manually fix this user's page but I don't think it's something that will come up in the future when enabling/disabling flow.

I am leaning towards abandoning https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Flow/+/455897, 1) rollback on exception is working properly now and 2) in the patch, there's no UI to show to the user that we didn't actually enable Flow when the permissions check failed.

Fixed the user's board just now:

INCONSISTENT: Core title for 'uadxde2amz2ypajk' is 'User talk:Seojeongah80', but Flow title is 'User talk:Seojeongah80/Structured Discussions Archive 1'
FIXED: Updated 'uadxde2amz2ypajk' to match core title, 'User talk:Seojeongah80'

Checked a total of 1200 Flow boards.  Of those, 1 boards had an inconsistent title; 1 were fixed.

Change 455897 abandoned by Kosta Harlan:
WIP: Check permissions before attempting to move talk page

https://gerrit.wikimedia.org/r/455897

Recent cases:

  • 2019-01-13 ERROR [XFMPKQpAMFQAAEFAjZQAAACB] mw1249
  • 2019-02-01 ERROR [XFSPxwpAAD8AAFTaB90AAACE] mw1268

From https://www.mediawiki.org/wiki/Topic:050e2fb772abdae40bd7842b2b77d26b, which still reproduces the error when viewed, at time of writing.

From https://www.mediawiki.org/wiki/Topic:050e2fb772abdae40bd7842b2b77d26b, which still reproduces the error when viewed, at time of writing.

I looked at this and other examples, it looks like it's all older content. I ran mwscript extensions/Flow/maintenance/FlowFixInconsistentBoards.php --wiki=mediawikiwiki --dry-run but it didn't pick up anything to be fixed.

I think the options here are either 1) a maintenance script to clean up this older, inconsistent content or 2) an alternative to throwing InvalidDataException in WorkflowLoaderFactory.php#loadWorkflowById, perhaps redirecting the user somewhere.

kostajh removed kostajh as the assignee of this task.Feb 20 2019, 9:10 PM
kostajh added a subscriber: kostajh.

Unassigning myself although I would like to resolve this one someday.