Page MenuHomePhabricator

Error in Flow page move
Closed, ResolvedPublic

Description

2016-02-12 00:55:06 mw1081 enwiki 1.27.0-wmf.13 exception ERROR: [de946867] /w/index.php?title=Special:MovePage&action=submit Flow\Exception\DataModelException from line 156 of /srv/mediawiki/php-1.27.0-wmf.13/extensions/Flow/includes/Data/Index/TopKIndex.php: Unable to find specified offset in query results {"exception_id":"de946867"}
[Exception Flow\Exception\DataModelException] (/srv/mediawiki/php-1.27.0-wmf.13/extensions/Flow/includes/Data/Index/TopKIndex.php:156) Unable to find specified offset in query results
#0 /srv/mediawiki/php-1.27.0-wmf.13/extensions/Flow/includes/Data/Index/TopKIndex.php(104): Flow\Data\Index\TopKIndex->getOffsetFromOffsetValue(array, string)


mattflaschen@fluorine:/a/mw-log$ grep -A30 de946867 exception.log 
2016-02-12 00:55:06 mw1081 enwiki 1.27.0-wmf.13 exception ERROR: [de946867] /w/index.php?title=Special:MovePage&action=submit   Flow\Exception\DataModelException from line 156 of /srv/mediawiki/php-1.27.0-wmf.13/extensions/Flow/includes/Data/Index/TopKIndex.php: Unable to find specified offset in query results {"exception_id":"de946867"} 
[Exception Flow\Exception\DataModelException] (/srv/mediawiki/php-1.27.0-wmf.13/extensions/Flow/includes/Data/Index/TopKIndex.php:156) Unable to find specified offset in query results
  #0 /srv/mediawiki/php-1.27.0-wmf.13/extensions/Flow/includes/Data/Index/TopKIndex.php(104): Flow\Data\Index\TopKIndex->getOffsetFromOffsetValue(array, string)
  #1 /srv/mediawiki/php-1.27.0-wmf.13/extensions/Flow/includes/Data/Index/TopKIndex.php(70): Flow\Data\Index\TopKIndex->getOffsetLimit(array, array)
  #2 /srv/mediawiki/php-1.27.0-wmf.13/extensions/Flow/includes/Data/Index/FeatureIndex.php(311): Flow\Data\Index\TopKIndex->filterResults(array, array)
  #3 /srv/mediawiki/php-1.27.0-wmf.13/extensions/Flow/includes/Data/ObjectLocator.php(89): Flow\Data\Index\FeatureIndex->findMulti(array, array)
  #4 /srv/mediawiki/php-1.27.0-wmf.13/extensions/Flow/includes/Data/ObjectLocator.php(64): Flow\Data\ObjectLocator->findMulti(array, array)
  #5 /srv/mediawiki/php-1.27.0-wmf.13/extensions/Flow/includes/Data/Pager/Pager.php(130): Flow\Data\ObjectLocator->find(array, array)
  #6 /srv/mediawiki/php-1.27.0-wmf.13/extensions/Flow/includes/Block/TopicList.php(479): Flow\Data\Pager\Pager->getPage(Closure$Flow\Block\TopicListBlock::getPage;2050032642)
  #7 /srv/mediawiki/php-1.27.0-wmf.13/extensions/Flow/includes/Block/TopicList.php(266): Flow\Block\TopicListBlock->getPage(array)
  #8 /srv/mediawiki/php-1.27.0-wmf.13/extensions/Flow/includes/View.php(191): Flow\Block\TopicListBlock->renderApi(array)
  #9 /srv/mediawiki/php-1.27.0-wmf.13/extensions/Flow/includes/View.php(67): Flow\View->buildApiResponse(Flow\WorkflowLoader, array, string, array)
  #10 /srv/mediawiki/php-1.27.0-wmf.13/extensions/Flow/includes/Content/BoardContent.php(217): Flow\View->show(Flow\WorkflowLoader, string)
  #11 /srv/mediawiki/php-1.27.0-wmf.13/extensions/Flow/includes/Content/BoardContent.php(168): Flow\Content\BoardContent->generateHtml(Title, User)
  #12 /srv/mediawiki/php-1.27.0-wmf.13/includes/page/WikiPage.php(2172): Flow\Content\BoardContent->getParserOutput(Title, integer, ParserOptions)
  #13 /srv/mediawiki/php-1.27.0-wmf.13/includes/page/WikiPage.php(2231): WikiPage->prepareContentForEdit(Flow\Content\BoardContent, Revision, User)
  #14 /srv/mediawiki/php-1.27.0-wmf.13/includes/MovePage.php(522): WikiPage->doEditUpdates(Revision, User, array)
  #15 /srv/mediawiki/php-1.27.0-wmf.13/includes/MovePage.php(255): MovePage->moveToInternal(User, Title, string, boolean)
  #16 /srv/mediawiki/php-1.27.0-wmf.13/includes/specials/SpecialMovepage.php(591): MovePage->move(User, string, boolean)
  #17 /srv/mediawiki/php-1.27.0-wmf.13/includes/specials/SpecialMovepage.php(128): MovePageForm->doSubmit()
  #18 /srv/mediawiki/php-1.27.0-wmf.13/includes/specialpage/SpecialPage.php(407): MovePageForm->execute(NULL)
  #19 /srv/mediawiki/php-1.27.0-wmf.13/includes/specialpage/SpecialPageFactory.php(565): SpecialPage->run(NULL)
  #20 /srv/mediawiki/php-1.27.0-wmf.13/includes/MediaWiki.php(280): SpecialPageFactory::executePath(Title, RequestContext)
  #21 /srv/mediawiki/php-1.27.0-wmf.13/includes/MediaWiki.php(736): MediaWiki->performRequest()
  #22 /srv/mediawiki/php-1.27.0-wmf.13/includes/MediaWiki.php(517): MediaWiki->main()
  #23 /srv/mediawiki/php-1.27.0-wmf.13/index.php(43): MediaWiki->run()
  #24 /srv/mediawiki/w/index.php(3): include(string)
  #25 {main}

Event Timeline

Mattflaschen-WMF claimed this task.
Mattflaschen-WMF raised the priority of this task from to Needs Triage.
Mattflaschen-WMF updated the task description. (Show Details)
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptFeb 12 2016, 1:05 AM
Mattflaschen-WMF set Security to None.

generateHtml is in the stacktrace, but when I try to isolate it to that and use the old page, it succeeds:

$oldTitle = Title::newFromText( 'Wikipedia talk:WikiProject Breakfast' );
$user = User::newFromName( 'Roan Kattouw (WMF)' );

$boardContentRevision = Revision::newFromTitle( $oldTitle );
$boardContent = $boardContentRevision->getContent( Revision::RAW );

$generateHtmlMethod = new ReflectionMethod( 'Flow\Content\BoardContent::generateHtml' );
$generateHtmlMethod->setAccessible( true );

$oldTitleParserOutput = $generateHtmlMethod->invoke( $boardContent, $oldTitle, $user );
echo substr( $oldTitleParserOutput->getText(), 0, 100 );
Quiddity added a subscriber: Quiddity.

I haven't figured it out yet.

Tomorrow, I would like to try debugging through mwrepl with FauxRequst and API:Move. (I couldn't recall using the move API with Flow before, but I just did a move test on Beta with ApiSandbox, and it works fine.).

I will either need someone's account to use from the shell, or for my account to get flow-create-board (temporarily is okay) so I can use it.

Notes below:


First, we can try working around it be doing a Flow action=purge, but I don't know if that will work (since it seems fine on the old page name). It might, but it also means we can't troubleshoot whatever the problematic values in MC are. So holding off on that for now, in case someone (i.e. @matthiasmullie) has a better idea.

It looks like the problematic index is either 'storage.topic_list.indexes.last_updated' or 'storage.topic_list.indexes.reverse_lookup'. When I debug through as Roan's user, it's last_updated, but it also doesn't fail when I debug through...

Ultimately, what it's doing is re-rendering the page at the destination. https://en.wikipedia.org/w/api.php?action=parse&page=Wikipedia_talk:WikiProject_Breakfast works, but it's probably pulling that out of parser cache.

We could try API:Purge to force a re-render (without moving the page) to see if that makes the error reproducible (I don't think a regular purge would do it, since I think Flow intercepts that to only purge the Flow memcached). But I don't know why that would do something different from generateHtml above.

@Quiddity and I decided to defer this to Tuesday. I've been using variants of:

$username = 'Roan Kattouw (WMF)';
$user = User::newFromName( $username );

// Get move token
$moveToken = $user->getEditToken();

$moveParams = array( 'action' => 'move', 'from' => 'Wikipedia talk:WikiProject Breakfast', 'to' => 'Wikipedia talk:WikiProject Breakfast/Flow archive', 'reason' => 'Moving to subpage. The converted page will be relocated here, shortly.', 'token' => $moveToken );

RequestContext::getMain()->setUser( $user );
$GLOBALS['wgUser'] = $user;

$derivativeRequest = new DerivativeRequest( $wgRequest, $moveParams, true );

// b ApiMain::__construct()
$api = new ApiMain( $derivativeRequest, true );
$api->getContext()->setUser( $user );

// b ApiMain::execute()
b ApiMove::movePage()

$api->execute();

to debug through.

I've reproduced and understand the issue in production now.

It's actually not an issue with the data stored in memcached (it still happens even after it expires and re-fetches from the DB).

It basically is trying to get the 11 (one extra for pagination purposes) most recently updated topics. However, some of those are moderated and can't be displayed. So it needs to get all the ones from the first 11 that can be displayed, then keep fetching until it gets 11 it can display, or runs out.

However, it's not doing the offset part (start looking for topics updated earlier than X, since I already have X and later) correctly.

I'm working on setting up the same situation locally.

The root cause was that WorkflowTopicListListener (which is triggered by the page ID fix in BoardMover) queried for the TopicListEntry via TopicListStorage. That actually does reads normally, which means last updated isn't in the query result because it's not in the table.

Later on, we do a query that does get back the last updated data, but CachingObjectMapper effectively discards it by remembering the object without last updated. This does not trigger on normal views because the wrong data is never cached by the mapper.

It also doesn't trigger on all moves, only if the offset is needed.

Change 271190 had a related patch set uploaded (by Mattflaschen):
Switch to BasicObjectMapper for TopicListEntry

https://gerrit.wikimedia.org/r/271190

Mattflaschen-WMF added a comment.EditedFeb 17 2016, 4:13 AM

Can be reproduced by having 3+ deleted topics on the first page in the 'recently active' ordering, then moving the board.

Change 271190 merged by jenkins-bot:
Switch to BasicObjectMapper for TopicListEntry

https://gerrit.wikimedia.org/r/271190

Change 271327 had a related patch set uploaded (by Mattflaschen):
Switch to BasicObjectMapper for TopicListEntry

https://gerrit.wikimedia.org/r/271327

Change 271327 merged by jenkins-bot:
Switch to BasicObjectMapper for TopicListEntry

https://gerrit.wikimedia.org/r/271327

Catrope closed this task as Resolved.Feb 19 2016, 10:51 PM