Convert LQT pages on MediaWiki.org to Flow (tracking)
Closed, ResolvedPublic2 Story Points

Description

Plan is at: https://www.mediawiki.org/wiki/Flow/LQT_conversion_process

Each checkpoint is

Day 1 (Mon, April 20):
2 pages
Talk:Echo_(Notifications) and User_talk:Mattflaschen

Day 2 (Tues, April 21):
5 pages
User talk:Werdna, Talk:Flow Portal/Archive2, Extension talk:Flow, Extension talk:WikiLove, Talk:Collaboration

Day 4 (Thurs, April 23):
24 pages
User_talk:Qgil-WMF, Talk:Search/status, Help_talk:CirrusSearch, Talk:Compatibility, Talk:MediaWiki_1.18, Extension talk:CirrusSearch, Extension talk:VisualEditor, Extension talk:Echo, VisualEditor_talk:Welcome, Extension talk:TemplateData, Extension talk:Lucene-search, Extension talk:MobileFrontend, Extension talk:Scribunto, Manual_talk:Installation_requirements, Manual_talk:Interwiki, Extension talk:Contribution Scores, Help_talk:Extension:Translate, Project_talk:WikiProject_Bug_Squad, Talk:Flow_Portal/Interactive_Prototype, Talk:LiquidThreads_3.0/Design, Talk:Athena, Talk:GlobalProfile/design, Talk:Groups/Proposals/Bug_Squad, Talk:Groups/San_Francisco

Day 8 (Mon, April 27)
250 pages
Chosen by the devs, not including user talk if possible

Day 10 (Wed, April 29)
Start automatic conversion of the remainder, except for Support desk and VE/Feedback

Day 15 (Mon, May 4)
Convert Project:Support_desk.

Hold VE/Feedback until we can review with the Community & Editing team. and before converting give a 2-3 days warning to T98465 and T92661 (crossed out part is deferred until Flow and VE both support centralized feedback) let CLs know so that CLs can prepare and send messaging beforehand.

We should use logfile so it's restartable.

Related Objects

StatusAssignedTask
OpenJdforrester-WMF
OpenJdforrester-WMF
OpenNone
OpenNone
ResolvedCatrope
ResolvedMattflaschen-WMF
Resolvedmatthiasmullie
ResolvedEBernhardson
Resolvedmatthiasmullie
ResolvedMattflaschen-WMF
Resolvedmatthiasmullie
ResolvedEBernhardson
ResolvedEBernhardson
ResolvedDannyH
ResolvedEBernhardson
DuplicateMattflaschen-WMF
ResolvedEBernhardson
ResolvedMattflaschen-WMF
ResolvedMattflaschen-WMF
ResolvedMattflaschen-WMF
ResolvedMattflaschen-WMF
ResolvedMattflaschen-WMF
ResolvedNone
ResolvedEBernhardson
ResolvedEBernhardson
ResolvedMattflaschen-WMF
ResolvedMattflaschen-WMF
ResolvedMattflaschen-WMF
ResolvedMattflaschen-WMF
ResolvedMattflaschen-WMF
ResolvedMattflaschen-WMF
ResolvedMattflaschen-WMF
ResolvedCatrope
ResolvedMattflaschen-WMF
ResolvedMattflaschen-WMF
ResolvedMattflaschen-WMF
ResolvedMattflaschen-WMF
ResolvedMattflaschen-WMF
ResolvedMattflaschen-WMF
ResolvedMattflaschen-WMF
ResolvedSBisson
ResolvedSBisson
ResolvedSBisson
There are a very large number of changes, so older changes are hidden. Show Older Changes
Mattflaschen-WMF added a comment.EditedJul 9 2015, 11:42 PM

I put it there when I first put the last blocker (T104087: E8. LQT conversion sends email notifications for every single topic I've got unread notifications for) up for review. Sorry, I see how it was unintentionally confusing.

Mattflaschen-WMF added a comment.EditedJul 10 2015, 6:35 PM

"Day 10" conversion is done. There are a small handful of issues to resolve, which I will work on today.

Grep shows no references to fatal. error, or warning (excluding false positives from page titles).

There are a few exceptions, though (I removed matches from the below grep when it was just the page title). A couple appear to be due to intermittent Parsoid downtime or failures. There are also two separate issues that need to be investigated.

mattflaschen@terbium:~$ grep -i exception ~/2015_07_09_LQT_Flow.log 
[2015-07-10 02:00:28] exception 'Flow\Exception\NoParsoidException' with message 'Failed contacting Parsoid: There was a problem during the HTTP request: 503 Service Unavailable' in /srv/mediawiki/php-1.26wmf13/extensions/Flow/includes/Parsoid/Utils.php:122
[2015-07-10 02:57:40] exception 'Flow\Exception\WikitextException' with message 'Invalid char in CDATA 0x8
[2015-07-10 11:12:20] exception 'Flow\Exception\NoParsoidException' with message 'Failed contacting Parsoid: There was a problem during the HTTP request: 503 Service Unavailable' in /srv/mediawiki/php-1.26wmf13/extensions/Flow/includes/Parsoid/Utils.php:122
[2015-07-10 11:18:45] exception 'Flow\Import\ImportException' with message 'User does not exist: Ecliptica' in /srv/mediawiki/php-1.26wmf13/extensions/Flow/includes/Import/Importer.php:388

There are also a small number that did not run at all due to encoding issues (sorry, should have caught that), or where the LQT archive page is still marked as LQT (seems to be due to "{{#useliquidthreads:01}}"): https://www.mediawiki.org/w/api.php?action=query&generator=pageswithprop&gpwppropname=use-liquid-threads&prop=info&gpwplimit=500&indexpageids

I'll prioritize between these.

Mattflaschen-WMF added a comment.EditedJul 11 2015, 5:17 AM

I made the mistake of trying to revert the two conversions that failed with NoParsoidException, instead of resuming (for one, this might have been simple enough, but the other is really too long for this to be a good option). However, I was able to hack up a resume with P944.

We solved one of the problems (the % in "Talk:Wikimedia engineering 20% policy" was causing the header not to convert), and that page is fully done; see T105562: URLs with % sign are rejected by Parsoid server. (grep -B10 -A29 '11:12:20.*NoParsoid' 2015_07_09_LQT_Flow.log).

However, the other is failing to import this post, and it's not clear why. It has a apostrophe, but I don't think that's relevant since I believe it's passing in the topic title (Topic:bunch of alphadecimal) as the Parsoid context title. It's edited, but there are plenty of edited posts that imported fine (e.g. https://www.mediawiki.org/w/index.php?title=Topic:Sgk7t8qn9eup04iy&action=compare-post-revisions&topic_newRevision=sgk7vcao6y7rqv89).

It works fine with a different context title, so that suggests the title is the issue, but I need to figure out what it is (I think it is deterministic due to HistoricalUIDGenerator:

mattflaschen@terbium:~$ mwscript eval.php --wiki=mediawikiwiki
> $parsoidContextTitle = Title::newFromText( 'MediaWiki' );

> $postTitle = Title::newFromText( "Thread:Extension_talk:Header_Tabs/Header_tabs_don't_show_when_ads_appear_on_webkit_browsers/reply_(3)" );

> $postRevision = Revision::newFromTitle( $postTitle );

> $content = $postRevision->getText( Revision::RAW );

> echo $content;
This is what I got from the inspection element report:

<pre>
Exception thrown by ext.headertabs: Cannot read property 'length' of null load.php?debug=false&lang=en&modules=jquery%2Cmediawiki&only=scripts&skin=monobook&version=20140311…:151
TypeError
message: "Cannot read property 'length' of null"
stack: (...)
get stack: function () { [native code] }
set stack: function () { [native code] }
__proto__: Error
 load.php?debug=false&lang=en&modules=jquery%2Cmediawiki&only=scripts&skin=monobook&version=20140311…:151
log
</pre>
> echo Flow\Parsoid\Utils::convert( 'wikitext', 'html', $content, $parsoidContextTitle );
<body data-parsoid='{"dsr":[0,528,0,0]}' lang="en" class="mw-content-ltr sitedir-ltr ltr mw-body mw-body-content mediawiki" dir="ltr"><p data-parsoid='{"dsr":[0,54,0,0]}'>This is what I got from the inspection element report:</p>

<pre data-parsoid='{"stx":"html","strippedNL":true,"dsr":[56,528,5,6]}'>Exception thrown by ext.headertabs: Cannot read property 'length' of null load.php?debug=false&amp;lang=en&amp;modules=jquery%2Cmediawiki&amp;only=scripts&amp;skin=monobook&amp;version=20140311…:151
TypeError
message: "Cannot read property 'length' of null"
stack: (...)
get stack: function () { [native code] }
set stack: function () { [native code] }
__proto__: Error
 load.php?debug=false&amp;lang=en&amp;modules=jquery%2Cmediawiki&amp;only=scripts&amp;skin=monobook&amp;version=20140311…:151
log
</pre></body>
DannyH updated the task description. (Show Details)Jul 13 2015, 6:46 PM

@DannyH, we said at retro today we're not making centralized feedback a blocker (since mediawiki.messagePoster doesn't yet support it, and we didn't want it to block). That means T98465: Message wikis about upcoming transition to centralized VE feedback and T92661: Plan to redirect inactive VisualEditor feedback pages for some wikis to mediawiki.org are not applicable.

Mattflaschen-WMF updated the task description. (Show Details)

The second one is not caused by the title. I can consistently reproduce the problem (using a script very similar to P944, but with the right titles).

That shows the same error in the log, but now with the topic title:

1​2015-07-14 00:29:08 terbium mediawikiwiki Flow INFO: Flow\Parsoid\Utils::parsoid: Failed contacting Parsoid for title "Topic:Rsmwhyrgc56rq2vb": There was a problem during the HTTP request: 5
2​03 Service Unavailable {"private":false}
3​2015-07-14 00:29:08 terbium mediawikiwiki Flow ERROR: Failed importing topic: topiclqt-api:local:thread_id:41828
4​2015-07-14 00:29:08 terbium mediawikiwiki Flow ERROR: exception 'Flow\Exception\NoParsoidException' with message 'Failed contacting Parsoid for title "Topic:Rsmwhyrgc56rq2vb": There was a pr
5​oblem during the HTTP request: 503 Service Unavailable' in /srv/mediawiki/php-1.26wmf13/extensions/Flow/includes/Parsoid/Utils.php:124
6​Stack trace:
7​#0 /srv/mediawiki/php-1.26wmf13/extensions/Flow/includes/Parsoid/Utils.php(42): Flow\Parsoid\Utils::parsoid('wikitext', 'html', 'This is what I ...', Object(Title))
8​#1 /srv/mediawiki/php-1.26wmf13/extensions/Flow/includes/Model/AbstractRevision.php(465): Flow\Parsoid\Utils::convert('wikitext', 'html', 'This is what I ...', Object(Title))
9​#2 /srv/mediawiki/php-1.26wmf13/extensions/Flow/includes/Model/PostRevision.php(174): Flow\Model\AbstractRevision->setContent('This is what I ...', 'wikitext', Object(Title))
10​#3 /srv/mediawiki/php-1.26wmf13/extensions/Flow/includes/Import/Importer.php(845): Flow\Model\PostRevision->reply(Object(Flow\Model\Workflow), Object(User), 'This is what I ...', 'wikitext')
11​#4 /srv/mediawiki/php-1.26wmf13/extensions/Flow/includes/Import/Importer.php(905): Flow\Import\{closure}(Object(Flow\Import\LiquidThreadsApi\ImportRevision))
12​#5 /srv/mediawiki/php-1.26wmf13/extensions/Flow/includes/Import/Importer.php(850): Flow\Import\TalkpageImportOperation->importObjectWithHistory(Object(Flow\Import\LiquidThreadsApi\ImportPost
13​), Object(Closure), 'edit-post', Object(Flow\Import\PageImportState), Object(Title))
14​#6 /srv/mediawiki/php-1.26wmf13/extensions/Flow/includes/Import/Importer.php(873): Flow\Import\TalkpageImportOperation->importPost(Object(Flow\Import\TopicImportState), Object(Flow\Import\Li
15​quidThreadsApi\ImportPost), Object(Flow\Model\PostRevision), ' ')
16​#7 /srv/mediawiki/php-1.26wmf13/extensions/Flow/includes/Import/Importer.php(873): Flow\Import\TalkpageImportOperation->importPost(Object(Flow\Import\TopicImportState), Object(Flow\Import\Li
17​quidThreadsApi\ImportPost), Object(Flow\Model\PostRevision), ' ')
18​#8 /srv/mediawiki/php-1.26wmf13/extensions/Flow/includes/Import/Importer.php(873): Flow\Import\TalkpageImportOperation->importPost(Object(Flow\Import\TopicImportState), Object(Flow\Import\Li
19​quidThreadsApi\ImportPost), Object(Flow\Model\PostRevision), ' ')
20​#9 /srv/mediawiki/php-1.26wmf13/extensions/Flow/includes/Import/Importer.php(668): Flow\Import\TalkpageImportOperation->importPost(Object(Flow\Import\TopicImportState), Object(Flow\Import\Li
21​quidThreadsApi\ImportPost), Object(Flow\Model\PostRevision))
22​#10 /srv/mediawiki/php-1.26wmf13/extensions/Flow/includes/Import/Importer.php(589): Flow\Import\TalkpageImportOperation->importTopic(Object(Flow\Import\TopicImportState), Object(Flow\Import\LiquidThreadsApi\ImportTopic))
23​#11 /srv/mediawiki/php-1.26wmf13/extensions/Flow/includes/Import/Importer.php(128): Flow\Import\TalkpageImportOperation->import(Object(Flow\Import\PageImportState))
24​#12 /srv/mediawiki/php-1.26wmf13/extensions/Flow/includes/Import/Converter.php(188): Flow\Import\Importer->import(Object(Flow\Import\LiquidThreadsApi\ImportSource), Object(Title), Object(Flow\Import\FileImportSourceStore))
25​#13 [internal function]: Flow\Import\Converter->doConversion(Object(Title), Object(Title))
26​#14 /srv/mediawiki/php-1.26wmf13/maintenance/eval.php(78) : eval()'d code(1): ReflectionMethod->invoke(Object(Flow\Import\Converter), Object(Title), Object(Title))
27​#15 /srv/mediawiki/php-1.26wmf13/maintenance/eval.php(78): eval()
28​#16 /srv/mediawiki/multiversion/MWScript.php(97): require_once('/srv/mediawiki/...')
29​#17 {main}

Running just the conversion in isolation with that topic title works:

1​mattflaschen@terbium:~/LQT_logs_2015-07-09$ mwscript eval.php --wiki=mediawikiwiki
2​> $parsoidContextTitle = Title::newFromText( 'Topic:Rsmwhyrgc56rq2vb' );
3
4​>
5
6​> $postTitle = Title::newFromText( "Thread:Extension_talk:Header_Tabs/Header_tabs_don't_show_when_ads_appear_on_webkit_browsers/reply_(3)" );
7
8​>
9
10​> $postRevision = Revision::newFromTitle( $postTitle );
11
12​>
13
14​> $content = $postRevision->getText( Revision::RAW );
15
16​>
17
18​> echo $content;
19​This is what I got from the inspection element report:
20
21​<pre>
22​Exception thrown by ext.headertabs: Cannot read property 'length' of null load.php?debug=false&lang=en&modules=jquery%2Cmediawiki&only=scripts&skin=monobook&version=20140311…:151
23​TypeError
24​message: "Cannot read property 'length' of null"
25​stack: (...)
26​get stack: function () { [native code] }
27​set stack: function () { [native code] }
28​__proto__: Error
29​ load.php?debug=false&lang=en&modules=jquery%2Cmediawiki&only=scripts&skin=monobook&version=20140311…:151
30​log
31​</pre>
32​>
33
34​> echo Flow\Parsoid\Utils::convert( 'wikitext', 'html', $content, $parsoidContextTitle );
35​<body data-parsoid='{"dsr":[0,528,0,0]}' lang="en" class="mw-content-ltr sitedir-ltr ltr mw-body mw-body-content mediawiki" dir="ltr"><p data-parsoid='{"dsr":[0,54,0,0]}'>This is what I got from the inspection element report:</p>
36
37​<pre data-parsoid='{"stx":"html","strippedNL":true,"dsr":[56,528,5,6]}'>Exception thrown by ext.headertabs: Cannot read property 'length' of null load.php?debug=false&amp;lang=en&amp;modules=jquery%2Cmediawiki&amp;only=scripts&amp;skin=monobook&amp;version=20140311…:151
38​TypeError
39​message: "Cannot read property 'length' of null"
40​stack: (...)
41​get stack: function () { [native code] }
42​set stack: function () { [native code] }
43​__proto__: Error
44​ load.php?debug=false&amp;lang=en&amp;modules=jquery%2Cmediawiki&amp;only=scripts&amp;skin=monobook&amp;version=20140311…:151
45​log
46​</pre></body>

That suggests the topic title is not the issue, which makes me suspect the content is. Somewhere to look is the setup code in setContent (especially the preSaveTransform).

Next I will try to debug through.

(n.b. the topic title is not 100% constant despite the HistoricalUIDGenerator, though it is close: P956).

It is the content, but the first version, not the current one:

$parsoidContextTitle = Title::newFromText( 'Topic:Rsmwhyrgc56rq2vb' );
$postRevision = Revision::newFromId( 957899 );
$content = $postRevision->getText( Revision::RAW );

echo $content;

echo Flow\Parsoid\Utils::convert( 'wikitext', 'html', $content, $parsoidContextTitle );

Caused by T105997: F13. Parsoid crashes on __proto__. I just fixed that, but we'll need to wait for it to be reviewed and deployed to continue with that one.

Change 225260 had a related patch set uploaded (by Mattflaschen):
Don't abort on invalid characters when creating DOM

https://gerrit.wikimedia.org/r/225260

greg removed a subscriber: greg.Jul 16 2015, 10:45 PM

Change 225260 merged by jenkins-bot:
Don't abort on invalid characters when creating DOM

https://gerrit.wikimedia.org/r/225260

Mattflaschen-WMF added a comment.EditedJul 18 2015, 9:00 AM

I didn't plan it, but it does seem appropriate that the last talk page I convert in Mexico City is likely to be Talk:Groups/Proposals/Ciudad_de_México.

Recap:

In T92303#1446129, @Mattflaschen wrote:
mattflaschen@terbium:~$ grep -i exception ~/2015_07_09_LQT_Flow.log 
[2015-07-10 02:00:28] exception 'Flow\Exception\NoParsoidException' with message 'Failed contacting Parsoid: There was a problem during the HTTP request: 503 Service Unavailable' in /srv/mediawiki/php-1.26wmf13/extensions/Flow/includes/Parsoid/Utils.php:122

This is T105997: F13. Parsoid crashes on __proto__ (https://www.mediawiki.org/wiki/Thread:Extension_talk:Header_Tabs/Header_tabs_don%27t_show_when_ads_appear_on_webkit_browsers/reply_%283%29). Waiting on deployment.

[2015-07-10 02:57:40] exception 'Flow\Exception\WikitextException' with message 'Invalid char in CDATA 0x8

This is T106079: Parsoid includes control characters that are not allowed in HTML 5 (https://www.mediawiki.org/wiki/Thread:Extension_talk:LinkedWiki/Notice:_Undefined_index:_Beschrijving_in_/var/www/wikifarm-mw1.19/extensions/LinkedWiki/LinkedWiki.php_on_line_283/reply_%282%29), worked around by https://gerrit.wikimedia.org/r/225260 . Waiting on deployment.

[2015-07-10 11:12:20] exception 'Flow\Exception\NoParsoidException' with message 'Failed contacting Parsoid: There was a problem during the HTTP request: 503 Service Unavailable' in /srv/mediawiki/php-1.26wmf13/extensions/Flow/includes/Parsoid/Utils.php:122

This is T105562: URLs with % sign are rejected by Parsoid server. Fully done.

[2015-07-10 11:18:45] exception 'Flow\Import\ImportException' with message 'User does not exist: Ecliptica' in /srv/mediawiki/php-1.26wmf13/extensions/Flow/includes/Import/Importer.php:388

This is T106122: F12. Rename of user on MediaWiki.org is not fully reflected, causing LQT->Flow error. Fully done.

The remainder previously at https://www.mediawiki.org/w/api.php?action=query&generator=pageswithprop&gpwppropname=use-liquid-threads&prop=info&gpwplimit=500&indexpageids&format=jsonfm were the ones we decided not to do, plus ones with things like {{#useliquidthreads:0}} or {{#useliquidthreads:01}} or a redirect (I went through this type and made trivial edits to clean it up), and finally the ones with non-Latin characters.

Now the only ones there are the two waiting on deployment and the two that were deliberately not part of "Day 10".

These non-Latin ones were:

Talk:Groups/Proposals/Ciudad de México
User talk:Σ
User talk:زكريا

I ran those, and hit a bash bug (apparently fixed in later versions):

mattflaschen@terbium:~$ page='Talk:Groups/Proposals/Ciudad de México'; echo "/tmp/${page//\//-}.log";
/tmp/Talk:Groups/Proposals/Ciudad de México.log
mattflaschen@terbium:~$ page='Talk:Groups/Proposals/Ciudad de Mexico'; echo "/tmp/${page//\//-}.log";
/tmp/Talk:Groups-Proposals-Ciudad de Mexico.log

1​while read page; do mwscript extensions/Flow/maintenance/convertLqtPageOnLocalWiki.php --wiki=mediawikiwiki --debug --srcpage "$page" --logfile "/tmp/${page//\//-}.log"; done < Non-latin\ characters\ 2015-07-18.txt| tee --append ~/2015_07_18_LQT_Flow_Non-Latin.log
2​[2015-07-18 07:31:33] Starting LQT conversion of page Talk:Groups/Proposals/Ciudad de México
3​[2015-07-18 07:31:33] Archiving page from Talk:Groups/Proposals/Ciudad de México to Talk:Groups/Proposals/Ciudad de México/LQT Archive 1
4​[2015-07-18 07:31:34] Importing to Talk:Groups/Proposals/Ciudad de México
5​[2015-07-18 07:31:34] Workflow isNew: true
6​[2015-07-18 07:31:34] ensureFlowRevision status isOK: true
7​[2015-07-18 07:31:34] ensureFlowRevision status isGood: true
8​[2015-07-18 07:31:34] ensureFlowRevision already-existed: false
9​[2015-07-18 07:31:34] ensureFlowRevision revision ID: 1758746, page ID: 471715
10​[2015-07-18 07:31:34] Importing header
11​[2015-07-18 07:31:35] Imported 2 revisions for header
12
13​Warning: file_put_contents(/tmp/Talk:Groups/Proposals/Ciudad de México.log): failed to open stream: No such file or directory in /srv/mediawiki/php-1.26wmf14/extensions/Flow/includes/Import/ImportSourceStore.php on line 54
14​[2015-07-18 07:31:35] Exception while importing: Talk:Groups/Proposals/Ciudad de México
15​[2015-07-18 07:31:35] exception 'Flow\Import\ImportSourceStoreException' with message 'Could not write out source store to /tmp/Talk:Groups/Proposals/Ciudad de México.log' in /srv/mediawiki/php-1.26wmf14/extensions/Flow/includes/Import/ImportSourceStore.php:56
16​Stack trace:
17​#0 /srv/mediawiki/php-1.26wmf14/extensions/Flow/includes/Import/Importer.php(402): Flow\Import\FileImportSourceStore->save()
18​#1 /srv/mediawiki/php-1.26wmf14/extensions/Flow/includes/Import/Importer.php(567): Flow\Import\PageImportState->commit()
19​#2 /srv/mediawiki/php-1.26wmf14/extensions/Flow/includes/Import/Importer.php(128): Flow\Import\TalkpageImportOperation->import(Object(Flow\Import\PageImportState))
20​#3 /srv/mediawiki/php-1.26wmf14/extensions/Flow/includes/Import/Converter.php(188): Flow\Import\Importer->import(Object(Flow\Import\LiquidThreadsApi\ImportSource), Object(Title), Object(Flow\Import\FileImportSourceStore))
21​#4 /srv/mediawiki/php-1.26wmf14/extensions/Flow/includes/Import/Converter.php(139): Flow\Import\Converter->doConversion(Object(Title), NULL)
22​#5 /srv/mediawiki/php-1.26wmf14/extensions/Flow/includes/Import/Converter.php(108): Flow\Import\Converter->convert(Object(Title))
23​#6 /srv/mediawiki/php-1.26wmf14/extensions/Flow/maintenance/convertLqtPageOnLocalWiki.php(77): Flow\Import\Converter->convertAll(Array)
24​#7 /srv/mediawiki/php-1.26wmf14/maintenance/doMaintenance.php(103): ConvertLqtPageOnLocalWiki->execute()
25​#8 /srv/mediawiki/php-1.26wmf14/extensions/Flow/maintenance/convertLqtPageOnLocalWiki.php(84): require_once('/srv/mediawiki/...')
26​#9 /srv/mediawiki/multiversion/MWScript.php(97): require_once('/srv/mediawiki/...')
27​#10 {main}
28​[2015-07-18 07:31:35] Finished LQT conversion of page Talk:Groups/Proposals/Ciudad de México
29​[2015-07-18 07:31:35] Starting LQT conversion of page User talk:Σ
30​[2015-07-18 07:31:35] Archiving page from User talk:Σ to User talk:Σ/LQT Archive 1
31​[2015-07-18 07:31:36] Importing to User talk:Σ
32​[2015-07-18 07:31:36] Workflow isNew: true
33​[2015-07-18 07:31:36] ensureFlowRevision status isOK: true
34​[2015-07-18 07:31:36] ensureFlowRevision status isGood: true
35​[2015-07-18 07:31:36] ensureFlowRevision already-existed: false
36​[2015-07-18 07:31:36] ensureFlowRevision revision ID: 1758749, page ID: 471716
37​[2015-07-18 07:31:36] Importing header
38​[2015-07-18 07:31:37] Imported 2 revisions for header
39​[2015-07-18 07:31:37] Flow\Import\LiquidThreadsApi\ApiBackend::retrieveThreadData: Did not find thread with conditions: {"thpage":"User talk:\u03a3\/LQT Archive 1","thstartid":1}
40​[2015-07-18 07:31:37] Flow\Import\LiquidThreadsApi\ApiBackend::retrieveThreadData: Did not find thread with conditions: {"thpage":"User talk:\u03a3\/LQT Archive 1","thstartid":1}
41​[2015-07-18 07:31:37] Imported 1 items, failed 0
42​[2015-07-18 07:31:37] Completed import to User talk:Σ from User talk:Σ/LQT Archive 1
43​[2015-07-18 07:31:37] Finished LQT conversion of page User talk:Σ
44​[2015-07-18 07:31:38] Starting LQT conversion of page User talk:زكريا
45​[2015-07-18 07:31:38] Finished LQT conversion of page User talk:زكريا

So I had to redo that one (for posterity we can use $(echo "$page"|sed 's#/#-#g') but it wasn't necessary since there was only one left).

The redo went fine:

1​mattflaschen@terbium:~$ mwscript extensions/Flow/maintenance/convertLqtPageOnLocalWiki.php --wiki=mediawikiwiki --debug --srcpage "Talk:Groups/Proposals/Ciudad_de_México/LQT_Archive_1" --logfile "/tmp/Talk:Groups-Proposals-Ciudad de México.log"| tee --append ~/2015_07_18_LQT_Flow_Ciudad_de_México.log
2​[2015-07-18 08:35:26] Starting LQT conversion of page Talk:Groups/Proposals/Ciudad_de_México/LQT_Archive_1
3​[2015-07-18 08:35:26] Page previously archived from Talk:Groups/Proposals/Ciudad de México to Talk:Groups/Proposals/Ciudad de México/LQT Archive 1
4​[2015-07-18 08:35:26] Importing to Talk:Groups/Proposals/Ciudad de México
5​[2015-07-18 08:35:26] Workflow isNew: false
6​[2015-07-18 08:35:26] Importing header
7​[2015-07-18 08:35:27] Imported 2 revisions for header
8​[2015-07-18 08:35:27] Importing new topic
9​[2015-07-18 08:35:27] Finished importing topic title with 1 revisions
10​[2015-07-18 08:35:27] Importing post
11​[2015-07-18 08:35:27] Finished importing post with 2 revisions
12​[2015-07-18 08:35:27] Importing post
13​[2015-07-18 08:35:27] Finished importing post with 1 revisions
14​[2015-07-18 08:35:27] Importing post
15​[2015-07-18 08:35:28] Finished importing post with 2 revisions
16​[2015-07-18 08:35:28] Importing post
17​[2015-07-18 08:35:28] Finished importing post with 2 revisions
18​[2015-07-18 08:35:28] Importing post
19​[2015-07-18 08:35:28] Finished importing post with 1 revisions
20​[2015-07-18 08:35:28] Importing post
21​[2015-07-18 08:35:29] Finished importing post with 3 revisions
22​[2015-07-18 08:35:34] Importing new topic
23​[2015-07-18 08:35:34] Finished importing topic title with 1 revisions
24​[2015-07-18 08:35:34] Importing post
25​[2015-07-18 08:35:34] Finished importing post with 2 revisions
26​[2015-07-18 08:35:34] Importing post
27​[2015-07-18 08:35:34] Finished importing post with 1 revisions
28​[2015-07-18 08:35:38] Importing new topic
29​[2015-07-18 08:35:38] Finished importing topic title with 1 revisions
30​[2015-07-18 08:35:38] Importing post
31​[2015-07-18 08:35:38] Finished importing post with 2 revisions
32​[2015-07-18 08:35:41] Importing new topic
33​[2015-07-18 08:35:41] Finished importing topic title with 1 revisions
34​[2015-07-18 08:35:41] Importing post
35​[2015-07-18 08:35:42] Finished importing post with 2 revisions
36​[2015-07-18 08:35:42] Importing post
37​[2015-07-18 08:35:42] Finished importing post with 1 revisions
38​[2015-07-18 08:35:42] Importing post
39​[2015-07-18 08:35:42] Finished importing post with 1 revisions
40​[2015-07-18 08:35:49] Importing new topic
41​[2015-07-18 08:35:49] Finished importing topic title with 1 revisions
42​[2015-07-18 08:35:49] Importing post
43​[2015-07-18 08:35:49] Finished importing post with 2 revisions
44​[2015-07-18 08:35:53] Flow\Import\LiquidThreadsApi\ApiBackend::retrieveThreadData: Did not find thread with conditions: {"thpage":"Talk:Groups\/Proposals\/Ciudad de M\u00e9xico\/LQT Archive 1","thstartid":21656}
45​[2015-07-18 08:35:53] Imported 6 items, failed 0
46​[2015-07-18 08:35:53] Completed import to Talk:Groups/Proposals/Ciudad de México from Talk:Groups/Proposals/Ciudad de México/LQT Archive 1
47​[2015-07-18 08:35:53] Finished LQT conversion of page Talk:Groups/Proposals/Ciudad_de_México/LQT_Archive_1

DannyH renamed this task from U1. Convert LQT pages on MediaWiki.org to Flow to Convert LQT pages on MediaWiki.org to Flow (tracking).Jul 22 2015, 9:27 PM
DannyH moved this task from Untriaged to Tracking tickets on the Collaboration-Team-Triage board.

The two posts with special bugs from day 10 are done:

In T92303#1446129, @Mattflaschen wrote:
mattflaschen@terbium:~$ grep -i exception ~/2015_07_09_LQT_Flow.log 
[2015-07-10 02:00:28] exception 'Flow\Exception\NoParsoidException' with message 'Failed contacting Parsoid: There was a problem during the HTTP request: 503 Service Unavailable' in /srv/mediawiki/php-1.26wmf13/extensions/Flow/includes/Parsoid/Utils.php:122

This is T105997: F13. Parsoid crashes on __proto__ (https://www.mediawiki.org/wiki/Thread:Extension_talk:Header_Tabs/Header_tabs_don%27t_show_when_ads_appear_on_webkit_browsers/reply_%283%29). Waiting on deployment.

Done. Log at P1078 .

[2015-07-10 02:57:40] exception 'Flow\Exception\WikitextException' with message 'Invalid char in CDATA 0x8

This is T106079: Parsoid includes control characters that are not allowed in HTML 5 (https://www.mediawiki.org/wiki/Thread:Extension_talk:LinkedWiki/Notice:_Undefined_index:_Beschrijving_in_/var/www/wikifarm-mw1.19/extensions/LinkedWiki/LinkedWiki.php_on_line_283/reply_%282%29), worked around by https://gerrit.wikimedia.org/r/225260 . Waiting on deployment.

Done. Log at P1079 .

I also did https://www.mediawiki.org/wiki/User_talk:SPage_%28WMF%29/LQT_test_moved (a test page created since the original day 10 run), so day 10 is now done.

Next up, VisualEditor/Feedback.

Change 227407 had a related patch set uploaded (by Mattflaschen):
Allow 0's {{useliquidthreads:01}} when handling magic word.

https://gerrit.wikimedia.org/r/227407

That fixes a rare issue that we saw on MediaWiki.org. It doesn't affect either of the two remaining pages.

Change 227407 merged by jenkins-bot:
Allow 0's {{useliquidthreads:01}} when handling magic word.

https://gerrit.wikimedia.org/r/227407

I don't know how happened, but apparently someone was able to reply to an already converted LQT post in the LQT board instead of the Flow board...

Link to the LQT reply.

I'm only able to access the LQT post from my watchlist. And from there, access the direct link to the message. I'm wondering if the user opened the Flow board on another tab but replied on the Special:NewMessages itself. Still, on the Special:NewMessages page, previous messages are redirects, so it's weird that the user would reply from there. Am I missing something?

@Ciencia_Al_Poder Thank you for the report. I don't know how they did their reply, but I confirmed your suggestion that it was possible to reply in Special:NewMessages, even when useliquidthreads has already been removed (so it is no longer an LQT talk page). This is a bug in LQT: T107894 . Tested at https://www.mediawiki.org/w/index.php?title=Thread:User_talk:Superm401/Hello!/reply&oldid=1812176 .

@Paladox: Please stop enabling LiquidThreads on MediaWiki.org.

Other people (e.g. @Jackmcbarn): There is no need to delete them (it's fine to do so if they're completely empty though). I will do a final round of conversion after we're done the support desk, then we'll technically prevent people from turning on LQT on MediaWiki.org

Ok sorry. Could the special page that allows us to enable flow on pages be opended to everyone please. But not allow people to convert pages when protection is enable like ones that only allow admins to edit the page. It would be easer to allow us to use the page and enable flow on those talk pages.

Support desk is done. :)

So we just need to schedule a window where we can convert the remaining page(s) then immediately freeze it: https://gerrit.wikimedia.org/r/#/c/228192/

Description needs to be updated and cross out support desk like the rest has.

Thanks, done.

Mattflaschen-WMF added a comment.EditedAug 17 2015, 10:13 PM

It's done!

Unfortunately, there were some final technical difficulties, but it's all solved now.

XDebug (which was only installed to help with this) triggered its infamous "Fatal error: Maximum function nesting level of '100' reached, aborting".

This failure was during flushDeferredQueue, so the DB commit, cache commit, and source store save were done. I made a mistake when resuming it which caused a duplicate topic, which I cleaned up manually.

Logs:

Initial run: P1889

Final resume: P1890

Good to have another two-pointer done.