Page MenuHomePhabricator
Create Task
Maniphest T106614

F9. Run LQT conversion script on mediawiki.org Project:Support desk
Closed, ResolvedPublic
Actions

Description

When blocking tasks are done, run the LQT conversion script on Project:Support desk.

Support desk script run started at 2015-07-30 23:05:45 UTC:

[2015-07-30 23:05:45] Starting LQT conversion of page Project:Support_desk

Details

SubjectRepoBranchLines +/-
Eliminate some memory leaks affecting LQT->Flow:mediawiki/extensions/Flowwmf/1.26wmf18+103 -33
Eliminate some memory leaks affecting LQT->Flow:mediawiki/extensions/Flowmaster+103 -33
Customize query in gerrit

Related Objects
Search...

Event Timeline

• DannyH created this task.Jul 22 2015, 9:40 PM
• DannyH raised the priority of this task from to High.
• DannyH updated the task description. (Show Details)
• DannyH added a project: Collaboration-Team-Archive-2015-2016.
• DannyH moved this task to Untriaged on the Collaboration-Team-Archive-2015-2016 board.
• DannyH subscribed.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 22 2015, 9:40 PM
• DannyH added a parent task: T92303: Convert LQT pages on MediaWiki.org to Flow (tracking).Jul 22 2015, 9:41 PM
• DannyH added subtasks: T106569: F7. Check for other missing users on MediaWiki.org in revision or LQT tables, T105484: F6. Posts edited by Flow talk page manager get the wrong timestamp on LQT->Flow conversion (conversion time, not edit time), which makes the "recently active topics" sort option wrong, T105997: F13. Parsoid crashes on __proto__, T106122: F12. Rename of user on MediaWiki.org is not fully reflected, causing LQT->Flow error.
• DannyH added a subtask: T106079: Wikitext includes control characters that are not allowed in HTML 5.Jul 22 2015, 9:47 PM
• DannyH added a project: StructuredDiscussions.Jul 22 2015, 11:17 PM
• DannyH set Security to None.
Restricted Application added a project: Collaboration-Team-Triage. · View Herald TranscriptJul 22 2015, 11:17 PM
• DannyH moved this task from Untriaged to Current workboard on the Collaboration-Team-Triage board.Jul 22 2015, 11:18 PM
• DannyH assigned this task to • Mattflaschen-WMF.Jul 23 2015, 12:34 AM
• DannyH renamed this task from Run LQT conversion script on mediawiki.org Project:Support desk to F9. Run LQT conversion script on mediawiki.org Project:Support desk.Jul 23 2015, 12:42 AM
Catrope closed subtask T106569: F7. Check for other missing users on MediaWiki.org in revision or LQT tables as Resolved.Jul 23 2015, 6:42 PM
• DannyH closed subtask T105997: F13. Parsoid crashes on __proto__ as Resolved.Jul 23 2015, 9:42 PM
• DannyH closed subtask T106122: F12. Rename of user on MediaWiki.org is not fully reflected, causing LQT->Flow error as Resolved.
• DannyH closed subtask T105484: F6. Posts edited by Flow talk page manager get the wrong timestamp on LQT->Flow conversion (conversion time, not edit time), which makes the "recently active topics" sort option wrong as Resolved.Jul 27 2015, 10:51 PM
• Mattflaschen-WMF removed a subtask: T106079: Wikitext includes control characters that are not allowed in HTML 5.Jul 29 2015, 6:03 AM

Removed blocker since the workaround is sufficient for now.

• Mattflaschen-WMF moved this task from Untriaged to In Development on the Collaboration-Team-Archive-2015-2016 board.Jul 31 2015, 2:15 AM
• Mattflaschen-WMF added a comment.Aug 3 2015, 10:05 PM

Back of the envelope calculation is that we're about half done. I tried comparing to previous conversions to estimate time per topic, but the resulting time estimate was too low (not sure why).

This is instead based on looking at the progress of competed topics (since they are turned into redirects):

mysql:research@s3-analytics-slave [mediawikiwiki]> SELECT COUNT(*) FROM page WHERE page_namespace = 90 AND page_title LIKE 'Project:Support_desk/%' AND page_title NOT LIKE '%/reply%%%%' AND page_is_redirect;
+----------+
| COUNT(*) |
+----------+
|     3333 |
+----------+
1 row in set (0.51 sec)

mysql:research@s3-analytics-slave [mediawikiwiki]> SELECT COUNT(*) FROM page WHERE page_namespace = 90 AND page_title LIKE 'Project:Support_desk/%' AND page_title NOT LIKE '%/reply%%%%';
+----------+
| COUNT(*) |
+----------+
|     6835 |
+----------+
1 row in set (0.02 sec)

3333/6835 = 48.8%

• Mattflaschen-WMF updated the task description. (Show Details)Aug 3 2015, 10:10 PM
• DannyH moved this task from Current workboard to LQT tickets (temporary) on the Collaboration-Team-Triage board.Aug 4 2015, 11:05 PM
• Mattflaschen-WMF added a comment.Aug 5 2015, 2:20 AM

TLDR, it seems to be getting slower. For now we can probably still just leave it, but it might be like 10 days. If it gets even slower, we will have to probably stop it and fix the problem (it is resumable):

+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| (SELECT COUNT(*) FROM page WHERE page_namespace = 90 AND page_title LIKE 'Project:Support_desk/%' AND page_title NOT LIKE '%/reply%%%%' AND page_is_redirect) / (SELECT COUNT(*) FROM page WHERE page_namespace = 90 AND page_title LIKE 'Project:Support_desk/% |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                                                                                                                                                                                                                                           0.5324 |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.06 sec)

It does seem to be slowing done on the topic iteration:

[2015-07-30 22:12:46] Imported 2 revisions for header
[2015-07-30 22:12:47] Importing new topic
--
[2015-07-30 22:12:48]    Finished importing post with 1 revisions
[2015-07-30 22:13:25] Importing new topic
--
[2015-07-30 22:13:26]    Finished importing post with 1 revisions
[2015-07-30 22:13:31] Importing new topic
--
[2015-07-30 22:13:32]   Finished importing post with 1 revisions
[2015-07-30 22:13:37] Importing new topic
--
[2015-07-30 22:13:38]     Finished importing post with 1 revisions
[2015-07-30 22:13:49] Importing new topic
--
[2015-07-30 22:13:51]     Finished importing post with 1 revisions
[2015-07-30 22:13:58] Importing new topic
--
[2015-07-30 22:13:58]    Finished importing post with 1 revisions
[2015-07-30 22:14:02] Importing new topic
--
[2015-07-30 22:14:03]   Finished importing post with 1 revisions
[2015-07-30 22:14:07] Importing new topic
--
[2015-07-30 22:14:09]     Finished importing post with 1 revisions
[2015-07-30 22:14:15] Importing new topic
--
[2015-07-30 22:14:20]         Finished importing post with 2 revisions
[2015-07-30 22:14:28] Importing new topic
--
[2015-08-05 00:26:26]                 Finished importing post with 1 revisions
[2015-08-05 00:44:02] Importing new topic
--
[2015-08-05 00:44:19]  Finished importing post with 1 revisions
[2015-08-05 00:46:07] Importing new topic
--
[2015-08-05 00:46:34]   Finished importing post with 1 revisions
[2015-08-05 00:49:18] Importing new topic
--
[2015-08-05 00:50:19]      Finished importing post with 1 revisions
[2015-08-05 00:55:46] Importing new topic
--
[2015-08-05 00:56:25]    Finished importing post with 1 revisions
[2015-08-05 01:00:01] Importing new topic
--
[2015-08-05 01:01:23]     Finished importing post with 1 revisions
[2015-08-05 01:06:41] Importing new topic
--
[2015-08-05 01:10:03]                 Finished importing post with 1 revisions
[2015-08-05 01:29:08] Importing new topic
--
[2015-08-05 01:30:09]       Finished importing post with 1 revisions
[2015-08-05 01:36:03] Importing new topic
--
[2015-08-05 01:41:10]   Finished importing post with 3 revisions
[2015-08-05 02:05:49] Importing new topic
--
[2015-08-05 02:06:15]   Finished importing post with 1 revisions
[2015-08-05 02:08:51] Importing new topic

Something is slowing down the iteration of topics; in the beginning there were not these gaps, but now they're long.

I think the query is roughly:

SELECT  thread_id,thread_id,thread_subject,thread_article_namespace,thread_article_title,thread_parent,thread_ancestor,thread_created,thread_modified,thread_author_id,thread_author_name,thread_summary_page,thread_root,thread_type,thread_signature  FROM `thread`   WHERE thread_article_namespace = '0' AND thread_article_title = 'Test_LQT' AND (thread_id>='674') AND (thread_type != '2')  ORDER BY thread_id LIMIT 501

(that's taken from my test wiki debugging, but can be changed to match prod).

Even doing it without the LIMIT doesn't take forever:

SELECT  thread_id,thread_id,thread_subject,thread_article_namespace,thread_article_title,thread_parent,thread_ancestor,thread_created,thread_modified,thread_author_id,thread_author_name,thread_summary_page,thread_root,thread_type,thread_signature  FROM `thread`   WHERE thread_article_namespace = '4' AND thread_article_title = 'Support_desk/LQT_Archive_1' AND (thread_id>='674') AND (thread_type != '2')  ORDER BY thread_id;

27456 rows in set (0.42 sec)

• Mattflaschen-WMF added a comment.Aug 5 2015, 6:12 PM
mysql:research@s3-analytics-slave [mediawikiwiki]> SELECT (SELECT COUNT(*) FROM page WHERE page_namespace = 90 AND page_title LIKE 'Project:Support_desk/%' AND page_title NOT LIKE '%/reply%%%%' AND page_is_redirect) / (SELECT COUNT(*) FROM page WHERE page_namespace = 90 AND page_title LIKE 'Project:Support_desk/%' AND page_title NOT LIKE '%/reply%%%%');
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| (SELECT COUNT(*) FROM page WHERE page_namespace = 90 AND page_title LIKE 'Project:Support_desk/%' AND page_title NOT LIKE '%/reply%%%%' AND page_is_redirect) / (SELECT COUNT(*) FROM page WHERE page_namespace = 90 AND page_title LIKE 'Project:Support_desk/% |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                                                                                                                                                                                                                                           0.5568 |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.93 sec)
• Mattflaschen-WMF added a comment.Aug 5 2015, 11:10 PM

The slowdown may be partly to writing larger and larger source store logs. Currently at:

-rw-r--r-- 1 www-data www-data 2.0M Aug  5 23:03 Project:Support_desk.log
• Mattflaschen-WMF added a comment.Aug 7 2015, 5:53 PM
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (3.76 sec)

mysql:research@s3-analytics-slave [mediawikiwiki]> SELECT (SELECT COUNT(*) FROM page WHERE page_namespace = 90 AND page_title LIKE 'Project:Support_desk/%' AND page_title NOT LIKE '%/reply%%%%' AND page_is_redirect) / (SELECT COUNT(*) FROM page WHERE page_namespace = 90 AND page_title LIKE 'Project:Support_desk/%' AND page_title NOT LIKE '%/reply%%%%');
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| (SELECT COUNT(*) FROM page WHERE page_namespace = 90 AND page_title LIKE 'Project:Support_desk/%' AND page_title NOT LIKE '%/reply%%%%' AND page_is_redirect) / (SELECT COUNT(*) FROM page WHERE page_namespace = 90 AND page_title LIKE 'Project:Support_desk/% |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                                                                                                                                                                                                                                           0.6119 |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.80 sec)
• Mattflaschen-WMF added a subtask: T108601: Fix memory leak in LQT->Flow conversion script.Aug 10 2015, 5:50 PM
• Mattflaschen-WMF added a subscriber: hoo.EditedAug 10 2015, 6:02 PM

So it turned out to have a memory leak (@hoo noticed it last night). It was about 16 GB when I had to kill it:

www-data  7846 92.4 49.8 17474100 16370300 pts/47 R<+ Jul30 13483:52 php /srv/mediawiki-staging/multiversion/MWScript.php extensions/Flow/maintenance/convertLqtPageOnLocalWiki.php --wiki=mediawikiwiki --debug --srcpage Project:Support_desk --logfile /tmp/Project:Support_desk.log

I should have checked this before (this is almost definitely the cause of the slowdown (SWAP usage rather than RAM)), but it didn't occur to me since I was more thinking about the possibility of repeated iterations or inefficient database queries.

Log of the initial run is at http://people.wikimedia.org/~mattflaschen/2015_07_30_LQT_Flow_VE_Feedback_and_Support_desk.log (VE is also before that).

Good news is that we're done two thirds:

mysql:research@s3-analytics-slave [mediawikiwiki]> SELECT (SELECT COUNT(*) FROM page WHERE page_namespace = 90 AND page_title LIKE 'Project:Support_desk/%' AND page_title NOT LIKE '%/reply%%%%' AND page_is_redirect) / (SELECT COUNT(*) FROM page WHERE page_namespace = 90 AND page_title LIKE 'Project:Support_desk/%' AND page_title NOT LIKE '%/reply%%%%');
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| (SELECT COUNT(*) FROM page WHERE page_namespace = 90 AND page_title LIKE 'Project:Support_desk/%' AND page_title NOT LIKE '%/reply%%%%' AND page_is_redirect) / (SELECT COUNT(*) FROM page WHERE page_namespace = 90 AND page_title LIKE 'Project:Support_desk/% |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|                                                                                                                                                                                                                                                           0.6748 |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.73 sec)

Bad news is when we resume it will have to loop through and detect all the completed ones from the resume log, but it won't have to do any Parsoid requests or inserts.

Good news is after the memory leak is done, the remaining third should go faster.

gerritbot subscribed.Aug 11 2015, 1:18 AM

Change 230723 had a related patch set uploaded (by Mattflaschen):
WIP: Eliminate some memory leaks affecting LQT->Flow:

https://gerrit.wikimedia.org/r/230723

gerritbot added a project: Patch-For-Review.Aug 11 2015, 1:18 AM
• Mattflaschen-WMF moved this task from In Development to Needs Review on the Collaboration-Team-Archive-2015-2016 board.Aug 11 2015, 1:55 AM
• Mattflaschen-WMF moved this task from Needs Review to In Development on the Collaboration-Team-Archive-2015-2016 board.Aug 11 2015, 3:23 AM
gerritbot added a comment.Aug 11 2015, 6:00 PM

Change 230723 merged by jenkins-bot:
Eliminate some memory leaks affecting LQT->Flow:

https://gerrit.wikimedia.org/r/230723

• Mattflaschen-WMF mentioned this in rEFLWd9e250d1339a: Eliminate some memory leaks affecting LQT->Flow:.Aug 11 2015, 6:00 PM
• Mattflaschen-WMF mentioned this in rMEXT149ab0f431f4: Updated mediawiki/extensions Project: mediawiki/extensions/Flow….
gerritbot added a comment.Aug 11 2015, 10:18 PM

Change 230929 had a related patch set uploaded (by Mattflaschen):
Eliminate some memory leaks affecting LQT->Flow:

https://gerrit.wikimedia.org/r/230929

gerritbot added a comment.Aug 11 2015, 11:13 PM

Change 230929 merged by jenkins-bot:
Eliminate some memory leaks affecting LQT->Flow:

https://gerrit.wikimedia.org/r/230929

• Mattflaschen-WMF mentioned this in rEFLWa4bdad4b66a8: Eliminate some memory leaks affecting LQT->Flow:.Aug 11 2015, 11:13 PM
• Mattflaschen-WMF mentioned this in rMW494e30149490: Updated mediawiki/core Project: mediawiki/extensions/Flow….
• Mattflaschen-WMF added projects: Performance Issue, Performance-Team.EditedAug 12 2015, 2:41 AM

It's still leaking. I'm wondering if we have anything set up (xhprof?) for memory profiling, and whether I have to kill it to add some line enabling that or not.

www-data 32477 53.6 18.3 6536248 6025732 pts/47 R<+ 00:01  85:16 php /srv/mediawiki-staging/multiversion/MWScript.php extensions/Flow/maintenance/convertLqtPageOnLocalWiki.php --wiki=mediawikiwiki --debug --srcpage Project:Support_desk/LQT_Archive_1 --logfile /tmp/Project:Support_desk.log
• Mattflaschen-WMF added a subscriber: ori.EditedAug 12 2015, 4:37 AM

@ori installed XDebug on terbium for me. It looks like the relevant parameters we would need to set are:

xdebug.trace_format = 1
xdebug.show_mem_delta = 1
xdebug.auto_trace = 1

So the following should work:

php -d "xdebug.trace_format=1" -d "xdebug.show_mem_delta=1" -d "xdebug.auto_trace=1"  /srv/mediawiki-staging/multiversion/MWScript.php extensions/Flow/maintenance/convertLqtPageOnLocalWiki.php --wiki=mediawikiwiki --debug --srcpage Project:Support_desk/LQT_Archive_1 --logfile /tmp/Project:Support_desk.log

The output should go to /tmp. Holding off on trying this, since I want to see how far it will get before the memory becomes intolerable. Currently it's at:

www-data 32477 53.2 25.7 8999080 8447788 pts/47 R<+ 00:01 141:34 php /srv/mediawiki-staging/multiversion/MWScript.php extensions/Flow/maintenance/convertLqtPageOnLocalWiki.php --wiki=mediawikiwiki --debug --srcpage Project:Support_desk/LQT_Archive_1 --logfile /tmp/Project:Support_desk.log
mysql:research@s3-analytics-slave [mediawikiwiki]> SELECT CURRENT_TIMESTAMP(), (SELECT COUNT(*) FROM page WHERE page_namespace = 90 AND page_title LIKE 'Project:Support_desk/%' AND page_title NOT LIKE '%/reply%%%%' AND page_is_redirect) / (SELECT COUNT(*) FROM page WHERE page_namespace = 90 AND page_title LIKE 'Project:Support_desk/%' AND page_title NOT LIKE '%/reply%%%%');
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| CURRENT_TIMESTAMP() | (SELECT COUNT(*) FROM page WHERE page_namespace = 90 AND page_title LIKE 'Project:Support_desk/%' AND page_title NOT LIKE '%/reply%%%%' AND page_is_redirect) / (SELECT COUNT(*) FROM page WHERE page_namespace = 90 AND page_title LIKE 'Project:Support_desk/% |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 2015-08-12 04:29:53 |                                                                                                                                                                                                                                                           0.7331 |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.08 sec)

It's definitely moving faster now, but it's not clear how much of that is due to having done ~5 percentage points since restart vs. memory leak improvements. It's done 5.83 (73.31 - 67.48) percentage points since ~00:01 UTC (and that calculation penalizes it for the "check off what I already did" phase. It's 04:29 in the above, so that's 4 hours, 28 minutes. That's 1.31 an hour.

Whereas before, at one point it was doing less than 5 percentage points in over a day.

• Mattflaschen-WMF added a comment.Aug 12 2015, 5:53 AM

The RAM reached 10 GB, and I don't really feel comfortable letting it run overnight, so I decided to kill it.

Plus, we still really ought to figure out the memory leak.

So I'll re-run it with the profiling parameters from above tomorrow. Final status for tonight:

0.7494 (fraction)

and:

www-data 32477 54.5 31.1 10812664 10223660 pts/47 S<+ 00:01 189:59 php /srv/mediawiki-staging/multiversion/MWScript.php extensions/Flow/maintenance/convertLqtPageOnLocalWiki.php --wiki=mediawikiwiki --debug --srcpage Project:Support_desk/LQT_Archive_1 --logfile /tmp/Project:Support_desk.log
• Mattflaschen-WMF mentioned this in T108601: Fix memory leak in LQT->Flow conversion script.Aug 13 2015, 12:32 AM
• Mattflaschen-WMF added a comment.EditedAug 13 2015, 6:53 AM

It's still leaking, but there was a decent amount of progress (12 percentage points) on the HHVM run (see T108601):

0.8758

I'll restart it tomorrow (again on HHVM if possible, since I think it's faster).

• Mattflaschen-WMF added a comment.EditedAug 14 2015, 7:22 AM

The script run is done. :)

mysql:research@s3-analytics-slave [mediawikiwiki]> SELECT CURRENT_TIMESTAMP(), (SELECT COUNT(*) FROM page WHERE page_namespace = 90 AND page_title LIKE 'Project:Support_desk/%' AND page_title NOT LIKE '%/reply%%%%' AND page_is_redirect) / (SELECT COUNT(*) FROM page WHERE page_namespace = 90 AND page_title LIKE 'Project:Support_desk/%' AND page_title NOT LIKE '%/reply%%%%');
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| CURRENT_TIMESTAMP() | (SELECT COUNT(*) FROM page WHERE page_namespace = 90 AND page_title LIKE 'Project:Support_desk/%' AND page_title NOT LIKE '%/reply%%%%' AND page_is_redirect) / (SELECT COUNT(*) FROM page WHERE page_namespace = 90 AND page_title LIKE 'Project:Support_desk/% |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 2015-08-14 07:12:14 |                                                                                                                                                                                                                                                           0.9999 |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.69 sec)

...

mysql:research@s3-analytics-slave [mediawikiwiki]> SELECT * FROM page WHERE page_namespace = 90 AND page_title LIKE 'Project:Support_desk/%' AND page_title NOT LIKE '%/reply%%%%' AND NOT page_is_redirect;
+---------+----------------+-------------------------------------------------------+-------------------+--------------+------------------+-------------+----------------+----------------+--------------------+-------------+----------+-----------------------+--------------------+
| page_id | page_namespace | page_title                                            | page_restrictions | page_counter | page_is_redirect | page_is_new | page_random    | page_touched   | page_links_updated | page_latest | page_len | page_no_title_convert | page_content_model |
+---------+----------------+-------------------------------------------------------+-------------------+--------------+------------------+-------------+----------------+----------------+--------------------+-------------+----------+-----------------------+--------------------+
|   56726 |             90 | Project:Support_desk/How_can_I_clear_my_New_Messages? |                   |            0 |                0 |           1 | 0.693698862857 | 20120906235530 | NULL               |      387699 |      655 |                     0 | wikitext           |
+---------+----------------+-------------------------------------------------------+-------------------+--------------+------------------+-------------+----------------+----------------+--------------------+-------------+----------+-----------------------+--------------------+
1 row in set (0.02 sec)

This thread is in an inconsistent state. Compare:

https://www.mediawiki.org/wiki/Thread:Project:Support_desk/Invalid_email_confirmation_code.?redirect=no

vs.

https://www.mediawiki.org/wiki/Thread:Project:Support_desk/How_can_I_clear_my_New_Messages%3F?redirect=no

(It is a post about the server being slow when there are too many LiquidThreads messages: Really).

It's apparently since it's not in the thread table, which unfortunately we can't really do anything about:

mysql:research@s3-analytics-slave [mediawikiwiki]> SELECT thread_id, page_id, rev_id, page_namespace, page_title FROM revision JOIN page on rev_page = page_id LEFT OUTER JOIN thread ON thread_root = page_id WHERE page_id = 56726;
+-----------+---------+--------+----------------+-------------------------------------------------------+
| thread_id | page_id | rev_id | page_namespace | page_title                                            |
+-----------+---------+--------+----------------+-------------------------------------------------------+
|      NULL |   56726 | 387699 |             90 | Project:Support_desk/How_can_I_clear_my_New_Messages? |
+-----------+---------+--------+----------------+-------------------------------------------------------+
1 row in set (0.03 sec)

So I'm prepared to call it done. Logs coming next.

• Mattflaschen-WMF added a comment.EditedAug 14 2015, 8:13 AM

First log: http://people.wikimedia.org/~mattflaschen/2015_07_30_LQT_Flow_VE_Feedback_and_Support_desk.log
Second log (resume): http://people.wikimedia.org/~mattflaschen/2015_08_11_LQT_Flow_Support_desk_resume.log
Third log (resume): http://people.wikimedia.org/~mattflaschen/2015_08_12_LQT_Flow_Support_desk_resume.log
Fourth log (resume on HHVM): http://people.wikimedia.org/~mattflaschen/2015_08_12_LQT_Flow_Support_desk_hhvm_resume.log
Fifth and final log (resume on HHVM): http://people.wikimedia.org/~mattflaschen/2015_08_13_LQT_Flow_Support_desk_hhvm_resume.log

Just one error (see T108972), which is already fixed.

mattflaschen@terbium:~$ grep -iE 'warning|exception|error|fatal' ./public_html/2015_07_30_LQT_Flow_VE_Feedback_and_Support_desk.log ./public_html/2015_08_13_LQT_Flow_Support_desk_hhvm_resume.log ./public_html/2015_08_12_LQT_Flow_Support_desk_hhvm_resume.log ./public_html/2015_08_12_LQT_Flow_Support_desk_resume.log ./public_html/2015_08_11_LQT_Flow_Support_desk_resume.log
./public_html/2015_08_12_LQT_Flow_Support_desk_hhvm_resume.log:[2015-08-13 05:14:50] exception 'Flow\Exception\NoParsoidException' with message 'Failed contacting Parsoid for title "Topic:S52bxzn9rgi3j9e0": There was a problem during the HTTP request: 503 Service Unavailable' in /srv/mediawiki/php-1.26wmf18/extensions/Flow/includes/Parsoid/Utils.php:124
• Mattflaschen-WMF moved this task from In Development to QA Review on the Collaboration-Team-Archive-2015-2016 board.Aug 14 2015, 8:17 AM
• Mattflaschen-WMF added a comment.Aug 14 2015, 8:25 AM

Final summary, for posterity:

[2015-08-14 06:47:54] Imported 6955 items, failed 0
[2015-08-14 06:47:56] Completed import to Project:Support desk from Project:Support desk/LQT Archive 1
[2015-08-14 06:47:56] Finished LQT conversion of page Project:Support_desk/LQT_Archive_1
• Mattflaschen-WMF removed a subtask: T108601: Fix memory leak in LQT->Flow conversion script.Aug 14 2015, 11:47 PM
Etonkovidova subscribed.Aug 17 2015, 7:10 PM

In betalabs some additional check was done with interrupting the script to check for completeness of topic(s) conversion.

Etonkovidova moved this task from QA Review to Product Review on the Collaboration-Team-Archive-2015-2016 board.Aug 17 2015, 7:10 PM
ori removed projects: Performance-Team, Performance Issue.Aug 19 2015, 5:00 PM
• DannyH closed this task as Resolved.Aug 19 2015, 6:34 PM

Well done, it's great to close this down. :)

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL