Page MenuHomePhabricator

Run hadoop analysis on wb_terms migration for entities below 29 million to check state
Closed, ResolvedPublic

Description

Relates to T208425
Holes should be checked in the same way as in T239470: Check the success of the initial terms migration (does it have holes), specifically T239470#5732008
Analytics will need to resqoop these tables, we will need to ask for that!

Event Timeline

Restricted Application added a project: Wikidata. · View Herald TranscriptJan 27 2020, 12:20 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Addshore updated the task description. (Show Details)Jan 27 2020, 12:47 PM
Addshore triaged this task as Medium priority.Jan 27 2020, 3:33 PM
Addshore moved this task from Incoming to Ready to pick up on the Wikidata-Campsite board.
Addshore updated the task description. (Show Details)
Ladsgroup added a subscriber: Ladsgroup.

Doing it now.

Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptJan 28 2020, 5:19 PM

It seems that our term store is badly broken:

MariaDB [wikidatawiki_p]> SELECT   wbit_item_id as id,   wby_name as type,   wbxl_language as language,   wbx_text as text FROM wbt_item_terms LEFT JOIN wbt_term_in_lang ON wbit_term_in_lang_id = wbtl_id LEFT JOIN wbt_type ON wbtl_type_id = wby_id LEFT JOIN wbt_text_in_lang ON wbtl_text_in_lang_id = wbxl_id LEFT JOIN wbt_text ON wbxl_text_id = wbx_id WHERE wbit_item_id = 452581;
+--------+-------+----------+--------------------------------+
| id     | type  | language | text                           |
+--------+-------+----------+--------------------------------+
| 452581 | label | de       | Asantehene                     |
| 452581 | label | pl       | Asantehene                     |
| 452581 | label | en       | list of rulers of Asante       |
| 452581 | label | lt       | Asantehene                     |
| 452581 | label | nl       | Asantehene                     |
| 452581 | label | ja       | 君主                           |
| 452581 | label | ru       | Ашантихене                     |
| 452581 | alias | ja       | アシャンティ王の一覧           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
| 452581 | NULL  | NULL     | NULL                           |
+--------+-------+----------+--------------------------------+
83 rows in set (0.01 sec)
MariaDB [wikidatawiki_p]> SELECT   wbit_item_id as id,   wby_name as type,   wbxl_language as language,   wbx_text as text FROM wbt_item_terms LEFT JOIN wbt_term_in_lang ON wbit_term_in_lang_id = wbtl_id LEFT JOIN wbt_type ON wbtl_type_id = wby_id LEFT JOIN wbt_text_in_lang ON wbtl_text_in_lang_id = wbxl_id LEFT JOIN wbt_text ON wbxl_text_id = wbx_id WHERE wbit_item_id = 8739816;
+---------+-------+----------+---------------------------------+
| id      | type  | language | text                            |
+---------+-------+----------+---------------------------------+
| 8739816 | label | en       | Category:People from Lanškroun  |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
| 8739816 | NULL  | NULL     | NULL                            |
+---------+-------+----------+---------------------------------+
129 rows in set (0.09 sec)
Addshore closed this task as Resolved.Jan 31 2020, 9:11 AM