Page MenuHomePhabricator

Run mediawiki edit reconstruction 2017-07 snapshot with new set of wikis
Closed, ResolvedPublic

Description

We added recently a whole new set of hosts that still do not have access for our scoop user.

  • to trigger the history reconstruction, I would now have to add a _SUCCESS flag manually to <<table-name>>/2017-07/ directory for all tables.
  • or I could wait until they give us access and rerun the sqoop for the failed wikis (I listed the ones that failed in a comment on the phab task)

, here's the script to start the normal 2017_07 job (run with sudo -u hdfs and double check it 'cause I haven't run it):

touch _SUCCESS
hdfs dfs -put _SUCCESS /wmf/data/raw/mediawiki/tables/revision/snapshot\=2017-07/
hdfs dfs -put _SUCCESS /wmf/data/raw/mediawiki/tables/archive/snapshot\=2017-07/
hdfs dfs -put _SUCCESS /wmf/data/raw/mediawiki/tables/ipblocks/snapshot\=2017-07/
hdfs dfs -put _SUCCESS /wmf/data/raw/mediawiki/tables/logging/snapshot\=2017-07/
hdfs dfs -put _SUCCESS /wmf/data/raw/mediawiki/tables/page/snapshot\=2017-07/
hdfs dfs -put _SUCCESS /wmf/data/raw/mediawiki/tables/pagelinks/snapshot\=2017-07/
hdfs dfs -put _SUCCESS /wmf/data/raw/mediawiki/tables/redirect/snapshot\=2017-07/
hdfs dfs -put _SUCCESS /wmf/data/raw/mediawiki/tables/revision/snapshot\=2017-07/
hdfs dfs -put _SUCCESS /wmf/data/raw/mediawiki/tables/user/snapshot\=2017-07/
hdfs dfs -put _SUCCESS /wmf/data/raw/mediawiki/tables/user_groups/snapshot\=2017-07/

Event Timeline

Nuria renamed this task from Run mediawiki edit reconstruction with new set of wikis to Run mediawiki edit reconstruction 2017-07 snapshot with new set of wikis .Aug 3 2017, 11:32 PM