Prepare and check production and labs-side filtering for olowiki
Closed, ResolvedPublic

Description

please prepare the storage layer for new olo.wikipedia wiki

"olo" is "Livvi-Karelian" and approved in https://meta.wikimedia.org/wiki/Requests_for_new_languages/Wikipedia_Livvi-Karelian

It will be a regular public-facing Wikipedia language version.

this refers to the instructions on https://wikitech.wikimedia.org/wiki/Add_a_wiki in step 0.

//"Tell in advance the Ops list, jcrespo (jynus) or Coren that this is happening so the storage layer can be prepared and checked (labs, backups, dumps). In particular, it needs to be made clear whether the wiki will be public - if so, ops will arrange for the wiki to be replicated to labs. If not, ops will need to add the wiki to $private_wikis in the puppet repository's manifests/realm.pp.

This can be fixed after the wiki creation, but it is more painful/involves custom queries. "//

I will mail the list and refer to this ticket.

Dzahn created this task.Oct 4 2016, 12:15 PM
Restricted Application added a project: Cloud-Services. · View Herald TranscriptOct 4 2016, 12:15 PM
Dzahn updated the task description. (Show Details)Oct 4 2016, 12:16 PM
Dzahn added a project: Operations.
Dzahn added a subscriber: Marostegui.
Dzahn added a subscriber: chasemp.Oct 4 2016, 12:18 PM

This is not a blocking step at this point- the process can continue but this must be kept open until the production side of filtering is run.

Dzahn added a comment.Oct 4 2016, 3:07 PM

Thank you, i will go ahead with adding it to DNS.

MarcoAurelio renamed this task from prepare storage layer for olo.wikipedia to Prepare storage layer for olo.wikipedia.Oct 4 2016, 9:24 PM
MarcoAurelio triaged this task as Normal priority.
jcrespo moved this task from Triage to Next on the DBA board.Oct 6 2016, 3:07 PM
jcrespo claimed this task.

Claimed, but will be done together with @Marostegui for demonstration purposes.

jcrespo renamed this task from Prepare storage layer for olo.wikipedia to Prepare and check production and labs-side filtering for olowiki.Oct 6 2016, 3:09 PM

Since the wiki has been created and is now live, is this resolved?

No it is not, it is not available on labs- and it should not be until this is resolved.

@Marostegui we should do this tomorrow, with special guest @chasemp , if he wants.

Marostegui added a comment.EditedOct 10 2016, 7:19 PM

Sounds good to me, let's do it tomorrow!

We have added olowiki filtering.
We executed the following command

root@neodymium:/home/jynus/software/redactatron/scripts# ./redact_standard_output.sh db1069 3313 olowiki | mysql -h db1069 -P3313 olowiki

We did some sanity checks to make sure everything was in place and the filtering was correct.

MariaDB SANITARIUM localhost olowiki > show triggers in olowiki\G
*************************** 1. row ***************************
             Trigger: abuse_filter_log_insert
               Event: INSERT
               Table: abuse_filter_log
           Statement: SET NEW.afl_ip = ''
              Timing: BEFORE
             Created: NULL
            sql_mode: IGNORE_BAD_TABLE_OPTIONS
             Definer: root@localhost
character_set_client: utf8
collation_connection: utf8_general_ci
  Database Collation: binary
*************************** 2. row ***************************
             Trigger: abuse_filter_log_update
               Event: UPDATE
               Table: abuse_filter_log
           Statement: SET NEW.afl_ip = ''
              Timing: BEFORE
             Created: NULL
            sql_mode: IGNORE_BAD_TABLE_OPTIONS
             Definer: root@localhost
character_set_client: utf8
collation_connection: utf8_general_ci
  Database Collation: binary
*************************** 3. row ***************************
             Trigger: archive_insert
               Event: INSERT
               Table: archive
           Statement: SET NEW.ar_text = '', NEW.ar_comment = ''
              Timing: BEFORE
             Created: NULL
            sql_mode: IGNORE_BAD_TABLE_OPTIONS
             Definer: root@localhost
character_set_client: utf8
collation_connection: utf8_general_ci
  Database Collation: binary
*************************** 4. row ***************************
             Trigger: archive_update
               Event: UPDATE
               Table: archive
           Statement: SET NEW.ar_text = '', NEW.ar_comment = ''
              Timing: BEFORE
             Created: NULL
            sql_mode: IGNORE_BAD_TABLE_OPTIONS
             Definer: root@localhost
character_set_client: utf8
collation_connection: utf8_general_ci
  Database Collation: binary
*************************** 5. row ***************************
             Trigger: recentchanges_insert
               Event: INSERT
               Table: recentchanges
           Statement: SET NEW.rc_ip = ''
              Timing: BEFORE
             Created: NULL
            sql_mode: IGNORE_BAD_TABLE_OPTIONS
             Definer: root@localhost
character_set_client: utf8
collation_connection: utf8_general_ci
  Database Collation: binary
*************************** 6. row ***************************
             Trigger: recentchanges_update
               Event: UPDATE
               Table: recentchanges
           Statement: SET NEW.rc_ip = ''
              Timing: BEFORE
             Created: NULL
            sql_mode: IGNORE_BAD_TABLE_OPTIONS
             Definer: root@localhost
character_set_client: utf8
collation_connection: utf8_general_ci
  Database Collation: binary
*************************** 7. row ***************************
             Trigger: revision_insert
               Event: INSERT
               Table: revision
           Statement: SET NEW.rev_text_id = 0
              Timing: BEFORE
             Created: NULL
            sql_mode: IGNORE_BAD_TABLE_OPTIONS
             Definer: root@localhost
character_set_client: utf8
collation_connection: utf8_general_ci
  Database Collation: binary
*************************** 8. row ***************************
             Trigger: revision_update
               Event: UPDATE
               Table: revision
           Statement: SET NEW.rev_text_id = 0
              Timing: BEFORE
             Created: NULL
            sql_mode: IGNORE_BAD_TABLE_OPTIONS
             Definer: root@localhost
character_set_client: utf8
collation_connection: utf8_general_ci
  Database Collation: binary
*************************** 9. row ***************************
             Trigger: user_insert
               Event: INSERT
               Table: user
           Statement: SET NEW.user_password = '', NEW.user_newpassword = '', NEW.user_email = '', NEW.user_token = '', NEW.user_email_authenticated = '', NEW.user_email_token = '', NEW.user_email_token_expires = '', NEW.user_newpass_time = ''
              Timing: BEFORE
             Created: NULL
            sql_mode: IGNORE_BAD_TABLE_OPTIONS
             Definer: root@localhost
character_set_client: utf8
collation_connection: utf8_general_ci
  Database Collation: binary
*************************** 10. row ***************************
             Trigger: user_update
               Event: UPDATE
               Table: user
           Statement: SET NEW.user_password = '', NEW.user_newpassword = '', NEW.user_email = '', NEW.user_token = '', NEW.user_email_authenticated = '', NEW.user_email_token = '', NEW.user_email_token_expires = '', NEW.user_newpass_time = ''
              Timing: BEFORE
             Created: NULL
            sql_mode: IGNORE_BAD_TABLE_OPTIONS
             Definer: root@localhost
character_set_client: utf8
collation_connection: utf8_general_ci
  Database Collation: binary
10 rows in set (0.00 sec)
MariaDB SANITARIUM localhost olowiki > SELECT * FROM user WHERE user_password != '';
Empty set (0.01 sec)

MariaDB SANITARIUM localhost olowiki > SELECT * FROM recentchanges WHERE rc_ip != '';
Empty set (0.00 sec)

Nominatively assigning to Chase, as the production part is done, but of course feel free to reassign it/unclaim it for now.

chasemp closed this task as Resolved.Nov 2 2016, 7:56 PM

olowiki_p exists

Meno25 removed a subscriber: Meno25.Nov 3 2016, 12:33 PM