Page MenuHomePhabricator
Create Task
Maniphest T184179

Missing references to s8 on maintenance and cloud scripts (and potentially others)
Closed, ResolvedPublic
Actions

Description

By grepping s7:

modules/mediawiki/manifests/maintenance/refreshlinks.pp:    mediawiki::maintenance::refreshlinks::cronjob { ['s1@1', 's2@2', 's3@3', 's4@4', 's5@5', 's6@6', 's7@7', 'silver@8']: }
modules/mediawiki/manifests/maintenance/updatequerypages.pp:    mediawiki::maintenance::updatequerypages::cronjob { ['s1@11', 's2@12', 's3@13', 's4@14', 's5@15', 's6@16', 's7@17', 'silver@18']: }
modules/openstack/files/util/wikireplica_dns.py:    all_shards = ['s1', 's2', 's3', 's4', 's5', 's6', 's7']
modules/openstack/files/util/wikireplica_dns.yaml:    s7:
modules/openstack/files/util/wikireplica_dns.yaml:    s7:
modules/openstack/files/util/wikireplica_dns.yaml:  s7:
Binary file modules/planet/files/theme/images/planet-wm.png matches
modules/profile/files/openstack/base/pdns/recursor/labsdb.zone:[several here]
modules/role/files/labs/db/views/maintain-meta_p.py:    for slice in ['s1', 's2', 's3', 's4', 's5', 's6', 's7']:
modules/role/files/mariadb/dump_shards.sh:SHARDS=${SHARDS:-"s2 s3 s5 s6 s7 x1"}
modules/role/manifests/mariadb/dbstore.pp:        ['s1','s2','s3','s4','s5','s6','s7','m2','m3','x1']:
modules/role/templates/mariadb/mysqld_config/dbstore.my.cnf.erb:[several here]

Details

SubjectRepoBranchLines +/-
wmcs: Add s8.labsdb and move wikidatawikioperations/puppetproduction+3 -1
dbstore: include s8 on the dbstore2001 backupsoperations/puppetproduction+1 -1
mediawiki-maintenance: Run maintenance on new s8 replica set, toooperations/puppetproduction+2 -2
wmcs: Add s8 to maintain-meta_poperations/puppetproduction+9 -6
wikireplica_dns: Add s8 shardoperations/puppetproduction+15 -7
Customize query in gerrit

Related Objects
Search...

Event Timeline

jcrespo triaged this task as Medium priority.Jan 4 2018, 1:28 PM
jcrespo created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 4 2018, 1:28 PM
jcrespo added a project: Data-Services.Jan 4 2018, 1:34 PM
jcrespo added subscribers: Andrew, bd808.

@bd808 @Andrew This contains the self-actionable part of T181643 (mostly dns-related changes, I assume, for wikireplicas), although probably that was probably already know it was pending.

The ones I realized by pure chance is the maintenance tasks.

gerritbot subscribed.Jan 4 2018, 1:40 PM

Change 402047 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] mediawiki-maintenance: Run maintenance on new s8 replica set, too

https://gerrit.wikimedia.org/r/402047

gerritbot added a project: Patch-For-Review.Jan 4 2018, 1:40 PM
jcrespo added subscribers: Joe, akosiaris, Dzahn, ArielGlenn.Jan 4 2018, 1:47 PM

^see if that patch makes sense

mark raised the priority of this task from Medium to High.Jan 4 2018, 1:48 PM
• chasemp subscribed.Jan 4 2018, 3:53 PM

Just an extra ping for @bd808 as he wrote most of what I think needs to be updated here

gerritbot added a comment.Jan 4 2018, 5:28 PM

Change 402086 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[operations/puppet@production] wikireplica_dns: Add s8 shard

https://gerrit.wikimedia.org/r/402086

gerritbot added a comment.Jan 4 2018, 5:28 PM

Change 402087 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[operations/puppet@production] wmcs: Add s8 to maintain-meta_p

https://gerrit.wikimedia.org/r/402087

gerritbot added a comment.Jan 4 2018, 5:35 PM

Change 402086 merged by Andrew Bogott:
[operations/puppet@production] wikireplica_dns: Add s8 shard

https://gerrit.wikimedia.org/r/402086

gerritbot added a comment.Jan 4 2018, 5:35 PM

Change 402087 merged by Andrew Bogott:
[operations/puppet@production] wmcs: Add s8 to maintain-meta_p

https://gerrit.wikimedia.org/r/402087

bd808 added a comment.Jan 4 2018, 7:24 PM

I think my patches above take care of the Cloud Services/Data Services side of this (DNS and views support). The scripts I patched will still need to be run to create/update the DNS records. I don't think that anything in the views will actually change as a result of the slice move.

gerritbot added a comment.Jan 5 2018, 2:30 PM

Change 402047 merged by Jcrespo:
[operations/puppet@production] mediawiki-maintenance: Run maintenance on new s8 replica set, too

https://gerrit.wikimedia.org/r/402047

jcrespo mentioned this in T177274: Provide a new s8 master for sanitarium.Jan 9 2018, 11:03 AM
jcrespo assigned this task to bd808.Jan 9 2018, 11:07 AM

I think cloud stuff is ready -although there is a bit of lag at the moment on wikidatawiki rightnow 12K seconds-, only things pending would be cloud specific setup such as dns changes, script changes and https://tools.wmflabs.org/replag/ update to include s8 (it may also need a meta_p update). Assigning symbolically to @bd808 but of course, it doesn't have to be himself- this is just a handover to indicate everything is already read on our side- loop us back if you see something strange.

gerritbot added a comment.Jan 9 2018, 1:19 PM

Change 403147 had a related patch set uploaded (by Jcrespo; owner: Jcrespo):
[operations/puppet@production] dbstore: include s8 on the dbstore2001 backups

https://gerrit.wikimedia.org/r/403147

gerritbot added a comment.Jan 9 2018, 1:19 PM

Change 403147 merged by Jcrespo:
[operations/puppet@production] dbstore: include s8 on the dbstore2001 backups

https://gerrit.wikimedia.org/r/403147

Stashbot mentioned this in T181643: Announce wikidata move to s8 to cloud-announce & update wiki docs.Jan 9 2018, 4:09 PM
Stashbot subscribed.

Mentioned in SAL (#wikimedia-operations) [2018-01-09T16:09:37Z] <bd808> data-services: added s8.{analytics,web}.db.svc.eqiad.wmflabs and aliases (T181643, T184179)

bd808 added a comment.Jan 9 2018, 4:14 PM

After running maintain-meta_p:

(u3518@s7.labsdb) [meta_p]> SELECT * FROM meta_p.wiki WHERE dbname like 'wikidata%'\G
*************************** 1. row ***************************
          dbname: wikidatawiki
            lang: en
            name: Wikidata
          family: wikidata
             url: https://www.wikidata.org
            size: 3
           slice: s8.labsdb
       is_closed: 0
        has_echo: 1
 has_flaggedrevs: 0
has_visualeditor: 0
    has_wikidata: 1
    is_sensitive: 0
1 row in set (0.00 sec)
Dzahn unsubscribed.Jan 9 2018, 7:07 PM
bd808 mentioned this in R1944:be6109b3b7d2: Add s8 slice.Jan 9 2018, 10:58 PM
gerritbot added a comment.Jan 9 2018, 11:10 PM

Change 403315 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[operations/puppet@production] wmcs: Add s8.labsdb and move wikidatawiki

https://gerrit.wikimedia.org/r/403315

gerritbot added a comment.Jan 9 2018, 11:20 PM

Change 403315 merged by Madhuvishy:
[operations/puppet@production] wmcs: Add s8.labsdb and move wikidatawiki

https://gerrit.wikimedia.org/r/403315

bd808 closed this task as Resolved.Jan 9 2018, 11:56 PM
bd808 edited projects, added cloud-services-team (Kanban); removed Patch-For-Review.

I think the Cloud-Services bits are done now.

bd808 moved this task from Inbox to Done on the cloud-services-team (Kanban) board.Jan 31 2018, 12:51 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL