Page MenuHomePhabricator
Create Task
Maniphest T181431

Setup NFS on dumps servers
Closed, ResolvedPublic
Actions

Description

  • Test mounting shares from instances
  • Test failover strategy

Details

SubjectRepoBranchLines +/-
dumps: Move nfs exports config to templateoperations/puppetproduction+19 -12
dumps: Add stat* to nfs exportsoperations/puppetproduction+2 -2
dumps_distribution: Set up initial NFS exportsoperations/puppetproduction+20 -1
dumps: Set up NFS on the dumps distribution serversoperations/puppetproduction+79 -1
public_dumps: Revert inclusion of labstore::init classoperations/puppetproduction+1 -3
Add ldap client nsswitch config file to labstore1006|7 hieraoperations/puppetproduction+2 -0
public_dumps: Add puppet class to set up NFS for dumps serversoperations/puppetproduction+78 -1
public_dumps: Set up NFS on the dumps serversoperations/puppetproduction+2 -1
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 Resolvedbd808T166402 Program 7 Outcome 3: data services
 ResolvedArielGlennT182540 get datset1001, ms1001 ready for decommission
 Resolved• madhuvishyT168486 Migrate customer-facing Dumps endpoints to Cloud Services
 Resolved• madhuvishyT171508 Investigate and implement alternative for showmount based check at instance boot time
 Resolved• madhuvishyT174590 Revert patch that adds a temporary exception to the block-for-export check for the testlabs project
 Resolved• madhuvishyT171539 Puppetize and setup initial lvms and directory structures for labstore1006|7
 ResolvedArielGlennT171541 Setup periodic rsync jobs from dumps generation hosts to labstore1006|7
 ResolvedArielGlennT188726 make sure all datasets in xmldatadumps/public/other on dataset1001 are accounted for on new labs boxes
 Resolved• ezachteT189283 Replace cron jobs from EZachte's home directory on stat1005 with rsync fetches
 ResolvedArielGlennT189284 Stop serving slowparse logs from dumps distribution servers
 Resolved• madhuvishyT181431 Setup NFS on dumps servers
 Resolved• madhuvishyT171540 Figure out how NFS failovers will work for the dumps servers - labstore1006|7
 Resolved• madhuvishyT188073 Maintain symlinks for WMCS NFS Dumps users with new directory structure
 Resolved• madhuvishyT188641 Set up the web service that serves dumps.wikimedia.org
 Resolved• madhuvishyT188642 Set up labstore1006|7 as the source for rsync mirror sites
 Resolved• madhuvishyT188643 Migrate Dumps WMCS NFS users from labstore1003 to labstore1006/7
 Resolved• madhuvishyT188644 Migrate the stat* mount from dataset1001 to labstore1006/7
 Resolved• madhuvishyT188645 Get all the rsync mirror sites to to switch over to labstore1006,7
 Resolved• madhuvishyT188646 Point dumps.wikimedia.org to labstore1006/7
 Resolved• madhuvishyT185101 Labstore1006/7 profile for meltdown kernel
 Resolved• madhuvishyT188647 Announce/Communicate dumps migration to labstore1006|7 to stakeholders
Restricted Task

Event Timeline

• madhuvishy created this task.Nov 27 2017, 7:54 PM
• madhuvishy added a subtask: T171540: Figure out how NFS failovers will work for the dumps servers - labstore1006|7.
• madhuvishy mentioned this in T168486: Migrate customer-facing Dumps endpoints to Cloud Services.Nov 27 2017, 7:56 PM
gerritbot subscribed.Nov 27 2017, 11:11 PM

Change 393680 had a related patch set uploaded (by Madhuvishy; owner: Madhuvishy):
[operations/puppet@production] public_dumps: Set up NFS on the dumps servers

https://gerrit.wikimedia.org/r/393680

gerritbot added a project: Patch-For-Review.Nov 27 2017, 11:11 PM
gerritbot added a comment.Nov 27 2017, 11:14 PM

Change 393680 merged by Madhuvishy:
[operations/puppet@production] public_dumps: Set up NFS on the dumps servers

https://gerrit.wikimedia.org/r/393680

gerritbot added a comment.Nov 28 2017, 12:34 AM

Change 393695 had a related patch set uploaded (by Madhuvishy; owner: Madhuvishy):
[operations/puppet@production] public_dumps: Add puppet class to set up NFS for dumps servers

https://gerrit.wikimedia.org/r/393695

gerritbot added a comment.Nov 28 2017, 12:38 AM

Change 393695 merged by Madhuvishy:
[operations/puppet@production] public_dumps: Add puppet class to set up NFS for dumps servers

https://gerrit.wikimedia.org/r/393695

gerritbot added a comment.Nov 28 2017, 6:31 PM

Change 393821 had a related patch set uploaded (by Madhuvishy; owner: Madhuvishy):
[operations/puppet@production] Add ldap client nsswitch config file to labstore1006|7 hiera

https://gerrit.wikimedia.org/r/393821

gerritbot added a comment.Nov 28 2017, 6:32 PM

Change 393821 merged by Madhuvishy:
[operations/puppet@production] Add ldap client nsswitch config file to labstore1006|7 hiera

https://gerrit.wikimedia.org/r/393821

gerritbot added a comment.Nov 29 2017, 11:35 PM

Change 394224 had a related patch set uploaded (by Madhuvishy; owner: Madhuvishy):
[operations/puppet@production] public_dumps: Revert inclusion of labstore::init class

https://gerrit.wikimedia.org/r/394224

gerritbot added a comment.Nov 29 2017, 11:37 PM

Change 394224 merged by Madhuvishy:
[operations/puppet@production] public_dumps: Revert inclusion of labstore::init class

https://gerrit.wikimedia.org/r/394224

ops-monitoring-bot subscribed.Nov 29 2017, 11:53 PM

Script wmf-auto-reimage was launched by madhuvishy on neodymium.eqiad.wmnet for hosts:

['labstore1007.wikimedia.org']

The log can be found in /var/log/wmf-auto-reimage/201711292351_madhuvishy_29911.log.

ops-monitoring-bot added a comment.Nov 30 2017, 1:58 AM

Completed auto-reimage of hosts:

['labstore1007.wikimedia.org']

Of which those FAILED:

['labstore1007.wikimedia.org']
ops-monitoring-bot added a comment.Nov 30 2017, 9:48 PM

Script wmf-auto-reimage was launched by madhuvishy on neodymium.eqiad.wmnet for hosts:

['labstore1006.wikimedia.org']

The log can be found in /var/log/wmf-auto-reimage/201711302148_madhuvishy_18830.log.

ops-monitoring-bot added a comment.Nov 30 2017, 10:39 PM

Completed auto-reimage of hosts:

['labstore1006.wikimedia.org']

Of which those FAILED:

['labstore1006.wikimedia.org']
ArielGlenn mentioned this in T181895: Simplify the xml/sql cleanup jobs and deploy everywhere.Dec 3 2017, 3:08 PM
ArielGlenn moved this task from Backlog to Active on the Datasets-General-or-Unknown board.Dec 4 2017, 9:59 AM
ops-monitoring-bot added a comment.Dec 8 2017, 9:08 PM

Script wmf-auto-reimage was launched by madhuvishy on neodymium.eqiad.wmnet for hosts:

['labstore1007.wikimedia.org']

The log can be found in /var/log/wmf-auto-reimage/201712082107_madhuvishy_16564.log.

ops-monitoring-bot added a comment.Dec 8 2017, 9:08 PM

Completed auto-reimage of hosts:

['labstore1007.wikimedia.org']

Of which those FAILED:

['labstore1007.wikimedia.org']
gerritbot added a comment.Jan 2 2018, 2:50 AM

Change 401420 had a related patch set uploaded (by Madhuvishy; owner: Madhuvishy):
[operations/puppet@production] dumps: Set up NFS on the dumps distribution servers

https://gerrit.wikimedia.org/r/401420

gerritbot added a comment.Jan 2 2018, 2:56 AM

Change 401420 merged by Madhuvishy:
[operations/puppet@production] dumps: Set up NFS on the dumps distribution servers

https://gerrit.wikimedia.org/r/401420

gerritbot added a comment.Jan 11 2018, 11:18 PM

Change 403837 had a related patch set uploaded (by Madhuvishy; owner: Madhuvishy):
[operations/puppet@production] dumps_distribution: Set up initial NFS exports

https://gerrit.wikimedia.org/r/403837

gerritbot added a comment.Jan 11 2018, 11:26 PM

Change 403837 merged by Madhuvishy:
[operations/puppet@production] dumps_distribution: Set up initial NFS exports

https://gerrit.wikimedia.org/r/403837

ArielGlenn added a parent task: T182540: get datset1001, ms1001 ready for decommission.Feb 8 2018, 9:53 AM
bd808 edited projects, added cloud-services-team (Kanban), Data-Services; removed cloud-services-team (FY2017-18), Goal.Feb 19 2018, 2:13 AM
bd808 moved this task from Inbox to Doing on the cloud-services-team (Kanban) board.Feb 23 2018, 9:49 PM
bd808 moved this task from Backlog to Dumps on the Data-Services board.Mar 2 2018, 1:43 PM
gerritbot added a comment.Mar 15 2018, 11:44 PM

Change 419952 had a related patch set uploaded (by Madhuvishy; owner: Madhuvishy):
[operations/puppet@production] dumps: Add stat* to nfs exports

https://gerrit.wikimedia.org/r/419952

gerritbot added a comment.Mar 15 2018, 11:51 PM

Change 419952 merged by Madhuvishy:
[operations/puppet@production] dumps: Add stat* to nfs exports

https://gerrit.wikimedia.org/r/419952

• madhuvishy closed this task as Resolved.Mar 16 2018, 12:19 AM
• madhuvishy closed subtask T171540: Figure out how NFS failovers will work for the dumps servers - labstore1006|7 as Resolved.Mar 16 2018, 5:44 PM
bd808 moved this task from Doing to Done on the cloud-services-team (Kanban) board.Mar 19 2018, 9:46 PM
gerritbot added a comment.Mar 30 2018, 2:03 AM

Change 423081 had a related patch set uploaded (by Madhuvishy; owner: Madhuvishy):
[operations/puppet@production] dumps: Move nfs exports config to template

https://gerrit.wikimedia.org/r/423081

gerritbot added a comment.Mar 30 2018, 2:06 AM

Change 423081 merged by Madhuvishy:
[operations/puppet@production] dumps: Move nfs exports config to template

https://gerrit.wikimedia.org/r/423081

ArielGlenn moved this task from Active to Done on the Datasets-General-or-Unknown board.Jul 2 2018, 12:35 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL