Page MenuHomePhabricator
Create Task
Maniphest T201343

rack/setup/install mwmaint1002.eqiad.wmnet
Closed, ResolvedPublic
Actions

Description

This task will track the racking, setup, and installation of mwmaint1002.eqiad.wmnet.

Please note that mwmaint1001.eqiad.wmnet was a temporary reallocation of an image scaler. Once this new host is fully online and in service, a new task needs to be created (or existing linked) to track the return/reimage of mwmaint1001 back to image scaler duties.

Racking Proposal: Any 1G rack will do, this will have an internal vlan/ip.

mwmaint1002:

  • - receive in system on procurement task T195418
  • - rack system with proposed racking plan (see above) & update racktables (include all system info plus location)
  • - bios/drac/serial setup/testing
  • - mgmt dns entries added for both asset tag and hostname
  • - network port setup (description, enable, vlan)
    • end on-site specific steps
  • - production dns entries added
  • - operations/puppet update (install_server at minimum, other files if possible)
  • - OS installation
  • - puppet accept/initial run
  • - handoff for service implementation

Details

SubjectRepoBranchLines +/-
mariadb: remove mwmaint1001 from prod-m5 SQL grantsoperations/puppetproduction+3 -9
network::constants: remove mwmaint1001operations/puppetproduction+0 -2
mediawiki_maintenance: switch home rsync to 1002->2001operations/puppetproduction+2 -2
scap/tcpircbot: remove mwmaint1001 from scap and allowed hostsoperations/puppetproduction+0 -4
site: turn mwmaint1001 into a spare::systemoperations/puppetproduction+2 -2
mw_maintenance: let $motd_ensure be based on $ensure for warning motdoperations/puppetproduction+4 -3
mw_maintenance: temp hack to avoid duplicate crons on switch to eqiadoperations/puppetproduction+19 -3
cache::text: replace (commented) mwmaint1001 with mwmaint1002operations/puppetproduction+1 -1
trafficserver: replace mwmaint1001 with 1002 as noc.wm.org backendoperations/puppetproduction+1 -1
scap/dsh: add mwmaint1002 to mw scap hostsoperations/puppetproduction+1 -0
site/admin: replace mwmaint1001 with 1002 in commentsoperations/puppetproduction+3 -2
site: add mediawiki_maintenance role to mwmaint1002operations/puppetproduction+1 -1
mariadb: add mwmaint1002 to grants for production-m5operations/puppetproduction+9 -3
mediawiki_maintenance: allow rsyncing home dirs from 1001 to 1002operations/puppetproduction+1 -1
network::constants:: add mwmaint1002 to maintenance serversoperations/puppetproduction+2 -0
tcpircbot: add mwmaint1002 to allowed hostsoperations/puppetproduction+3 -0
add IPv6 records for mwmaint1002operations/dnsmaster+2 -0
add mwmaint1002 to site.pp with spare role and IPv6operations/puppetproduction+6 -0
DHCP: add mwmaint1002 MAC addressoperations/puppetproduction+5 -0
mwmaint1002: add prod DNS entries (v4)operations/dnsmaster+4 -3
Adding mgmt dns for newly racked serversoperations/dnsmaster+32 -0
Adding mgmt dns for several new serversoperations/dnsmaster+185 -153
Show related patches Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
Unknown Object (Task)
Unknown Object (Task)
 ResolvedDzahnT201343 rack/setup/install mwmaint1002.eqiad.wmnet
 ResolvedDzahnT192457 Reallocate former image scalers
 Resolved• CmjohnsonT215332 reimage WMF6937/mw1298

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
• Cmjohnson moved this task from Backlog to Racking Tasks on the ops-eqiad board.Aug 7 2018, 2:02 PM
• Cmjohnson updated the task description. (Show Details)Aug 8 2018, 3:03 PM
Dzahn awarded a token.Aug 8 2018, 5:21 PM
gerritbot subscribed.Aug 9 2018, 6:10 PM

Change 451680 had a related patch set uploaded (by Cmjohnson; owner: Cmjohnson):
[operations/dns@master] Adding mgmt dns for several new servers

https://gerrit.wikimedia.org/r/451680

gerritbot added a project: Patch-For-Review.Aug 9 2018, 6:10 PM
gerritbot added a comment.Aug 13 2018, 2:34 PM

Change 451680 abandoned by Cmjohnson:
Adding mgmt dns for several new servers

https://gerrit.wikimedia.org/r/451680

gerritbot added a comment.Aug 14 2018, 5:44 PM

Change 452718 had a related patch set uploaded (by Cmjohnson; owner: Cmjohnson):
[operations/dns@master] Adding mgmt dns for newly racked servers

https://gerrit.wikimedia.org/r/452718

gerritbot added a comment.Aug 14 2018, 5:46 PM

Change 452718 merged by Cmjohnson:
[operations/dns@master] Adding mgmt dns for newly racked servers

https://gerrit.wikimedia.org/r/452718

• Cmjohnson updated the task description. (Show Details)Aug 16 2018, 3:21 PM
• Cmjohnson reassigned this task from • Cmjohnson to RobH.Aug 21 2018, 3:54 PM
• Cmjohnson updated the task description. (Show Details)
• Cmjohnson moved this task from Racking Tasks to Blocked on the ops-eqiad board.

assigning to @RobH for installs

Dzahn added a comment.Sep 18 2018, 4:08 PM

I'll take this. We should do it while eqiad is still not active, that will make it a lot easier than doing it later after the switch back.

Dzahn claimed this task.Sep 18 2018, 4:12 PM
gerritbot added a comment.Sep 18 2018, 7:58 PM

Change 461202 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/dns@master] mwmaint1002: add prod DNS entries (v4)

https://gerrit.wikimedia.org/r/461202

gerritbot added a comment.Sep 18 2018, 8:04 PM

Change 461204 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] DHCP: add mwmaint1002 MAC address

https://gerrit.wikimedia.org/r/461204

gerritbot added a comment.Sep 18 2018, 11:11 PM

Change 461202 merged by Dzahn:
[operations/dns@master] mwmaint1002: add prod DNS entries (v4)

https://gerrit.wikimedia.org/r/461202

gerritbot added a comment.Sep 18 2018, 11:12 PM

Change 461204 merged by Dzahn:
[operations/puppet@production] DHCP: add mwmaint1002 MAC address

https://gerrit.wikimedia.org/r/461204

Dzahn updated the task description. (Show Details)Sep 18 2018, 11:13 PM
ops-monitoring-bot subscribed.Sep 19 2018, 12:26 AM

Script wmf-auto-reimage was launched by dzahn on neodymium.eqiad.wmnet for hosts:

['mwmaint1002.eqiad.wmnet']

The log can be found in /var/log/wmf-auto-reimage/201809190022_dzahn_9192.log.

Dzahn mentioned this in T204789: wmf-auto-reimage tries to remove from Debmonitor even with --new.Sep 19 2018, 12:30 AM
ops-monitoring-bot added a comment.Sep 19 2018, 12:43 AM

Completed auto-reimage of hosts:

['mwmaint1002.eqiad.wmnet']

and were ALL successful.

gerritbot added a comment.Sep 19 2018, 12:46 AM

Change 461261 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] add mwmaint1002 to site.pp with spare role

https://gerrit.wikimedia.org/r/461261

gerritbot added a comment.Sep 19 2018, 12:55 AM

Change 461261 merged by Dzahn:
[operations/puppet@production] add mwmaint1002 to site.pp with spare role and IPv6

https://gerrit.wikimedia.org/r/461261

gerritbot added a comment.Sep 19 2018, 4:56 PM

Change 461427 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/dns@master] add IPv6 records for mwmaint1002

https://gerrit.wikimedia.org/r/461427

gerritbot added a comment.Sep 19 2018, 5:15 PM

Change 461427 merged by Dzahn:
[operations/dns@master] add IPv6 records for mwmaint1002

https://gerrit.wikimedia.org/r/461427

Dzahn added a comment.Sep 19 2018, 10:59 PM

mwmaint1002.eqiad.wmnet has address 10.64.16.77
mwmaint1002.eqiad.wmnet has IPv6 address 2620:0:861:102:10:64:16:77

[mwmaint1001:~] $ host 2620:0:861:102:10:64:16:77
7.7.0.0.6.1.0.0.4.6.0.0.0.1.0.0.2.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa domain name pointer mwmaint1002.eqiad.wmnet.

[mwmaint1001:~] $ host 10.64.16.77
77.16.64.10.in-addr.arpa domain name pointer mwmaint1002.eqiad.wmnet.

[mwmaint1001:~] $ ping6 mwmaint1002.eqiad.wmnet
PING mwmaint1002.eqiad.wmnet(mwmaint1002.eqiad.wmnet (2620:0:861:102:10:64:16:77)) 56 data bytes
64 bytes from mwmaint1002.eqiad.wmnet (2620:0:861:102:10:64:16:77): icmp_seq=1 ttl=64 time=0.454 ms

[mwmaint1001:~] $ ping mwmaint1002.eqiad.wmnet
PING mwmaint1002.eqiad.wmnet(mwmaint1002.eqiad.wmnet (2620:0:861:102:10:64:16:77)) 56 data bytes
64 bytes from mwmaint1002.eqiad.wmnet (2620:0:861:102:10:64:16:77): icmp_seq=1 ttl=64 time=0.255 ms

gerritbot added a comment.Sep 19 2018, 11:03 PM

Change 461486 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] tcpircbot: add mwmaint1002 to allowed hosts

https://gerrit.wikimedia.org/r/461486

gerritbot added a comment.Sep 19 2018, 11:07 PM

Change 461486 merged by Dzahn:
[operations/puppet@production] tcpircbot: add mwmaint1002 to allowed hosts

https://gerrit.wikimedia.org/r/461486

gerritbot added a comment.Sep 19 2018, 11:07 PM

Change 461487 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] site/admin: replace mwmaint1001 with 1002 in comments

https://gerrit.wikimedia.org/r/461487

gerritbot added a comment.Sep 19 2018, 11:10 PM

Change 461488 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] network::constants:: add mwmaint1002 to maintenance servers

https://gerrit.wikimedia.org/r/461488

gerritbot added a comment.Sep 19 2018, 11:12 PM

Change 461488 merged by Dzahn:
[operations/puppet@production] network::constants:: add mwmaint1002 to maintenance servers

https://gerrit.wikimedia.org/r/461488

gerritbot added a comment.Sep 19 2018, 11:19 PM

Change 461489 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] scap/dsh: add mwmaint1002 to mw scap hosts

https://gerrit.wikimedia.org/r/461489

gerritbot added a comment.Sep 19 2018, 11:21 PM

Change 461490 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] trafficserver: replace mwmaint1001 with 1002 as noc.wm.org backend

https://gerrit.wikimedia.org/r/461490

gerritbot added a comment.Sep 19 2018, 11:23 PM

Change 461491 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] site: add mediawiki_maintenance role to mwmaint1002

https://gerrit.wikimedia.org/r/461491

gerritbot added a comment.Sep 19 2018, 11:23 PM

Change 461492 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] site: turn mwmaint1001 into a spare::system

https://gerrit.wikimedia.org/r/461492

gerritbot added a comment.Sep 19 2018, 11:28 PM

Change 461493 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] mariadb: add mwmaint1002 to grants for production-m5

https://gerrit.wikimedia.org/r/461493

gerritbot added a comment.Sep 19 2018, 11:36 PM

Change 461494 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] mediawiki_maintenance: allow rsyncing home dirs from 1001 to 1002

https://gerrit.wikimedia.org/r/461494

gerritbot added a comment.Sep 19 2018, 11:41 PM

Change 461494 merged by Dzahn:
[operations/puppet@production] mediawiki_maintenance: allow rsyncing home dirs from 1001 to 1002

https://gerrit.wikimedia.org/r/461494

Dzahn changed the task status from Open to Stalled.Sep 20 2018, 12:07 AM
Dzahn updated the task description. (Show Details)

for now blocked on https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/461496/

gerritbot added a comment.Sep 20 2018, 7:13 AM

Change 461493 merged by Jcrespo:
[operations/puppet@production] mariadb: add mwmaint1002 to grants for production-m5

https://gerrit.wikimedia.org/r/461493

gerritbot added a comment.Sep 20 2018, 6:02 PM

Change 461491 merged by Dzahn:
[operations/puppet@production] site: add mediawiki_maintenance role to mwmaint1002

https://gerrit.wikimedia.org/r/461491

Dzahn added a comment.Sep 20 2018, 10:13 PM

role applied, now blocked on puppet error due to missing cert for mcrouter in the private repo. How to generate these certs isn't documented yet. working on changing that

Stashbot subscribed.Sep 21 2018, 4:29 PM

Mentioned in SAL (#wikimedia-operations) [2018-09-21T16:29:36Z] <mutante> puppetmaster: running mcrouter_generate_certs to add an mcrouter cert for mwmaint1002 (T201343) https://wikitech.wikimedia.org/wiki/Mcrouter#Generate_certs_for_a_new_host

Stashbot added a comment.Sep 21 2018, 10:13 PM

Mentioned in SAL (#wikimedia-operations) [2018-09-21T22:13:38Z] <mutante> mwmaint1002 - re-enabling puppet, now that it has mcrouter certs, that works (T201343)

Dzahn changed the task status from Stalled to Open.Sep 21 2018, 10:14 PM
gerritbot added a comment.Sep 21 2018, 10:37 PM

Change 461487 merged by Dzahn:
[operations/puppet@production] site/admin: replace mwmaint1001 with 1002 in comments

https://gerrit.wikimedia.org/r/461487

gerritbot added a comment.Sep 21 2018, 10:46 PM

Change 461489 merged by Dzahn:
[operations/puppet@production] scap/dsh: add mwmaint1002 to mw scap hosts

https://gerrit.wikimedia.org/r/461489

gerritbot added a comment.Sep 21 2018, 11:41 PM

Change 462036 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] cache::text: replace (commented) mwmaint1001 with mwmaint1002

https://gerrit.wikimedia.org/r/462036

Stashbot added a comment.Sep 21 2018, 11:47 PM

Mentioned in SAL (#wikimedia-operations) [2018-09-21T23:47:15Z] <mutante> mwmaint1002 - rsyncing home dirs over from mwmaint1001 which synced them from terbium. still large old files from terbium here, like a NASA video :) (T201343)

Dzahn raised the priority of this task from Medium to High.Sep 21 2018, 11:52 PM
• MoritzMuehlenhoff added a comment.Sep 24 2018, 12:17 PM

Disk space on the root partition of mwmaint1002 is depleted, which results in failing puppet runs

Dzahn added a comment.Sep 24 2018, 3:02 PM

the new root partition is smaller than before on mwmaint1001 and also terbium. So copying home dirs from there would not work anymore, not enough space. Fixed by deleting a large 60G videos directory

Stashbot added a comment.Sep 25 2018, 12:38 AM

Mentioned in SAL (#wikimedia-operations) [2018-09-25T00:38:05Z] <mutante> mwmaint1002 - created /var/run/nutcracker dir and fixed permissions on it, then started nutcracker with systemctl. this fixed icinga alerts T201343

gerritbot added a comment.Sep 25 2018, 10:46 PM

Change 461490 merged by Dzahn:
[operations/puppet@production] trafficserver: replace mwmaint1001 with 1002 as noc.wm.org backend

https://gerrit.wikimedia.org/r/461490

Dzahn added a comment.Sep 26 2018, 1:32 AM

The nutcracker issue is T204450 and could have also been solved by rebooting or applying the role without using spare first.

gerritbot added a comment.Sep 28 2018, 9:20 PM

Change 462036 merged by Dzahn:
[operations/puppet@production] cache::text: replace (commented) mwmaint1001 with mwmaint1002

https://gerrit.wikimedia.org/r/462036

gerritbot added a comment.Sep 28 2018, 9:42 PM

Change 463563 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] mw_maintenance: temp hack to avoid duplicate crons on switch to eqiad

https://gerrit.wikimedia.org/r/463563

Dzahn added a comment.Sep 28 2018, 9:45 PM

This should be all done now. The last open change is the one above to add a temp hack to avoid that both mwmaint servers in eqiad become activated at the same time when we switch back to eqiad.

Or we could remove mwmaint1001 completely before.. but i thought this way is nicer in case there is an unexpected issue with mwmaint1002. then mwmaint1001 would still be available to use and not already decom'ed.

gerritbot added a comment.Oct 5 2018, 8:04 PM

Change 463563 merged by Dzahn:
[operations/puppet@production] mw_maintenance: temp hack to avoid duplicate crons on switch to eqiad

https://gerrit.wikimedia.org/r/463563

Dzahn added a comment.Oct 5 2018, 8:08 PM

This server will become active when we switch Mediawiki back to eqiad on October 10th.

Dzahn edited projects, added Datacenter-Switchover; removed Patch-For-Review.Oct 5 2018, 8:10 PM
gerritbot added a comment.Oct 10 2018, 3:29 PM

Change 465646 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] mw_maintenance: use $ensure, not $motd_ensure for warning motd

https://gerrit.wikimedia.org/r/465646

gerritbot added a project: Patch-For-Review.Oct 10 2018, 3:29 PM
gerritbot added a comment.Oct 10 2018, 3:43 PM

Change 465646 merged by Dzahn:
[operations/puppet@production] mw_maintenance: let $motd_ensure be based on $ensure for warning motd

https://gerrit.wikimedia.org/r/465646

gerritbot added a comment.Oct 10 2018, 6:55 PM

Change 461492 merged by Dzahn:
[operations/puppet@production] site: turn mwmaint1001 into a spare::system

https://gerrit.wikimedia.org/r/461492

gerritbot added a comment.Oct 10 2018, 7:03 PM

Change 465681 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] scap/tcpircbot: remove mwmaint1001 from scap and allowed hosts

https://gerrit.wikimedia.org/r/465681

Dzahn added a subtask: T192457: Reallocate former image scalers.Oct 10 2018, 7:04 PM
gerritbot added a comment.Oct 10 2018, 7:14 PM

Change 465681 merged by Dzahn:
[operations/puppet@production] scap/tcpircbot: remove mwmaint1001 from scap and allowed hosts

https://gerrit.wikimedia.org/r/465681

gerritbot added a comment.Oct 10 2018, 7:22 PM

Change 465685 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] mariadb: remove mwmaint1001 from prod-m5 SQL grants

https://gerrit.wikimedia.org/r/465685

gerritbot added a comment.Oct 10 2018, 7:26 PM

Change 465686 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] network::constants: remove mwmaint1001

https://gerrit.wikimedia.org/r/465686

akosiaris subscribed.Oct 11 2018, 5:13 PM

@Dzahn anything left here ?

Dzahn added a comment.Oct 11 2018, 6:18 PM

Yes.

mariadb: remove mwmaint1001 from prod-m5 SQL grants - https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/465645/
network::constants: remove mwmaint1001 - https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/465686/
Revert "rename wmf6936 from mw1297 to mwmaint1001" - https://gerrit.wikimedia.org/r/#/c/operations/dns/+/465689/
Revert "mw_maintenance: temp hack to avoid duplicate crons on switch to eqiad" https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/465645/

But i can pretty much self-merge those except the mariadb one that needs deployment. Some need rebasing.

And then there is renaming mwmaint1001 in DHCP and DNS and reinstalling it.

(Yea, i could have done this in a separate task but already started doing both here, 1002 setup and removing 1001.)

Marostegui added subscribers: • Banyek, Marostegui.Oct 11 2018, 6:21 PM
In T201343#4659214, @Dzahn wrote:

Yes.

mariadb: remove mwmaint1001 from prod-m5 SQL grants - https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/465645/
But i can pretty much self-merge those except the mariadb one that needs deployment. Some need rebasing.

Regarding that one, we were added to the patch yesterday EU night, and today we've had two pretty massive incidents, so we have not been able to deal with this, probably we won't be able to do so till late next week once things have stabilized.

Dzahn added a comment.Oct 11 2018, 6:32 PM

There is no urgency at all and it wasn't expected. I only listed what is left to be done. Please dont worry about this at all, especially in the current situation.

Dzahn lowered the priority of this task from High to Medium.Oct 11 2018, 6:34 PM

lowering priority because mwmaint1002 is in production and the remaining steps are all just cleanup

Dzahn closed this task as Resolved.Oct 11 2018, 6:39 PM

Ok, let's keep the ticket within the original focus.. setting up mwmaint1002. That is done.

Normally there would be a separate "decom mwmaint1001" ticket but in this case there is only reclaiming it to be an appserver again. And that is already T192457.

So nevermind. resolving here.

Dzahn removed a project: Patch-For-Review.Oct 11 2018, 6:40 PM
Dzahn updated the task description. (Show Details)
gerritbot added a comment.Oct 11 2018, 8:35 PM

Change 466731 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] mediawiki_maintenance: switch home rsync to 1002->2001

https://gerrit.wikimedia.org/r/466731

gerritbot added a project: Patch-For-Review.Oct 11 2018, 8:35 PM
gerritbot added a comment.Oct 11 2018, 9:30 PM

Change 466731 merged by Dzahn:
[operations/puppet@production] mediawiki_maintenance: switch home rsync to 1002->2001

https://gerrit.wikimedia.org/r/466731

gerritbot added a comment.Oct 12 2018, 7:04 PM

Change 465686 merged by Dzahn:
[operations/puppet@production] network::constants: remove mwmaint1001

https://gerrit.wikimedia.org/r/465686

gerritbot added a comment.Oct 19 2018, 7:19 AM

Change 465685 merged by Marostegui:
[operations/puppet@production] mariadb: remove mwmaint1001 from prod-m5 SQL grants

https://gerrit.wikimedia.org/r/465685

Stashbot added a comment.Oct 19 2018, 7:20 AM

Mentioned in SAL (#wikimedia-operations) [2018-10-19T07:20:35Z] <marostegui> Remove mwmaint1001 grants from m5 - https://phabricator.wikimedia.org/T201343 https://phabricator.wikimedia.org/T192457

Marostegui added a comment.Oct 19 2018, 7:53 AM
In T201343#4659214, @Dzahn wrote:

Yes.

mariadb: remove mwmaint1001 from prod-m5 SQL grants - https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/465645/

Change merged and grants removed on MySQL.
Thanks for the patience!

Dzahn changed the status of subtask T192457: Reallocate former image scalers from Open to Stalled.Apr 24 2019, 6:49 PM
Dzahn closed subtask T192457: Reallocate former image scalers as Resolved.Sep 19 2019, 7:00 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits