Page MenuHomePhabricator
Create Task
Maniphest T243027

replacement for gerrit2001, decom gerrit2001
Closed, ResolvedPublic
Actions

Description

gerrit2001 is out of warranty

https://netbox.wikimedia.org/dcim/devices/315/

Papaul points out if something happens to it it would take a while to fix, if possible at all.

Also we don't keep spares around anymore.

That combined with the policy to keep things around after warranty expires seems suboptimal.

Can we get new hardware for it or have general spares around again please?

<time passes> <hardware request was granted, new hardware was delivered> T299081 <hardware was racked and got an OS> T299575 <time passes>

The new hardware has been installed and is named gerrit2002.

At this time we have these Gerrit hosts

HostRoleHardware status
gerrit1001Gerrit PrimaryOK
gerrit2001Gerrit ReplicaOut of warranty
gerrit2002Gerrit Replica (new)New

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
gerrit: drop gerrit-replica-new.wikimedia.org from list of replicasoperations/puppetproduction+0 -1
gerrit: remove gerrit1001 from site and gerrit2001 hiera dataoperations/puppetproduction+0 -10
Remove gerrit2001 from deployment targetsoperations/software/gerritdeploy/wmf/stable-3.4+0 -1
site: remove gerrit2001, merge gerrit1001/2002 regexoperations/puppetproduction+1 -4
gerrit: decom gerrit2001operations/puppetproduction+0 -5
gerrit: remove hiera data for old replicaoperations/puppetproduction+0 -10
gerrit: switch gerrit-replica to new machine, remove replica-newoperations/dnsmaster+4 -8
gerit: remove old replica on gerrit2001 from gerrit configoperations/puppetproduction+0 -10
Update the known host key for gerrit2002.wikimedia.orgoperations/puppetproduction+2 -1
gerrit: add hiera data for a second replicaoperations/puppetproduction+11 -0
acme_chief: add gerrit-replica-new to SNI listoperations/puppetproduction+1 -0
Add gerrit2002.wikimedia.org to scap targets listoperations/software/gerritdeploy/wmf/stable-3.4+1 -0
gerrit: add hiera settings and IP for new replica gerrit2002operations/puppetproduction+5 -0
add gerrit-replica-new.wikimedia.org, point to 208.80.153.104operations/dnsmaster+4 -0
gerrit: turn gerrit2002 into a gerrit migration dest hostoperations/puppetproduction+15 -3
gerrit: add gerrit2002 to puppetized known_hosts fileoperations/puppetproduction+1 -0
gerrit: add gerrit2002 to firewall rules for cluster supportoperations/puppetproduction+1 -0
acme_chief: add gerrit2002 to hosts allowed to fetch TLS certsoperations/puppetproduction+1 -0
admin/gerrit: add gerrit shell admins on gerrit2002operations/puppetproduction+4 -0
add gerrit-replica-new secondary/service IPsoperations/dnsmaster+4 -1
Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Dzahn triaged this task as Medium priority.Jan 16 2020, 9:53 PM
Dzahn updated the task description. (Show Details)
Dzahn added a project: Gerrit.
• Marostegui subscribed.Sep 24 2021, 10:07 AM

@Dzahn did this ever happen? Or should we maybe ping specific people to make it happen?

Dzahn added subscribers: • wkandek, wiki_willy.Sep 24 2021, 3:20 PM
Dzahn added a comment.Sep 24 2021, 4:15 PM

@Marostegui No, it's still open. I'll follow-up on it soon.

Dzahn claimed this task.Sep 24 2021, 4:16 PM
Dzahn added a project: serviceops.Jan 5 2022, 10:10 PM
Dzahn added subscribers: LSobanski, akosiaris.Jan 5 2022, 10:14 PM

@LSobanski @akosiaris This one is for consideration in a manager meeting with releng.

dcaro mentioned this in T272559: Unused puppet resources audit, 2021.Jan 7 2022, 9:35 AM
Dzahn added a subtask: Unknown Object (Task).Jan 13 2022, 8:59 PM

There is now the new procurement ticket T299081

Papaul closed subtask Unknown Object (Task) as Resolved.Feb 15 2022, 6:40 PM
hashar added subscribers: dancy, brennen, thcipriani, hashar.Feb 23 2022, 4:33 PM

gerrit2001.wikimedia.org is a replica and can also be used as a spare to switch the primary service. It also serves repos over gerrit-replica.wikimedia.org which is used by various scripts and robot that mass pull from all repositories for various use cases.

To replace the server, we can add the new one as a 2nd replica and have the repositories replicated there (should take a few hours at max). Then we can switch the DNS entry for gerrit-replica.wikimedia.org to that new host and once validated we can unconfigure gerrit2001 and decommissioned it.

The migration would require two public IPv4 addresses (one for the server, one for the Gerrit service).

@brennen , @thcipriani and @dancy also have some knowledge about our Gerrit setup.

TLDR: replacing the server should not be too complicated.

gerritbot added a comment.May 16 2022, 9:14 PM

Change 792281 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/dns@master] add gerrit-replica-new secondary/service IPs

https://gerrit.wikimedia.org/r/792281

gerritbot added a project: Patch-For-Review.May 16 2022, 9:14 PM
Dzahn added a comment.May 16 2022, 9:59 PM
In T243027#7732585, @hashar wrote:

To replace the server, we can add the new one as a 2nd replica and have the repositories replicated there (should take a few hours at max). Then we can switch the DNS entry for gerrit-replica.wikimedia.org to that new host and once validated we can unconfigure gerrit2001 and decommissioned it.

The migration would require two public IPv4 addresses (one for the server, one for the Gerrit service).

I tried to add the new DNS name gerrit-replica-new.wikimedia.org which nowdays means a process similar but not identical to https://wikitech.wikimedia.org/wiki/DNS/Netbox#How_to_manually_allocate_a_special_purpose_IP_address_in_Netbox

Unfortunately it turned out that these are special cases which can't be added manual only in DNS but als not only in netbox and IPs can only migrate within the same row ... and gerrit2002 was not installed in the same row as gerrit2001.

So now for this plan to work we have to ask dcops to move the physical host around it seems.

gerritbot added a comment.May 16 2022, 10:01 PM

Change 792281 abandoned by Dzahn:

[operations/dns@master] add gerrit-replica-new secondary/service IPs

Reason:

gerrit2002 is in the wrong row

https://gerrit.wikimedia.org/r/792281

Dzahn mentioned this in T299575: Q3:(Need By: TBD) rack/setup/install contint2002, gerrit2002.May 16 2022, 10:05 PM
Dzahn added a comment.May 16 2022, 10:25 PM

reopened T299575 to move the host around

Maintenance_bot removed a project: Patch-For-Review.May 16 2022, 10:30 PM
Dzahn added a project: collaboration-services.Jun 27 2022, 9:57 PM
Dzahn moved this task from Incoming to Gerrit on the collaboration-services board.Jul 9 2022, 3:19 AM
thcipriani added a project: Release-Engineering-Team (The Decommission Mission 💀).Jul 18 2022, 6:04 PM
thcipriani mentioned this in T313250: Bring up gerrit2002.Jul 18 2022, 6:09 PM
dancy updated the task description. (Show Details)Jul 18 2022, 7:20 PM
Dzahn changed the task status from Open to In Progress.Jul 19 2022, 3:27 AM
Dzahn raised the priority of this task from Medium to High.
Dzahn updated the task description. (Show Details)Jul 19 2022, 3:33 AM
gerritbot added a comment.Jul 20 2022, 12:28 AM

Change 815395 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/dns@master] add gerrit-replica-new.wikimedia.org, point to 208.80.153.109

https://gerrit.wikimedia.org/r/815395

gerritbot added a project: Patch-For-Review.Jul 20 2022, 12:28 AM
gerritbot added a comment.Jul 20 2022, 12:56 AM

Change 815396 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/puppet@production] gerrit: add gerrit role and hiera settings for replica to gerrit2002

https://gerrit.wikimedia.org/r/815396

gerritbot added a comment.Jul 20 2022, 1:01 AM

Change 815397 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/puppet@production] acme_chief: add gerrit2002 to hosts allowed to fetch TLS certs

https://gerrit.wikimedia.org/r/815397

gerritbot added a comment.Jul 20 2022, 1:08 AM

Change 815398 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/puppet@production] gerrit: add gerrit2002 to firewall rules for cluster support

https://gerrit.wikimedia.org/r/815398

gerritbot added a comment.Jul 20 2022, 1:10 AM

Change 815400 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/puppet@production] gerrit: add gerrit2002 to puppetized known_hosts file

https://gerrit.wikimedia.org/r/815400

gerritbot added a comment.Jul 20 2022, 1:15 AM

Change 815401 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/puppet@production] gerrit: add hiera data for a second replica

https://gerrit.wikimedia.org/r/815401

gerritbot added a comment.Jul 20 2022, 1:31 AM

Change 815402 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/puppet@production] admin/gerrit: add gerrit shell admins on gerrit2002

https://gerrit.wikimedia.org/r/815402

gerritbot added a comment.Jul 20 2022, 1:23 PM

Change 815402 merged by Vgutierrez:

[operations/puppet@production] admin/gerrit: add gerrit shell admins on gerrit2002

https://gerrit.wikimedia.org/r/815402

gerritbot added a comment.Jul 20 2022, 7:45 PM

Change 815397 merged by Dzahn:

[operations/puppet@production] acme_chief: add gerrit2002 to hosts allowed to fetch TLS certs

https://gerrit.wikimedia.org/r/815397

gerritbot added a comment.Jul 20 2022, 8:36 PM

Change 815398 merged by Dzahn:

[operations/puppet@production] gerrit: add gerrit2002 to firewall rules for cluster support

https://gerrit.wikimedia.org/r/815398

gerritbot added a comment.Jul 20 2022, 9:14 PM

Change 815400 merged by Dzahn:

[operations/puppet@production] gerrit: add gerrit2002 to puppetized known_hosts file

https://gerrit.wikimedia.org/r/815400

hashar moved this task from Backlog to In progress on the Release-Engineering-Team (The Decommission Mission 💀) board.Jul 21 2022, 8:23 PM
gerritbot added a comment.Jul 27 2022, 5:44 PM

Change 817841 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/puppet@production] gerrit: add gerrit2002 to list of migration dest hosts

https://gerrit.wikimedia.org/r/817841

gerritbot added a comment.Jul 28 2022, 4:52 PM

Change 817841 merged by Dzahn:

[operations/puppet@production] gerrit: turn gerrit2002 into a gerrit migration dest host

https://gerrit.wikimedia.org/r/817841

Dzahn added a comment.Jul 28 2022, 5:52 PM

on gerrit2002 we now have, created by the migration class:

  • a group "gerrit2"
  • a user "gerrit2"
  • a directory /srv/gerrit
  • package rsync installed, /etc/default/rsync configured
  • 3 rsync fragments created: frag-gerrit-data frag-gerrit-home frag-gerrit-var-lib that allow syncing specific paths
  • /srv/home-gerrit1001.wikimedia.org/ created
  • host added to gerrit "contacts"

The 3 rsync modules (config fragments) are:

  • gerrit-data: path: /srv/gerrit
  • gerrit-home: path: /srv/home-gerrit1001.wikimedia.org
  • gerrit-var-lib: /var/lib/gerrit2/review_site

The hosts_allow line is gerrit1001.wikimedia.org localhost for all of them.

Finally the firewall rule that has been created via puppet->ferm->iptables is ACCEPT tcp -- gerrit1001.wikimedia.org anywhere tcp dpt:rsync.

This means an rsyncd is listening on the new host and is ready to have data pushed to it.. from the prod gerrit server gerrit1001. (with the current settings).

Dzahn added a subtask: T313972: Add gerrit2002 as a replica of gerrit1001.Jul 28 2022, 5:53 PM
Stashbot added a comment.Jul 28 2022, 6:28 PM

Mentioned in SAL (#wikimedia-operations) [2022-07-28T18:28:19Z] <mutante> gerrit: rsyncing /home from prod gerrit1001 to /srv/home-gerrit1001.wikimedia.org on gerrit2002 new replica T243027 T313250

Dzahn changed the status of subtask T313972: Add gerrit2002 as a replica of gerrit1001 from Open to In Progress.Jul 29 2022, 7:09 PM
gerritbot added a comment.Jul 29 2022, 8:16 PM

Change 815395 merged by Dzahn:

[operations/dns@master] add gerrit-replica-new.wikimedia.org, point to 208.80.153.104

https://gerrit.wikimedia.org/r/815395

Dzahn added a comment.Jul 29 2022, 8:19 PM

new in DNS:

[authdns1001:~] $ host gerrit-replica-new.wikimedia.org
gerrit-replica-new.wikimedia.org has address 208.80.153.104
gerrit-replica-new.wikimedia.org has IPv6 address 2620:0:860:4:208:80:153:104

gerritbot added a comment.Jul 29 2022, 8:28 PM

Change 815396 merged by Dzahn:

[operations/puppet@production] gerrit: add hiera settings and IP for new replica gerrit2002

https://gerrit.wikimedia.org/r/815396

Dzahn added a comment.Jul 29 2022, 8:53 PM

T313972#8116463

ops-monitoring-bot added a comment.Aug 2 2022, 8:36 PM

Cookbook cookbooks.sre.hosts.reimage was started by dzahn@cumin2002 for host gerrit2002.wikimedia.org with OS buster

Stashbot added a comment.Aug 2 2022, 8:38 PM

Mentioned in SAL (#wikimedia-operations) [2022-08-02T20:38:01Z] <mutante> re-imaging gerrit2002 with buster - because it's on bullseye, needs git-fat and that has not been ported to python3 yet which blocks upgrading gerrit machines otherwise T313250 T243027 T279509

Stashbot mentioned this in T279509: git-fat replacement/removal.Aug 2 2022, 8:38 PM
ops-monitoring-bot added a comment.Aug 2 2022, 9:11 PM

Cookbook cookbooks.sre.hosts.reimage started by dzahn@cumin2002 for host gerrit2002.wikimedia.org with OS buster completed:

  • gerrit2002 (PASS)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present
    • Deleted any existing Puppet certificate
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Host up (new fresh buster OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202208022036_dzahn_1022103_gerrit2002.out
    • Checked BIOS boot parameters are back to normal
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
gerritbot added a comment.Aug 2 2022, 10:14 PM

Change 819760 had a related patch set uploaded (by Ahmon Dancy; author: Ahmon Dancy):

[operations/software/gerrit@deploy/wmf/stable-3.4] Add gerrit2002.wikimedia.org to scap targets list

https://gerrit.wikimedia.org/r/819760

gerritbot added a comment.Aug 2 2022, 10:23 PM

Change 819760 merged by jenkins-bot:

[operations/software/gerrit@deploy/wmf/stable-3.4] Add gerrit2002.wikimedia.org to scap targets list

https://gerrit.wikimedia.org/r/819760

gerritbot added a comment.Aug 3 2022, 7:44 PM

Change 820185 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/puppet@production] acme_chief: add gerrit-replica-new to SNI list

https://gerrit.wikimedia.org/r/820185

gerritbot added a comment.Aug 3 2022, 7:46 PM

Change 820185 merged by Dzahn:

[operations/puppet@production] acme_chief: add gerrit-replica-new to SNI list

https://gerrit.wikimedia.org/r/820185

gerritbot added a comment.Aug 3 2022, 8:05 PM

Change 815401 merged by Dzahn:

[operations/puppet@production] gerrit: add hiera data for a second replica

https://gerrit.wikimedia.org/r/815401

Dzahn renamed this task from replacement for gerrit2001 to replacement for gerrit2001, decom gerrit2001.Aug 4 2022, 12:18 AM
gerritbot added a comment.Aug 4 2022, 12:33 AM

Change 820248 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/puppet@production] gerrit: decom gerrit2001

https://gerrit.wikimedia.org/r/820248

gerritbot added a comment.Aug 4 2022, 12:35 AM

Change 820249 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/puppet@production] gerrit: remove hiera data for old replica

https://gerrit.wikimedia.org/r/820249

gerritbot added a comment.Aug 4 2022, 12:36 AM

Change 820250 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/puppet@production] site: remove gerrit2001, merge gerrit1001/2002 regex

https://gerrit.wikimedia.org/r/820250

gerritbot added a comment.Aug 4 2022, 3:45 PM

Change 820474 had a related patch set uploaded (by Ahmon Dancy; author: Ahmon Dancy):

[operations/puppet@production] Update the known host key for gerrit2002.wikimedia.org

https://gerrit.wikimedia.org/r/820474

gerritbot added a comment.Aug 4 2022, 4:08 PM

Change 820474 merged by Jbond:

[operations/puppet@production] Update the known host key for gerrit2002.wikimedia.org

https://gerrit.wikimedia.org/r/820474

gerritbot added a comment.Aug 4 2022, 10:28 PM

Change 820573 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/puppet@production] gerit: remove old replica on gerrit2001 from gerrit config

https://gerrit.wikimedia.org/r/820573

gerritbot added a comment.Aug 4 2022, 11:02 PM

Change 820577 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/dns@master] gerrit: switch gerrit-replica to new machine, remove replica-new

https://gerrit.wikimedia.org/r/820577

gerritbot added a comment.Aug 4 2022, 11:05 PM

Change 820577 merged by Dzahn:

[operations/dns@master] gerrit: switch gerrit-replica to new machine, remove replica-new

https://gerrit.wikimedia.org/r/820577

gerritbot added a comment.Aug 5 2022, 12:09 AM

Change 820573 merged by Dzahn:

[operations/puppet@production] gerit: remove old replica on gerrit2001 from gerrit config

https://gerrit.wikimedia.org/r/820573

gerritbot added a comment.Aug 5 2022, 12:22 AM

Change 820249 abandoned by Dzahn:

[operations/puppet@production] gerrit: remove hiera data for old replica

Reason:

duplicate

https://gerrit.wikimedia.org/r/820249

gerritbot added a comment.Aug 5 2022, 12:33 AM

Change 820248 merged by Dzahn:

[operations/puppet@production] gerrit: decom gerrit2001

https://gerrit.wikimedia.org/r/820248

ops-monitoring-bot added a comment.Aug 10 2022, 4:32 PM

cookbooks.sre.hosts.decommission executed by dzahn@cumin2002 for hosts: gerrit2001.wikimedia.org

  • gerrit2001.wikimedia.org (PASS)
    • Downtimed host on Icinga/Alertmanager
    • Found physical host
    • Downtimed management interface on Icinga/Alertmanager
    • Wiped all swraid, partition-table and filesystem signatures
    • Powered off
    • [Netbox] Set status to Decommissioning, deleted all non-mgmt IPs, updated switch interfaces (disabled, removed vlans, etc
    • Configured the linked switch interface(s)
    • Removed from DebMonitor
    • Removed from Puppet master and PuppetDB
ops-monitoring-bot added a comment.Aug 10 2022, 5:12 PM

cookbooks.sre.hosts.decommission executed by dzahn@cumin2002 for hosts: gerrit2001.wikimedia.org

  • gerrit2001.wikimedia.org (FAIL)
    • Host not found on Icinga, unable to downtime it
    • Found physical host
    • Management interface not found on Icinga, unable to downtime it
    • Unable to connect to the host, wipe of swraid, partition-table and filesystem signatures will not be performed: Cumin execution failed (exit_code=2)
    • No DNS record found for the mgmt interface gerrit2001.mgmt.codfw.wmnet, trying the asset tag one: wmf6408.mgmt.codfw.wmnet
    • Powered off
    • [Netbox] Set status to Decommissioning, deleted all non-mgmt IPs, updated switch interfaces (disabled, removed vlans, etc
    • Configured the linked switch interface(s)
    • Removed from DebMonitor
    • Removed from Puppet master and PuppetDB

ERROR: some step on some host failed, check the bolded items above

gerritbot added a comment.Aug 10 2022, 5:27 PM

Change 820250 merged by Dzahn:

[operations/puppet@production] site: remove gerrit2001, merge gerrit1001/2002 regex

https://gerrit.wikimedia.org/r/820250

Dzahn closed this task as Resolved.Aug 10 2022, 5:45 PM

gerrit2002 is production https://gerrit-replica.wikimedia.org

gerrit2001 is shut down and fully decom'ed.

Dzahn closed subtask T313972: Add gerrit2002 as a replica of gerrit1001 as Resolved.Aug 10 2022, 5:46 PM
Dzahn mentioned this in T315040: decommission gerrit2001.wikimedia.org (dcops).Aug 11 2022, 6:19 PM
Dzahn added a comment.Aug 11 2022, 6:28 PM

This is now handed over to dcops for physical decom steps and continues still at T315040.

gerritbot added a comment.Oct 6 2022, 2:04 PM

Change 839551 had a related patch set uploaded (by Hashar; author: Hashar):

[operations/software/gerrit@deploy/wmf/stable-3.4] Remove gerrit2001 from deployment targets

https://gerrit.wikimedia.org/r/839551

gerritbot added a comment.Oct 6 2022, 2:05 PM

Change 839551 merged by jenkins-bot:

[operations/software/gerrit@deploy/wmf/stable-3.4] Remove gerrit2001 from deployment targets

https://gerrit.wikimedia.org/r/839551

Dzahn mentioned this in T323262: gerrit1001 running out of space on /.Nov 17 2022, 6:46 PM
gerritbot added a comment.Jun 21 2023, 10:28 PM

Change 932021 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/puppet@production] gerrit: remove gerrit1001 from site and gerrit2001 hiera data

https://gerrit.wikimedia.org/r/932021

gerritbot added a comment.Jun 21 2023, 10:30 PM

Change 932021 merged by Dzahn:

[operations/puppet@production] gerrit: remove gerrit1001 from site and gerrit2001 hiera data

https://gerrit.wikimedia.org/r/932021

Maintenance_bot removed a project: Patch-For-Review.Jun 21 2023, 11:11 PM
gerritbot added a comment.Jul 25 2024, 6:00 PM

Change #1056996 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/puppet@production] gerrit: drop gerrit-replica-new.wikimedia.org from list of replicas

https://gerrit.wikimedia.org/r/1056996

gerritbot added a project: Patch-For-Review.Jul 25 2024, 6:00 PM
gerritbot added a comment.Jul 29 2024, 5:05 PM

Change #1056996 merged by Dzahn:

[operations/puppet@production] gerrit: drop gerrit-replica-new.wikimedia.org from list of replicas

https://gerrit.wikimedia.org/r/1056996

Maintenance_bot removed a project: Patch-For-Review.Jul 29 2024, 5:31 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits