Page MenuHomePhabricator
Create Task
Maniphest T313983

Q1:rack/setup/install cloudvirt10[54-61].eqiad.wmnet
Closed, ResolvedPublic
Actions

Description

This task will track the racking, setup, and OS installation of cloudvirt10[54-61].eqiad.wmnet

Hostname / Racking / Installation Details

Hostnames: cloudvirt10[54-61].eqiad.wmnet
Racking Proposal: Place in WMCS racks. They can be placed anywhere and do not need to be spread across rows. Can be racked next to other cloudvirt*. Row E/F Ok
Networking Setup: 2 10G connections per server. Requires cloud-hosts1-eqiad VLAN. (same as existing cloudvirts)
Partitioning/Raid: 2dev standard raid1 ('echo partman/standard.cfg partman/raid1-2dev.cfg ', see attached patch)
OS Distro: Bullseye
Sub-team Technical Contact: Andrew Bogott

Per host setup checklist

Each host should have its own setup checklist copied and pasted into the list below.

cloudvirt1054:
  • - receive in system on procurement task T311863 & in coupa
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
  • - add mgmt dns (asset tag and hostname) and production dns entries in netbox, run cookbook sre.dns.netbox.
  • - network port setup via netbox, run homer from an active cumin host to commit
  • - bios/drac/serial setup/testing, see Lifecycle Steps & Automatic BIOS setup details
  • - firmware update (idrac, bios, network, raid controller)
  • - operations/puppet update - this should include updates to netboot.pp, and site.pp role(insetup) or cp systems use role(insetup::nofirm).
  • - OS installation & initital puppet run via sre.hosts.reimage cookbook.
cloudvirt1055:
  • - receive in system on procurement task T311863 & in coupa
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
  • - add mgmt dns (asset tag and hostname) and production dns entries in netbox, run cookbook sre.dns.netbox.
  • - network port setup via netbox, run homer from an active cumin host to commit
  • - bios/drac/serial setup/testing, see Lifecycle Steps & Automatic BIOS setup details
  • - firmware update (idrac, bios, network, raid controller)
  • - operations/puppet update - this should include updates to netboot.pp, and site.pp role(insetup) or cp systems use role(insetup::nofirm).
  • - OS installation & initital puppet run via sre.hosts.reimage cookbook.
cloudvirt1056:
  • - receive in system on procurement task T311863 & in coupa
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
  • - add mgmt dns (asset tag and hostname) and production dns entries in netbox, run cookbook sre.dns.netbox.
  • - network port setup via netbox, run homer from an active cumin host to commit
  • - bios/drac/serial setup/testing, see Lifecycle Steps & Automatic BIOS setup details
  • - firmware update (idrac, bios, network, raid controller)
  • - operations/puppet update - this should include updates to netboot.pp, and site.pp role(insetup) or cp systems use role(insetup::nofirm).
  • - OS installation & initital puppet run via sre.hosts.reimage cookbook.
cloudvirt1057:
  • - receive in system on procurement task T311863 & in coupa
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
  • - add mgmt dns (asset tag and hostname) and production dns entries in netbox, run cookbook sre.dns.netbox.
  • - network port setup via netbox, run homer from an active cumin host to commit
  • - bios/drac/serial setup/testing, see Lifecycle Steps & Automatic BIOS setup details
  • - firmware update (idrac, bios, network, raid controller)
  • - operations/puppet update - this should include updates to netboot.pp, and site.pp role(insetup) or cp systems use role(insetup::nofirm).
  • - OS installation & initital puppet run via sre.hosts.reimage cookbook.
cloudvirt1058:
  • - receive in system on procurement task T311863 & in coupa
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
  • - add mgmt dns (asset tag and hostname) and production dns entries in netbox, run cookbook sre.dns.netbox.
  • - network port setup via netbox, run homer from an active cumin host to commit
  • - bios/drac/serial setup/testing, see Lifecycle Steps & Automatic BIOS setup details
  • - firmware update (idrac, bios, network, raid controller)
  • - operations/puppet update - this should include updates to netboot.pp, and site.pp role(insetup) or cp systems use role(insetup::nofirm).
  • - OS installation & initital puppet run via sre.hosts.reimage cookbook.
cloudvirt1059:
  • - receive in system on procurement task T311863 & in coupa
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
  • - add mgmt dns (asset tag and hostname) and production dns entries in netbox, run cookbook sre.dns.netbox.
  • - network port setup via netbox, run homer from an active cumin host to commit
  • - bios/drac/serial setup/testing, see Lifecycle Steps & Automatic BIOS setup details
  • - firmware update (idrac, bios, network, raid controller)
  • - operations/puppet update - this should include updates to netboot.pp, and site.pp role(insetup) or cp systems use role(insetup::nofirm).
  • - OS installation & initital puppet run via sre.hosts.reimage cookbook.
cloudvirt1060:
  • - receive in system on procurement task T311863 & in coupa
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
  • - add mgmt dns (asset tag and hostname) and production dns entries in netbox, run cookbook sre.dns.netbox.
  • - network port setup via netbox, run homer from an active cumin host to commit
  • - bios/drac/serial setup/testing, see Lifecycle Steps & Automatic BIOS setup details
  • - firmware update (idrac, bios, network, raid controller)
  • - operations/puppet update - this should include updates to netboot.pp, and site.pp role(insetup) or cp systems use role(insetup::nofirm).
  • - OS installation & initital puppet run via sre.hosts.reimage cookbook.
cloudvirt1061:
  • - receive in system on procurement task T311863 & in coupa
  • - rack system with proposed racking plan (see above) & update netbox (include all system info plus location, state of planned)
  • - add mgmt dns (asset tag and hostname) and production dns entries in netbox, run cookbook sre.dns.netbox.
  • - network port setup via netbox, run homer from an active cumin host to commit
  • - bios/drac/serial setup/testing, see Lifecycle Steps & Automatic BIOS setup details
  • - firmware update (idrac, bios, network, raid controller)
  • - operations/puppet update - this should include updates to netboot.pp, and site.pp role(insetup) or cp systems use role(insetup::nofirm).
  • - OS installation & initital puppet run via sre.hosts.reimage cookbook.

Details

SubjectRepoBranchLines +/-
Add hiera host defs for new cloudvirtsoperations/puppetproduction+8 -0
Add wmcs::openstack::eqiad1::virt_ceph to new cloudvirtsoperations/puppetproduction+1 -6
Add partman rules for cloudvirt10[54-61]operations/puppetproduction+1 -1
Add new cloudvirt node to site.ppoperations/puppetproduction+5 -0
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
Unknown Object (Task)
 ResolvedNoneT313984 cloudvirt1019: hpssacli not found
 ResolvedAndrewT313983 Q1:rack/setup/install cloudvirt10[54-61].eqiad.wmnet

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Jclark-ctr added a comment.Aug 31 2022, 1:47 PM

cloudvirt1054 E4 U29 Port 36/37 Cableid 20220045 / 20220041
cloudvirt1055 E4 U30 Port 38/39 Cableid 20220046 / 20220042
cloudvirt1056 E4 U31 Port 40/41 Cableid 20220040 / 20220043
cloudvirt1057 E4 U32 Port 42/43 Cableid 20220068 / 20220044
cloudvirt1058 F4 U29 Port 36/37 Cableid 20220093 / 20220092
cloudvirt1059 F4 U30 Port 38/39 Cableid 20220096 / 20220090
cloudvirt1060 F4 U31 Port 40/41 Cableid 20220097 / 20220089
cloudvirt1061 F4 U32 Port 42/43 Cableid 20220094 / 20220091

Jclark-ctr reassigned this task from Jclark-ctr to • Cmjohnson.Aug 31 2022, 1:48 PM
Jclark-ctr updated the task description. (Show Details)
Jclark-ctr subscribed.
• Cmjohnson reassigned this task from • Cmjohnson to Jclark-ctr.Sep 28 2022, 9:00 PM
• Cmjohnson subscribed.

@Jclark-ctr can you verify that mgmt cables are connected to these servers please?

• Cmjohnson updated the task description. (Show Details)Sep 28 2022, 9:01 PM
Jclark-ctr reassigned this task from Jclark-ctr to • Cmjohnson.Sep 29 2022, 2:18 PM

Verified all of these they are all connected to management and have link

• nskaggs subscribed.Oct 28 2022, 7:58 PM
wiki_willy moved this task from Racking Tasks to Remote Work on the ops-eqiad board.Nov 22 2022, 7:53 PM
Papaul updated the task description. (Show Details)Nov 23 2022, 6:01 PM
Papaul updated the task description. (Show Details)Nov 23 2022, 7:06 PM
Papaul updated the task description. (Show Details)EditedNov 29 2022, 1:23 AM
Papaul subscribed.

@Andrew can you please specify the partman recipe to use for those servers in the task description?
Thank you

Papaul added a comment.Nov 29 2022, 1:39 AM

@Jclark-ctr can you please double check and confirm that all those servers are not R640 like it says in Netbox but there are R440? Thanks

gerritbot added a comment.Dec 1 2022, 3:44 AM

Change 862406 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Add partman rules for cloudvirt10[54-61]

https://gerrit.wikimedia.org/r/862406

gerritbot added a project: Patch-For-Review.Dec 1 2022, 3:44 AM
Andrew updated the task description. (Show Details)Dec 1 2022, 3:45 AM
Jclark-ctr added a comment.Dec 1 2022, 2:15 PM

@Papaul they are R440 we are missing configG in netbox

gerritbot added a comment.Dec 1 2022, 3:54 PM

Change 862958 had a related patch set uploaded (by Papaul; author: Papaul):

[operations/puppet@production] Add new cloudvirt node to site.pp

https://gerrit.wikimedia.org/r/862958

gerritbot added a comment.Dec 1 2022, 3:55 PM

Change 862958 merged by Papaul:

[operations/puppet@production] Add new cloudvirt node to site.pp

https://gerrit.wikimedia.org/r/862958

ops-monitoring-bot added a comment.Dec 1 2022, 4:07 PM

Cookbook cookbooks.sre.hosts.reimage was started by pt1979@cumin2002 for host cloudvirt1054.eqiad.wmnet with OS bullseye

gerritbot added a comment.Dec 1 2022, 4:20 PM

Change 862406 merged by Papaul:

[operations/puppet@production] Add partman rules for cloudvirt10[54-61]

https://gerrit.wikimedia.org/r/862406

ops-monitoring-bot added a comment.Dec 1 2022, 4:28 PM

Cookbook cookbooks.sre.hosts.reimage was started by pt1979@cumin2002 for host cloudvirt1055.eqiad.wmnet with OS bullseye

Papaul updated the task description. (Show Details)Dec 1 2022, 4:29 PM
Maintenance_bot removed a project: Patch-For-Review.Dec 1 2022, 4:31 PM
aborrero added subscribers: cmooney, aborrero.Dec 1 2022, 4:45 PM

note, because T319184: Move WMCS servers to 1 single NIC these hosts only use 1 single network interface. It should be the default in puppet.

We need a particular switch setup for that to work. @cmooney can you please chime in a make sure they are correct?

ops-monitoring-bot added a comment.Dec 1 2022, 4:53 PM

Cookbook cookbooks.sre.hosts.reimage started by pt1979@cumin2002 for host cloudvirt1054.eqiad.wmnet with OS bullseye completed:

  • cloudvirt1054 (PASS)
    • Removed from Puppet and PuppetDB if present
    • Deleted any existing Puppet certificate
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Host up (new fresh bullseye OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202212011607_pt1979_2288755_cloudvirt1054.out
    • Checked BIOS boot parameters are back to normal
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
    • Updated Netbox status planned -> active
    • Cleared switch DHCP cache and MAC table for the host IP and MAC (row E/F)
ops-monitoring-bot added a comment.Dec 1 2022, 4:58 PM

Cookbook cookbooks.sre.hosts.reimage started by pt1979@cumin2002 for host cloudvirt1055.eqiad.wmnet with OS bullseye completed:

  • cloudvirt1055 (PASS)
    • Removed from Puppet and PuppetDB if present
    • Deleted any existing Puppet certificate
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Host up (new fresh bullseye OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202212011628_pt1979_2300414_cloudvirt1055.out
    • Checked BIOS boot parameters are back to normal
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
    • Updated Netbox status planned -> active
    • Cleared switch DHCP cache and MAC table for the host IP and MAC (row E/F)
ops-monitoring-bot added a comment.Dec 1 2022, 5:01 PM

Cookbook cookbooks.sre.hosts.reimage was started by pt1979@cumin2002 for host cloudvirt1056.eqiad.wmnet with OS bullseye

Papaul updated the task description. (Show Details)Dec 1 2022, 5:16 PM
ops-monitoring-bot added a comment.Dec 1 2022, 5:32 PM

Cookbook cookbooks.sre.hosts.reimage started by pt1979@cumin2002 for host cloudvirt1056.eqiad.wmnet with OS bullseye completed:

  • cloudvirt1056 (PASS)
    • Removed from Puppet and PuppetDB if present
    • Deleted any existing Puppet certificate
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Host up (new fresh bullseye OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202212011701_pt1979_2308578_cloudvirt1056.out
    • Checked BIOS boot parameters are back to normal
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
    • Updated Netbox status planned -> active
    • Cleared switch DHCP cache and MAC table for the host IP and MAC (row E/F)
Papaul updated the task description. (Show Details)Dec 1 2022, 5:37 PM
ops-monitoring-bot added a comment.Dec 1 2022, 5:38 PM

Cookbook cookbooks.sre.hosts.reimage was started by pt1979@cumin2002 for host cloudvirt1057.eqiad.wmnet with OS bullseye

ops-monitoring-bot added a comment.Dec 1 2022, 5:42 PM

Cookbook cookbooks.sre.hosts.reimage was started by pt1979@cumin2002 for host cloudvirt1058.eqiad.wmnet with OS bullseye

ops-monitoring-bot added a comment.Dec 1 2022, 5:44 PM

Cookbook cookbooks.sre.hosts.reimage was started by pt1979@cumin2002 for host cloudvirt1059.eqiad.wmnet with OS bullseye

ops-monitoring-bot added a comment.Dec 1 2022, 6:10 PM

Cookbook cookbooks.sre.hosts.reimage started by pt1979@cumin2002 for host cloudvirt1058.eqiad.wmnet with OS bullseye completed:

  • cloudvirt1058 (PASS)
    • Removed from Puppet and PuppetDB if present
    • Deleted any existing Puppet certificate
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Host up (new fresh bullseye OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202212011742_pt1979_2316553_cloudvirt1058.out
    • Checked BIOS boot parameters are back to normal
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
    • Updated Netbox status planned -> active
    • Cleared switch DHCP cache and MAC table for the host IP and MAC (row E/F)
ops-monitoring-bot added a comment.Dec 1 2022, 6:12 PM

Cookbook cookbooks.sre.hosts.reimage was started by pt1979@cumin2002 for host cloudvirt1060.eqiad.wmnet with OS bullseye

ops-monitoring-bot added a comment.Dec 1 2022, 6:16 PM

Cookbook cookbooks.sre.hosts.reimage started by pt1979@cumin2002 for host cloudvirt1059.eqiad.wmnet with OS bullseye completed:

  • cloudvirt1059 (PASS)
    • Removed from Puppet and PuppetDB if present
    • Deleted any existing Puppet certificate
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Host up (new fresh bullseye OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202212011744_pt1979_2316748_cloudvirt1059.out
    • Checked BIOS boot parameters are back to normal
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
    • Updated Netbox status planned -> active
    • Cleared switch DHCP cache and MAC table for the host IP and MAC (row E/F)
Papaul updated the task description. (Show Details)Dec 1 2022, 6:30 PM
ops-monitoring-bot added a comment.Dec 1 2022, 6:34 PM

Cookbook cookbooks.sre.hosts.reimage started by pt1979@cumin2002 for host cloudvirt1057.eqiad.wmnet with OS bullseye executed with errors:

  • cloudvirt1057 (FAIL)
    • Removed from Puppet and PuppetDB if present
    • Deleted any existing Puppet certificate
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • The reimage failed, see the cookbook logs for the details
ops-monitoring-bot added a comment.Dec 1 2022, 6:38 PM

Cookbook cookbooks.sre.hosts.reimage was started by pt1979@cumin2002 for host cloudvirt1057.eqiad.wmnet with OS bullseye

ops-monitoring-bot added a comment.Dec 1 2022, 7:09 PM

Cookbook cookbooks.sre.hosts.reimage started by pt1979@cumin2002 for host cloudvirt1057.eqiad.wmnet with OS bullseye completed:

  • cloudvirt1057 (PASS)
    • Removed from Puppet and PuppetDB if present
    • Deleted any existing Puppet certificate
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Host up (new fresh bullseye OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202212011837_pt1979_2331744_cloudvirt1057.out
    • Checked BIOS boot parameters are back to normal
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
    • Updated Netbox status planned -> active
    • Cleared switch DHCP cache and MAC table for the host IP and MAC (row E/F)
ops-monitoring-bot added a comment.Dec 1 2022, 7:22 PM

Cookbook cookbooks.sre.hosts.reimage started by pt1979@cumin2002 for host cloudvirt1060.eqiad.wmnet with OS bullseye completed:

  • cloudvirt1060 (PASS)
    • Removed from Puppet and PuppetDB if present
    • Deleted any existing Puppet certificate
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Host up (new fresh bullseye OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202212011812_pt1979_2326450_cloudvirt1060.out
    • Checked BIOS boot parameters are back to normal
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
    • Updated Netbox status planned -> active
    • Cleared switch DHCP cache and MAC table for the host IP and MAC (row E/F)
Papaul updated the task description. (Show Details)Dec 1 2022, 7:27 PM
ops-monitoring-bot added a comment.Dec 1 2022, 7:56 PM

Cookbook cookbooks.sre.hosts.reimage was started by pt1979@cumin2002 for host cloudvirt1061.eqiad.wmnet with OS bullseye

ops-monitoring-bot added a comment.Dec 1 2022, 8:27 PM

Cookbook cookbooks.sre.hosts.reimage started by pt1979@cumin2002 for host cloudvirt1061.eqiad.wmnet with OS bullseye completed:

  • cloudvirt1061 (PASS)
    • Removed from Puppet and PuppetDB if present
    • Deleted any existing Puppet certificate
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Host up (new fresh bullseye OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202212011956_pt1979_2348407_cloudvirt1061.out
    • Checked BIOS boot parameters are back to normal
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
    • Updated Netbox status planned -> active
    • Cleared switch DHCP cache and MAC table for the host IP and MAC (row E/F)
Papaul closed this task as Resolved.Dec 1 2022, 8:33 PM
Papaul updated the task description. (Show Details)

@Andrew all yours

aborrero reopened this task as Open.Dec 14 2022, 10:06 AM
aborrero reassigned this task from • Cmjohnson to cmooney.

Reopening until switch changes are made by @cmooney

cmooney added a comment.Dec 14 2022, 11:51 AM

Ok the ports are reconfigured now if you want to give it another shot @Andrew

Andrew claimed this task.Dec 14 2022, 3:40 PM
• nskaggs moved this task from Backlog to Racking / Decom on the cloud-services-team (Hardware) board.Dec 14 2022, 4:37 PM
gerritbot added a comment.Dec 14 2022, 8:04 PM

Change 868159 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Add wmcs::openstack::eqiad1::virt_ceph to new cloudvirts

https://gerrit.wikimedia.org/r/868159

gerritbot added a project: Patch-For-Review.Dec 14 2022, 8:04 PM

Change 868159 merged by Andrew Bogott:

[operations/puppet@production] Add wmcs::openstack::eqiad1::virt_ceph to new cloudvirts

https://gerrit.wikimedia.org/r/868159

ops-monitoring-bot added a comment.Dec 14 2022, 8:14 PM

Cookbook cookbooks.sre.hosts.reimage was started by andrew@cumin1001 for host cloudvirt1054.eqiad.wmnet with OS bullseye

ops-monitoring-bot added a comment.Dec 14 2022, 8:19 PM

Cookbook cookbooks.sre.hosts.reimage started by andrew@cumin1001 for host cloudvirt1054.eqiad.wmnet with OS bullseye executed with errors:

  • cloudvirt1054 (FAIL)
    • Downtimed on Icinga/Alertmanager
    • Unable to disable Puppet, the host may have been unreachable
    • Removed from Puppet and PuppetDB if present
    • Deleted any existing Puppet certificate
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • The reimage failed, see the cookbook logs for the details
ops-monitoring-bot added a comment.Dec 14 2022, 8:29 PM

Cookbook cookbooks.sre.hosts.reimage was started by andrew@cumin1001 for host cloudvirt1054.eqiad.wmnet with OS bullseye

Maintenance_bot removed a project: Patch-For-Review.Dec 14 2022, 8:30 PM
Andrew added a comment.Dec 14 2022, 8:32 PM

I enabled virtualization in the bios processor settings for each of these hosts.

ops-monitoring-bot added a comment.Dec 14 2022, 8:33 PM

Cookbook cookbooks.sre.hosts.reimage was started by andrew@cumin1001 for host cloudvirt1055.eqiad.wmnet with OS bullseye

ops-monitoring-bot added a comment.Dec 14 2022, 8:33 PM

Cookbook cookbooks.sre.hosts.reimage was started by andrew@cumin1001 for host cloudvirt1056.eqiad.wmnet with OS bullseye

ops-monitoring-bot added a comment.Dec 14 2022, 8:36 PM

Cookbook cookbooks.sre.hosts.reimage was started by andrew@cumin1001 for host cloudvirt1057.eqiad.wmnet with OS bullseye

ops-monitoring-bot added a comment.Dec 14 2022, 8:36 PM

Cookbook cookbooks.sre.hosts.reimage was started by andrew@cumin1001 for host cloudvirt1058.eqiad.wmnet with OS bullseye

ops-monitoring-bot added a comment.Dec 14 2022, 8:38 PM

Cookbook cookbooks.sre.hosts.reimage was started by andrew@cumin1001 for host cloudvirt1059.eqiad.wmnet with OS bullseye

ops-monitoring-bot added a comment.Dec 14 2022, 8:38 PM

Cookbook cookbooks.sre.hosts.reimage was started by andrew@cumin1001 for host cloudvirt1060.eqiad.wmnet with OS bullseye

ops-monitoring-bot added a comment.Dec 14 2022, 8:38 PM

Cookbook cookbooks.sre.hosts.reimage was started by andrew@cumin1001 for host cloudvirt1061.eqiad.wmnet with OS bullseye

ops-monitoring-bot added a comment.Dec 14 2022, 8:51 PM

Cookbook cookbooks.sre.hosts.reimage started by andrew@cumin1001 for host cloudvirt1054.eqiad.wmnet with OS bullseye executed with errors:

  • cloudvirt1054 (FAIL)
    • Removed from Puppet and PuppetDB if present
    • Deleted any existing Puppet certificate
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Host up (new fresh bullseye OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • First Puppet run failed, asking the operator what to do
    • First Puppet run failed, asking the operator what to do
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202212142029_andrew_1605054_cloudvirt1054.out
    • Checked BIOS boot parameters are back to normal
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • The reimage failed, see the cookbook logs for the details
ops-monitoring-bot added a comment.Dec 14 2022, 8:53 PM

Cookbook cookbooks.sre.hosts.reimage started by andrew@cumin1001 for host cloudvirt1055.eqiad.wmnet with OS bullseye executed with errors:

  • cloudvirt1055 (FAIL)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present
    • Deleted any existing Puppet certificate
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Host up (new fresh bullseye OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run failed, asking the operator what to do
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202212142033_andrew_1606250_cloudvirt1055.out
    • Checked BIOS boot parameters are back to normal
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • The reimage failed, see the cookbook logs for the details
gerritbot added a comment.Dec 14 2022, 8:54 PM

Change 868166 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Add hiera host defs for new cloudvirts

https://gerrit.wikimedia.org/r/868166

gerritbot added a project: Patch-For-Review.Dec 14 2022, 8:54 PM

Change 868166 merged by Andrew Bogott:

[operations/puppet@production] Add hiera host defs for new cloudvirts

https://gerrit.wikimedia.org/r/868166

ops-monitoring-bot added a comment.Dec 14 2022, 8:56 PM

Cookbook cookbooks.sre.hosts.reimage was started by andrew@cumin1001 for host cloudvirt1055.eqiad.wmnet with OS bullseye

ops-monitoring-bot added a comment.Dec 14 2022, 8:56 PM

Cookbook cookbooks.sre.hosts.reimage was started by andrew@cumin1001 for host cloudvirt1054.eqiad.wmnet with OS bullseye

ops-monitoring-bot added a comment.Dec 14 2022, 9:19 PM

Cookbook cookbooks.sre.hosts.reimage started by andrew@cumin1001 for host cloudvirt1056.eqiad.wmnet with OS bullseye completed:

  • cloudvirt1056 (FAIL)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present
    • Deleted any existing Puppet certificate
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Host up (new fresh bullseye OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run failed, asking the operator what to do
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202212142033_andrew_1606302_cloudvirt1056.out
    • Checked BIOS boot parameters are back to normal
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is not optimal, downtime not removed
    • Updated Netbox data from PuppetDB
    • Cleared switch DHCP cache and MAC table for the host IP and MAC (row E/F)
ops-monitoring-bot added a comment.Dec 14 2022, 9:19 PM

Cookbook cookbooks.sre.hosts.reimage started by andrew@cumin1001 for host cloudvirt1056.eqiad.wmnet with OS bullseye executed with errors:

  • cloudvirt1056 (FAIL)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present
    • Deleted any existing Puppet certificate
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Host up (new fresh bullseye OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run failed, asking the operator what to do
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202212142033_andrew_1606302_cloudvirt1056.out
    • Checked BIOS boot parameters are back to normal
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is not optimal, downtime not removed
    • Updated Netbox data from PuppetDB
    • Cleared switch DHCP cache and MAC table for the host IP and MAC (row E/F)
    • The reimage failed, see the cookbook logs for the details
ops-monitoring-bot added a comment.Dec 14 2022, 9:19 PM

Cookbook cookbooks.sre.hosts.reimage started by andrew@cumin1001 for host cloudvirt1057.eqiad.wmnet with OS bullseye completed:

  • cloudvirt1057 (FAIL)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present
    • Deleted any existing Puppet certificate
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Host up (new fresh bullseye OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run failed, asking the operator what to do
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202212142036_andrew_1606977_cloudvirt1057.out
    • Checked BIOS boot parameters are back to normal
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is not optimal, downtime not removed
    • Updated Netbox data from PuppetDB
    • Cleared switch DHCP cache and MAC table for the host IP and MAC (row E/F)
ops-monitoring-bot added a comment.Dec 14 2022, 9:19 PM

Cookbook cookbooks.sre.hosts.reimage started by andrew@cumin1001 for host cloudvirt1057.eqiad.wmnet with OS bullseye executed with errors:

  • cloudvirt1057 (FAIL)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present
    • Deleted any existing Puppet certificate
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Host up (new fresh bullseye OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run failed, asking the operator what to do
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202212142036_andrew_1606977_cloudvirt1057.out
    • Checked BIOS boot parameters are back to normal
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is not optimal, downtime not removed
    • Updated Netbox data from PuppetDB
    • Cleared switch DHCP cache and MAC table for the host IP and MAC (row E/F)
    • The reimage failed, see the cookbook logs for the details
ops-monitoring-bot added a comment.Dec 14 2022, 9:22 PM

Cookbook cookbooks.sre.hosts.reimage started by andrew@cumin1001 for host cloudvirt1061.eqiad.wmnet with OS bullseye completed:

  • cloudvirt1061 (WARN)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present
    • Deleted any existing Puppet certificate
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Host up (new fresh bullseye OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202212142038_andrew_1607459_cloudvirt1061.out
    • Checked BIOS boot parameters are back to normal
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is not optimal, downtime not removed
    • Updated Netbox data from PuppetDB
    • Cleared switch DHCP cache and MAC table for the host IP and MAC (row E/F)
ops-monitoring-bot added a comment.Dec 14 2022, 9:23 PM

Cookbook cookbooks.sre.hosts.reimage started by andrew@cumin1001 for host cloudvirt1058.eqiad.wmnet with OS bullseye completed:

  • cloudvirt1058 (WARN)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present
    • Deleted any existing Puppet certificate
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Host up (new fresh bullseye OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Unable to downtime the new host on Icinga/Alertmanager, the sre.hosts.downtime cookbook returned 99
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202212142036_andrew_1606815_cloudvirt1058.out
    • Checked BIOS boot parameters are back to normal
    • Unable to run puppet on puppetmaster2001.codfw.wmnet,puppetmaster1001.eqiad.wmnet to update configmaster.wikimedia.org with the new host SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is not optimal, downtime not removed
    • Updated Netbox data from PuppetDB
    • Cleared switch DHCP cache and MAC table for the host IP and MAC (row E/F)
ops-monitoring-bot added a comment.Dec 14 2022, 9:24 PM

Cookbook cookbooks.sre.hosts.reimage started by andrew@cumin1001 for host cloudvirt1059.eqiad.wmnet with OS bullseye completed:

  • cloudvirt1059 (WARN)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present
    • Deleted any existing Puppet certificate
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Host up (new fresh bullseye OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202212142038_andrew_1607531_cloudvirt1059.out
    • Checked BIOS boot parameters are back to normal
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is not optimal, downtime not removed
    • Updated Netbox data from PuppetDB
    • Cleared switch DHCP cache and MAC table for the host IP and MAC (row E/F)
ops-monitoring-bot added a comment.Dec 14 2022, 9:26 PM

Cookbook cookbooks.sre.hosts.reimage started by andrew@cumin1001 for host cloudvirt1060.eqiad.wmnet with OS bullseye completed:

  • cloudvirt1060 (WARN)
    • Downtimed on Icinga/Alertmanager
    • Disabled Puppet
    • Removed from Puppet and PuppetDB if present
    • Deleted any existing Puppet certificate
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Host up (new fresh bullseye OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202212142038_andrew_1607497_cloudvirt1060.out
    • Checked BIOS boot parameters are back to normal
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is not optimal, downtime not removed
    • Updated Netbox data from PuppetDB
    • Cleared switch DHCP cache and MAC table for the host IP and MAC (row E/F)
ops-monitoring-bot added a comment.Dec 14 2022, 9:35 PM

Cookbook cookbooks.sre.hosts.reimage started by andrew@cumin1001 for host cloudvirt1054.eqiad.wmnet with OS bullseye completed:

  • cloudvirt1054 (WARN)
    • Downtimed on Icinga/Alertmanager
    • Unable to disable Puppet, the host may have been unreachable
    • Removed from Puppet and PuppetDB if present
    • Deleted any existing Puppet certificate
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Host up (new fresh bullseye OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202212142055_andrew_1616153_cloudvirt1054.out
    • Checked BIOS boot parameters are back to normal
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is optimal
    • Icinga downtime removed
    • Updated Netbox data from PuppetDB
    • Cleared switch DHCP cache and MAC table for the host IP and MAC (row E/F)
ops-monitoring-bot added a comment.Dec 14 2022, 9:39 PM

Cookbook cookbooks.sre.hosts.reimage started by andrew@cumin1001 for host cloudvirt1055.eqiad.wmnet with OS bullseye completed:

  • cloudvirt1055 (WARN)
    • Downtimed on Icinga/Alertmanager
    • Unable to disable Puppet, the host may have been unreachable
    • Removed from Puppet and PuppetDB if present
    • Deleted any existing Puppet certificate
    • Removed from Debmonitor if present
    • Forced PXE for next reboot
    • Host rebooted via IPMI
    • Host up (Debian installer)
    • Host up (new fresh bullseye OS)
    • Generated Puppet certificate
    • Signed new Puppet certificate
    • Run Puppet in NOOP mode to populate exported resources in PuppetDB
    • Found Nagios_host resource for this host in PuppetDB
    • Downtimed the new host on Icinga/Alertmanager
    • Removed previous downtime on Alertmanager (old OS)
    • First Puppet run completed and logged in /var/log/spicerack/sre/hosts/reimage/202212142055_andrew_1616181_cloudvirt1055.out
    • Checked BIOS boot parameters are back to normal
    • configmaster.wikimedia.org updated with the host new SSH public key for wmf-update-known-hosts-production
    • Rebooted
    • Automatic Puppet run was successful
    • Forced a re-check of all Icinga services for the host
    • Icinga status is not optimal, downtime not removed
    • Updated Netbox data from PuppetDB
    • Cleared switch DHCP cache and MAC table for the host IP and MAC (row E/F)
Andrew closed this task as Resolved.Dec 14 2022, 9:50 PM

I've imaged all these servers and put 1054 in the 'ceph' pool and the others in the 'spare' pool. We can move all of them into 'ceph' once 1054 proves itself.

Also, all of these were already marked 'active' in netbox so I guess... that's good?

Jclark-ctr mentioned this in T321719: Port with no description on access switch.Feb 2 2023, 3:12 PM
Maintenance_bot removed a project: Patch-For-Review.Feb 2 2023, 3:31 PM
• nskaggs added a parent task: T313984: cloudvirt1019: hpssacli not found.Mar 3 2023, 2:08 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL