meanwhile there is another one in ATS backend.yaml.
Fri, Feb 14
@CGlenn You can but there is an extra step involved where @Ottomata or @elukey need to sync users (per https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Hue#Access)
new checks have been added to Icinga:
Setting to stalled to reflect that. We should change status to Open when it's ready to go.
Given that Luca also had an error during initial setup related to name resolution, this sounds like some error related to the DNS records for the new host?
Wed, Feb 12
The primary network interface is missing from /etc/network/interfaces. There is only loopback in there. Why that is is another question.
Debug: Augeas[ens5_v6_token](provider=augeas): sending command 'set' with params ["/files/etc/network/interfaces/iface[. = 'ens5']/pre-up", "/sbin/ip token set ::208:80:153:62 dev ens5"] Debug: Augeas[ens5_v6_token](provider=augeas): Put failed on one or more files, output from /augeas//error: Debug: Augeas[ens5_v6_token](provider=augeas): /augeas/files/etc/network/interfaces/error = put_failed Debug: Augeas[ens5_v6_token](provider=augeas): /augeas/files/etc/network/interfaces/error/path = /files/etc/network/interfaces/ Debug: Augeas[ens5_v6_token](provider=augeas): /augeas/files/etc/network/interfaces/error/lens = /usr/share/augeas/lenses/dist/interfaces.aug:125.13-.63: Debug: Augeas[ens5_v6_token](provider=augeas): /augeas/files/etc/network/interfaces/error/message = Failed to match tree under / Debug: Augeas[ens5_v6_token](provider=augeas): Closed the augeas connection Error: /Stage[main]/Profile::Standard/Interface::Add_ip6_mapped[main]/Augeas[ens5_v6_token]: Could not evaluate: Saving failed, see debug
Tue, Feb 11
@Marostegui I made some changes to make the db_user and db_pass configurable for gerrit. Thing is just i don't know the clear text version of the hashed password for 'gerritro'. I took a look at the relevant m2-master behind dbproxies, db1132 and i see
This should already work https://phabricator.wikimedia.org/T236209#5597161 Please confirm the credentials are working on Wikitech wiki and you are using the exact same ones on superset. Let us know if any remaining issues.
@ItamarWMDE This is done. You are a member of both "wmde" and "nda" LDAP groups like other WMDE employees before.
Thank you very much @Peachey88
- created Wikidata item: https://www.wikidata.org/wiki/Q84850642
Mon, Feb 10
I added 2 more canary appservers. now we have:
The problem i see with that is how do you define what a "small" change is? It feels like when we call things "trivial" but then there are unexpected changes anyways. The very nature of them being unexpected means they would all be called small or trivial before the fact.
Just to confirm. It should still have a public IP in wikimedia.org ?
@kzimmerman Gotcha, I created a request for them at https://wikimedia.zendesk.com/hc/en-us/requests/new
@Eevans You should have +2 on the mw-config repo now. Probably after logging out and back in.
@jijiki Eric is already in the "deployment" shell users group and membership in the relevant Gerrit group normally goes with that since it makes little sense to trust people with deploying whatever they want but not let them merge config changes in Gerrit. Gerrit access requests can be handled by any Gerrit admin, Gerrit manager or for this specific group also anyone in the ops LDAP group as Urbanecm points out.
@kzimmerman Hi, in OITs LDAP record the manager is neither of them but listed as "dzierten". That's the only place we can check though. Should this be fixed in OIT?
Sat, Feb 8
regarding disk requirements:
@Muehlenhoff This was the idea, right? Or was it to share with an existing webserver?
VM looks up and running, all green in Icinga.
Fri, Feb 7
VMs have been created. OS install now worked at second attempt.
Thu, Feb 6
@Muehlenhoff Added them with private IPs, created VMs with the cookbook, then attempted OS install but on both of them it failed at the very end with GRUB install. I don't think this happened to me before on a ganeti VM.
@jijiki What do you think ? Is this good now? 4 of each type and in different rows/racks.
mw2163 and mw2271 have been turned into canary appservers now. As opposed to canary API appservers this means actual puppet changes which are:
As of today these hosts are in site.pp with spare::system role and not in production yet. So while standard Icinga alerts should be downtimed, no actual Ganeti service would be affected and it could happen anytime. Before doing that just check in site.pp if they still have the spare role and not the ganeti role.
Aww, thanks for conforming and thanks Moritz for fixing it. This is exactly why i wanted to test it. The capitalization caught us a couple times before.
reverted / removed public IPs, added private IPs
currently blocked on T244438 , an installer issue on stretch that only happens on stretch and buster would not have a problem
Wed, Feb 5
@Pchelolo This should work now.
As @MarcoAurelio points out this normally goes together with getting the deployment admin group. Petr is already a member of that (and various other shell admin groups).
mw1267 was showing temperature issues today:
See T243983. I added a second disk to this VM, it's an additional 10GB and mounted on /srv/dbdump. Hope that does it.
@jijiki On our end in private repo. puppetmaster1001:/srv/private/modules/privateexim/files (already done though)
@SpramudyaDev You have been added to the "wmf" group. You should now be able to login with the same credentials used on Wikitech wiki.
Hmm.. fair enough. That means my DNS change was not correct though, it defined public IPs as before.
The following are now declared canary API appservers in site.pp: