@hashar Please let me know if you still see the issue or not. I am hopeful it might be fixed and can't reproduce it right now but also it seems we only had spikes during meetings when many people at once used it, like today's quarterly review meeting.
^ Uhm.. but this went up. Trying to manually reproduce I could not... also before the change. Need more users at once.
Yes, SRE has root access to it. There is no other admin role applied though.
Sat, Oct 17
@RobH Please do not assign service puppet roles on new hosts. That will almost never work. Just add new hosts in site with the "insetup" role and hand over. Service implementation will be to apply the role along with other changes in Hiera etc.
Fri, Oct 16
Since we never got a reply here I recently asked ITS (formerly OIT) to move optoutresearch@ over to them. Harry confirmed it was created on their side and then I removed it on the SRE side. So if there are edits needed please contact them now.
@Aklapper I see that "WMF-Office-IT" was archived but there does not seem to be a replacement for "ITS". There are tickets though that should still be tagged.. what should we do with those?
Alright, sounds good. Thank you @ssastry
Thu, Oct 15
I am not sure what exactly needs to be done here as the next step. I have created a VM and both rt client/server and vd client/server are on it. Also added the ability to stop both clients.
re-assigning based on the last comment
We will remove remaining personal aliases for non-staff once 2020 is over. The users will be contacted that they have time until end of year to switch to an alternative. This was decided in cooperation with ITS and the board.
"No such list fkc-l"
@RobH Please leave it on stretch for now. It relies on Mediawiki classes that are not ready for buster just yet.
ah, no it's antispam indeed:
Maybe the mails are not going out from @tools.wmflabs.org anymore but now from .wmcloud.org and that means it does not match the list of subscribers anymore. Since recently the names of instances changed in some places.
Do you want the URL changed or is that fine as it is?
Wed, Oct 14
wtp2001 - wtp2005 depooled
There is an existing setup for this. puppet already clones from design/strategy to /srv/org/wikimedia/design-strategy. All you have to do is merge stuff in that repo. It's just like design/landing-page or design/blog. It is NOT like design/style-guide because that uses scap and the deployment server but as far as I remember the only reason to do that was that you wanted to use git-lfs, right?
There was a request to decom the wtp2* servers. There is a new ticket for that at T265558.
Tue, Oct 13
21:08 <+icinga-wm> PROBLEM - Ensure local MW versions match expected deployment on mw2279 is CRITICAL: CRITICAL: 956 mismatched wikiversions https://wikitech.wikimedia.org/wiki/Application_servers 21:09 < mutante> interesting. is that being used as a test host right now?
Mon, Oct 12
Sat, Oct 10
Also MW version 1.36 is now recognized as good version and colored green.
Thank you very much for the fixes. Yes, wikivoyage was missing and has now been added.
Fri, Oct 9
Fix all of our puppet code for MediaWiki for incompatibilities with buster
As well as T250515 for the PHP packages.
A new ticket for this has been created at T264991.
There was already T245757 with dependency tickets.
Is there a ticket for moving these into production?
maps2010 is reported as down since about 3 days
Hi @sbassett security@ isn't in exim anymore nowadays. It's in Google, you'll have to ask OIT via Zendesk please:
Thu, Oct 8
in addition to having different message contents are also sent outward via the main mx host interface instead of the wiki-mail-site.wikimedia.org bulk mail interface.
+1. nginx is in use on install servers, puppetmaster, puppetdb, mirrors, docker registry. But if there is a goal to remove it maybe we should do another ticket for that specifically, with checkboxes.
Using the same key should be fine. But we will need a new "expiry_date" please. And should we use expiry_contact: nruiz@ like before?
The deploy-style-guide.sh script git pulls from https://gerrit.wikimedia.org/r/p/design/style-guide.git
I checked the git status on both backends, miscweb1002 and miscweb2002 and they are both at commit e3fda830a1573a398eda486341f2f6c2cc25bffe " “Design principles”: Improve “Consistency” image and make it mobile friendly (#390)" from June 30th 2020.
Wed, Oct 7
added to DNS:
I'm declaring this resolved. Everything is done according to the plan. Let me know if you think otherwise.
quoting from the original task description and replying inline:
☑️ eqsin tested - reinstalled testvm5001 and confirmed in syslog that install5001 is serving both DHCP and TFTP now; not bast5001 (tftp) or install2003 (dhcp) anymore; and OS install is working
Tue, Oct 6
Mon, Oct 5
asked Mukunda about it and he says it is probably obsolete. asked -traffic because i do see @ema still logged in there not too long ago
shinken instance and puppet module have been deleted. this leaves just the icinga part and / or is blocked by T250206
The shinken module has been deleted in https://gerrit.wikimedia.org/r/c/operations/puppet/+/629464
Sat, Oct 3
This seems somewhat but not exactly a duplicate of T254480. The other ticket would be resolved once we have removed all cases of it from the repo.
Fri, Oct 2
The hosts here are showing up in a weird state. When running the DNS cookbook you get warnings that these hosts exist but are not "in devices". Though running the cookbook above should have cleaned that up.
Thu, Oct 1
Wed, Sep 30
netops could you please deploy https://gerrit.wikimedia.org/r/c/operations/homer/public/+/631261 ?
Tue, Sep 29
This topic branch shows previous work I had done for this but never linked to this ticket: 2 more were just merged today:
Just a few minutes ago: db2125 crashed - mgmt iface also not available T260670
@bete You have been added to both groups as requested. Things should work now. Let us know if any issues.
@ArielGlenn LDAP-only users also need to be added to puppet admins module nowadays.
Mon, Sep 28
Sat, Sep 26
Fri, Sep 25
This was basically all done:
Thu, Sep 24
The maintenance periodic timer has been removed on mwmaint hosts.