Dzahn (Daniel Zahn)
Operations Engineer at WMF

Projects (17)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Thursday

  • Clear sailing ahead.

User Details

User Since
Sep 30 2014, 4:39 PM (158 w, 6 d)
Availability
Available
IRC Nick
mutante
LDAP User
Dzahn
MediaWiki User
Unknown

Recent Activity

Today

Dzahn added a comment to T162850: CPU throttling on DELL PowerEdge R320.

Only a single host is left to upgrade and reboot, baham. But this is a bit more complicated to coordinate.

Tue, Oct 17, 12:04 AM · Patch-For-Review, Operations

Yesterday

Dzahn added a comment to T127617: Upgrade phab-01.wmflabs.org.

This is T139475 and T144112 and T131899 . Somehow they are all closed yet none of them are really resolved.

Mon, Oct 16, 9:42 PM · VPS-project-Phabricator
Dzahn added a comment to T127617: Upgrade phab-01.wmflabs.org.

Additional work is needed to make modules/phabricator/templates/ flexible if we want to be able to actually test prod changes in labs/beta.

Mon, Oct 16, 9:36 PM · VPS-project-Phabricator
Dzahn added a comment to T127617: Upgrade phab-01.wmflabs.org.

It turns out we actually still don't have any instance that is using the prod role and can also be used for testing. The instance that does use the prod role redirects to prod, because the phabricator module has hardcoded "wikimedia.org" values in templates. So this probably never worked before.

Mon, Oct 16, 9:36 PM · VPS-project-Phabricator
Dzahn added a comment to T127617: Upgrade phab-01.wmflabs.org.

Just one thing to do maybe: Update all wiki references to "https://phab-01"

Mon, Oct 16, 8:32 PM · VPS-project-Phabricator
Dzahn added a comment to T127617: Upgrade phab-01.wmflabs.org.

Talked to paladox. Situation was:

Mon, Oct 16, 8:31 PM · VPS-project-Phabricator
Dzahn added a comment to T127617: Upgrade phab-01.wmflabs.org.

Upgrade phab-01.wmflabs.org to the same version like the production one, so new features can be tested there.

Mon, Oct 16, 8:22 PM · VPS-project-Phabricator
Dzahn added a project to T84148: fix disk space check on dataset1001: monitoring.
Mon, Oct 16, 6:53 PM · monitoring, Operations
Dzahn added a comment to T84148: fix disk space check on dataset1001.

@Aklapper thanks, resolved :) (and made public, NDA was just because this was an RT import, yea, that old)

Mon, Oct 16, 6:52 PM · monitoring, Operations
Dzahn closed T84148: fix disk space check on dataset1001 as Resolved.
Mon, Oct 16, 6:51 PM · monitoring, Operations
Dzahn lowered the priority of T84148: fix disk space check on dataset1001 from High to Low.
Mon, Oct 16, 6:51 PM · monitoring, Operations
Dzahn removed a project from T84148: fix disk space check on dataset1001: Patch-For-Review.
Mon, Oct 16, 6:51 PM · monitoring, Operations
Dzahn claimed T84148: fix disk space check on dataset1001.
Mon, Oct 16, 6:51 PM · monitoring, Operations
Dzahn closed T178220: Fix cronspam from /usr/local/sbin/pdns_gmetric as Resolved.

I see now that "labtestcontrol2003"/labservices100x" were also affected but mails stopped 3 days ago and crons are gone. I see nothing else new. Please reopen if you still get anything.

Mon, Oct 16, 4:21 PM · monitoring
Dzahn added a comment to T178220: Fix cronspam from /usr/local/sbin/pdns_gmetric.

@ArielGlenn Yes, it's Ganglia-only, and i started the spam when i removed that - because it's Ganglia - in https://gerrit.wikimedia.org/r/#/c/382929/ as part of the general effort to remove all of Ganglia (T177225).

Mon, Oct 16, 3:54 PM · monitoring
Dzahn added a comment to T177599: Request to be added to the ldap/wmde group.

Alright, thanks. So all members of 'wmde' should be added to ldap_only_users. I will add a patch to do that.

Mon, Oct 16, 2:08 PM · User-Addshore, Operations, LDAP-Access-Requests, WMF-NDA-Requests

Fri, Oct 13

Dzahn added a project to T176875: Allow access to wdqs.svc.eqiad.wmnet on port 8888: Traffic.
Fri, Oct 13, 10:55 PM · Traffic, Wikidata-Query-Service, Operations, WMDE-Analytics-Engineering, User-Addshore, Discovery, Wikidata
Dzahn added a subtask for T177782: Reduce false positives on database pages: T178008: ensure that services on labtest machines never create SMS from Icinga (not send sms pages for labtest* things to non-cloud folks).
Fri, Oct 13, 10:41 PM · Patch-For-Review, Epic, Wikimedia-Incident, monitoring, DBA
Dzahn added a parent task for T178008: ensure that services on labtest machines never create SMS from Icinga (not send sms pages for labtest* things to non-cloud folks): T177782: Reduce false positives on database pages.
Fri, Oct 13, 10:41 PM · Patch-For-Review, monitoring, Operations
Dzahn added a comment to T177782: Reduce false positives on database pages.

The change i uploaded above intents to disable paging for the specific check "mysql procs running" if a host is in labs/labtest. It's using existing regexes in Hiera that should match all labs hosts (the same that also sets the cluster). This is just for mysql process and just for labs, but i think it's a start. It's a reaction to the recent SMS we got when mysql/maria role was used on labtest hosts and then the process was stopped. I think it should be easy to agree that something called "labs" or "test" means it can't be important enough to send actual SMS, right?

Fri, Oct 13, 10:39 PM · Patch-For-Review, Epic, Wikimedia-Incident, monitoring, DBA
Dzahn added a comment to T178008: ensure that services on labtest machines never create SMS from Icinga (not send sms pages for labtest* things to non-cloud folks).

There are other existing checks though that would page and don't already have "is_critical" parameters like that.

Fri, Oct 13, 10:16 PM · Patch-For-Review, monitoring, Operations
Dzahn added a comment to T178008: ensure that services on labtest machines never create SMS from Icinga (not send sms pages for labtest* things to non-cloud folks).

The second patch should be a nicer solution, it avoids having a regex in the mariadb module or making any changes there and just uses Hiera.

Fri, Oct 13, 10:15 PM · Patch-For-Review, monitoring, Operations
Dzahn lowered the priority of T136735: create endowment.wm.org microsite from High to Normal.
Fri, Oct 13, 9:09 PM · Patch-For-Review, Operations
Dzahn added a comment to T177599: Request to be added to the ldap/wmde group.

I don't see the wmde LDAP group in the admins puppet module.

Fri, Oct 13, 8:58 PM · User-Addshore, Operations, LDAP-Access-Requests, WMF-NDA-Requests
Dzahn added a comment to T145659: Port application-specific metrics from ganglia to prometheus.

@fgiunchedi I was wondering if there is something here that replaces the PacketLossLogtailer from udp2log (https://gerrit.wikimedia.org/r/#/c/382913/) .. udp2log is on the way out too, right, Ottomata said +1 to remove it i see

Fri, Oct 13, 5:54 PM · Patch-For-Review, Prometheus-metrics-monitoring, Operations
Dzahn added a comment to T177225: Uninstall ganglia from the fleet.

^ I amended the one for dnsrecursors to include the "ganglia: false" Hiera setting right away and then merged that too. Metrics were converted to Diamond collector in T169600

Fri, Oct 13, 4:49 PM · Patch-For-Review, Operations, monitoring
Dzahn added a comment to T177225: Uninstall ganglia from the fleet.

So these hosts now having only the "basic" ganglia stuff. Should we switch has_ganglia to no for them (or preferably the role), so that the ganglia::monitor::decom class takes over and cleans up (and also to test it) ?

Fri, Oct 13, 4:18 PM · Patch-For-Review, Operations, monitoring
Dzahn added a comment to T177599: Request to be added to the ldap/wmde group.

No, sorry, the nda group was never requested and is used for different things, not for controlling access to repos.

Fri, Oct 13, 2:15 PM · User-Addshore, Operations, LDAP-Access-Requests, WMF-NDA-Requests
Dzahn added a comment to T147426: Port gdnsd statistics from ganglia to prometheus.

removed the ganglia stats for this today

Fri, Oct 13, 12:27 AM · Patch-For-Review, Traffic, Prometheus-metrics-monitoring, Operations
Dzahn added a comment to T177225: Uninstall ganglia from the fleet.

removed ganglia stats for gdnsd (authdns servers).

Fri, Oct 13, 12:26 AM · Patch-For-Review, Operations, monitoring
Dzahn added a comment to T136735: create endowment.wm.org microsite.

@kaythaney Great! Sounds good, thank you. I already removed everything on our side.

Fri, Oct 13, 12:03 AM · Patch-For-Review, Operations

Thu, Oct 12

Dzahn reassigned T164030: setup releases1001.eqiad.wmnet (was: setup mwreleases1001) from Dzahn to demon.

Back to Chad. Jenkins should be usable now.

Thu, Oct 12, 9:41 PM · Release-Engineering-Team (Kanban), RelEng-Archive-FY201718-Q1, Patch-For-Review, Operations, Security-General
Dzahn changed the status of T136735: create endowment.wm.org microsite from Open to Stalled.
Thu, Oct 12, 9:40 PM · Patch-For-Review, Operations
Dzahn added a comment to T178039: scap should not pull in HHVM on stretch hosts using PHP7.

I stopped hhvm on gerrit2001 but i can't remove the package just yet because that requirement also means both hhvm and scap get removed if you attempt to remove hhvm.

Thu, Oct 12, 7:59 PM · Operations, Scap
Dzahn closed T177599: Request to be added to the ldap/wmde group as Resolved.

@RStallman-legalteam Thank you for confirming. Done, i added him just now.

Thu, Oct 12, 6:29 PM · User-Addshore, Operations, LDAP-Access-Requests, WMF-NDA-Requests
Dzahn reassigned T178082: Shutdown ocg Cloud VPS project? from Dzahn to Andrew.

Hi Andrew, I created this project back in Wikitech and just deleted the only instance in it, so now it's empty. Could you delete the project itself? I don't think i can do it (in Horizon), what permissions do you need for that nowadays, btw? Thanks

Thu, Oct 12, 5:32 PM · OfflineContentGenerator, Cloud-VPS
Dzahn added a comment to T164209: webpagetest-alerts: Difference in size authenticated.

fyi, currently there is: CRITICAL: https://grafana.wikimedia.org/dashboard/db/webpagetest-alerts is alerting: Speed Index Internet Explorer Desktop [ALERT] alert, Speed Index Mobile [ALERT] alert.

Thu, Oct 12, 4:49 PM · Operations, Performance-Team
Dzahn claimed T178082: Shutdown ocg Cloud VPS project?.

The project/instance was originally created for T135034 but that is declined now because we don't run OCG anymore. so yea. this can go. I can handle it since i also created it.

Thu, Oct 12, 4:40 PM · OfflineContentGenerator, Cloud-VPS
Dzahn added a comment to T178082: Shutdown ocg Cloud VPS project?.

Looks like i created that back in May 2016 and there was something related to the issues with exim on labs instances (T135033)

Thu, Oct 12, 4:16 PM · OfflineContentGenerator, Cloud-VPS
Dzahn updated subscribers of T178039: scap should not pull in HHVM on stretch hosts using PHP7.
Thu, Oct 12, 3:10 AM · Operations, Scap
Dzahn added a comment to T178039: scap should not pull in HHVM on stretch hosts using PHP7.

23:00 < MaxSem> uhh, why wold it depend on PHP?

Thu, Oct 12, 3:10 AM · Operations, Scap
Dzahn updated subscribers of T178039: scap should not pull in HHVM on stretch hosts using PHP7.
Thu, Oct 12, 3:08 AM · Operations, Scap
Dzahn updated the task description for T178039: scap should not pull in HHVM on stretch hosts using PHP7.
Thu, Oct 12, 3:06 AM · Operations, Scap
Dzahn created T178039: scap should not pull in HHVM on stretch hosts using PHP7.
Thu, Oct 12, 3:03 AM · Operations, Scap

Wed, Oct 11

Dzahn assigned T178008: ensure that services on labtest machines never create SMS from Icinga (not send sms pages for labtest* things to non-cloud folks) to chasemp.
Wed, Oct 11, 9:55 PM · Patch-For-Review, monitoring, Operations
Dzahn created T178008: ensure that services on labtest machines never create SMS from Icinga (not send sms pages for labtest* things to non-cloud folks).
Wed, Oct 11, 9:53 PM · Patch-For-Review, monitoring, Operations
Dzahn added a comment to T177962: Upgrade Jenkins to 2.73.2 (security release).

19:20 mutante: apt: reprepro copy stretch-wikimedia jessie-wikimedia jenkins

Wed, Oct 11, 8:09 PM · Operations, Release-Engineering-Team (Kanban), Jenkins, Continuous-Integration-Infrastructure
Dzahn removed a project from T177974: Drop #wikimedia-codereview channel: Operations.
Wed, Oct 11, 6:18 PM · Patch-For-Review, wikimedia-irc-freenode
Dzahn added a comment to T177974: Drop #wikimedia-codereview channel.

I haven't found any "irc-managers" or something like that tag, so Operations seemd to fit best.

Wed, Oct 11, 6:17 PM · Patch-For-Review, wikimedia-irc-freenode
Dzahn added a comment to T177427: Remove non-interactive bots from #wikimedia-cloud.

Also see T177974 for a similar thing that was attempted in the past and lead to removing the channel again now.

Wed, Oct 11, 6:12 PM · Patch-For-Review, Cloud-Services, cloud-services-team (Kanban)
Dzahn added a comment to T177974: Drop #wikimedia-codereview channel.

This ticket should be linked to all other tickets that suggest _adding_ more channels as an example for why it might not be the best idea. Because the "another channel", just like "another list" and "another wiki" come up regularly and often end up like this later.

Wed, Oct 11, 6:11 PM · Patch-For-Review, wikimedia-irc-freenode
Dzahn added a comment to T166486: Decommission db1023.

https://gerrit.wikimedia.org/r/382495

Wed, Oct 11, 5:14 PM · Patch-For-Review, ops-eqiad, Operations, DBA
Dzahn added a comment to T177225: Uninstall ganglia from the fleet.

Looks like this is the relevant list https://wikitech.wikimedia.org/wiki/Prometheus#Ganglia_plugins to see which plugins have been replaced with what. Can any updates be made to that list?

Wed, Oct 11, 12:23 AM · Patch-For-Review, Operations, monitoring

Tue, Oct 10

Dzahn added a comment to T177602: analytics-privatedata-users access for Jeff Green.

Yea, aware Jeff has root on the mentioned stat boxes anyways, heh.

Tue, Oct 10, 10:14 PM · Patch-For-Review, Analytics, Ops-Access-Requests, Operations
Dzahn added a comment to T177602: analytics-privatedata-users access for Jeff Green.

@Jgreen Do you just need Hive/Hadoop or do you additionally need sampled webrequest logs and stat boxes with private data? Asking this way because the description for the requested group says "This group should not be used just to grant someone Hadoop access." and you mention just Hive.

Tue, Oct 10, 10:12 PM · Patch-For-Review, Analytics, Ops-Access-Requests, Operations
Dzahn added a comment to T177599: Request to be added to the ldap/wmde group.

Is L2 enough to get into the ldap/nda group?

Tue, Oct 10, 10:08 PM · User-Addshore, Operations, LDAP-Access-Requests, WMF-NDA-Requests
Dzahn added a comment to T177599: Request to be added to the ldap/wmde group.

I have contacted WMF legal to reach out to Pablo so he can sign the right NDA. I asked and L2 is only for Phabricator access to non-public tickets, but if LDAP groups are involved a different kind of NDA must be signed directly with legal.

Tue, Oct 10, 9:53 PM · User-Addshore, Operations, LDAP-Access-Requests, WMF-NDA-Requests
Dzahn added a comment to T177599: Request to be added to the ldap/wmde group.

@Tobi_WMDE_SW Ok, got it. Thanks for explaining and confirming that.

Tue, Oct 10, 8:06 PM · User-Addshore, Operations, LDAP-Access-Requests, WMF-NDA-Requests
Dzahn added a comment to T177225: Uninstall ganglia from the fleet.

equivalent metrics in Prometheus and Graphite, and these need to show up in a suitable Grafana dashboard. Has this been taken into account?

Tue, Oct 10, 5:54 PM · Patch-For-Review, Operations, monitoring
Dzahn added a comment to T177225: Uninstall ganglia from the fleet.
it got me thinking how are we going to clean up the fleet ? Should we use puppet (so all these changes require amending) ?
Tue, Oct 10, 5:51 PM · Patch-For-Review, Operations, monitoring

Sat, Oct 7

Dzahn updated subscribers of T177225: Uninstall ganglia from the fleet.

@Ottomata Hi, wondering what do you think should happen with modules/confluent/manifests/kafka/mirror/jmxtrans.pp and modules/confluent/manifests/kafka/broker/jmxtrans.pp: once Ganglia gets removed?

Sat, Oct 7, 3:04 AM · Patch-For-Review, Operations, monitoring
Dzahn triaged T177225: Uninstall ganglia from the fleet as Normal priority.
Sat, Oct 7, 12:54 AM · Patch-For-Review, Operations, monitoring
Dzahn awarded T177225: Uninstall ganglia from the fleet a Love token.
Sat, Oct 7, 12:53 AM · Patch-For-Review, Operations, monitoring
Dzahn claimed T177225: Uninstall ganglia from the fleet.
Sat, Oct 7, 12:53 AM · Patch-For-Review, Operations, monitoring
Dzahn added a comment to T99531: [Task] move wikiba.se webhosting to wikimedia misc-cluster.

Let's stick with ensure => latest.

Sat, Oct 7, 12:40 AM · Patch-For-Review, Traffic, wikiba.se, Operations, Wikidata-Sprint-2016-11-08, Wikidata
Dzahn added a comment to T99531: [Task] move wikiba.se webhosting to wikimedia misc-cluster.
Sat, Oct 7, 12:33 AM · Patch-For-Review, Traffic, wikiba.se, Operations, Wikidata-Sprint-2016-11-08, Wikidata
Dzahn added a comment to T177599: Request to be added to the ldap/wmde group.

Hi, re: "to be able to contribute to AdvancedSearch" does this mean you want the +2 permissions to be able to merge changes in Gerrit in a wikidata-related repo?

Sat, Oct 7, 12:25 AM · User-Addshore, Operations, LDAP-Access-Requests, WMF-NDA-Requests
Dzahn added a comment to T174370: Create elections committee private wiki.

added the mobile domain to DNS: https://electcom.m.wikimedia.org/

Sat, Oct 7, 12:20 AM · Support-and-Safety, DBA, Patch-For-Review, Operations, Wiki-Setup (Create)
Dzahn added a comment to T174370: Create elections committee private wiki.

@jrbs: re: patches rebasing: git review -d <number of gerrit patch>; git rebase -i origin/master; then if it fails "git rebase --continue" to see in which file the conflict is; manually edit the file to remove the conflict, git rebase --continue; git review

Sat, Oct 7, 12:18 AM · Support-and-Safety, DBA, Patch-For-Review, Operations, Wiki-Setup (Create)

Fri, Oct 6

Dzahn removed a project from T145885: Gerrit shows HTTP 500 error when pasting extended unicode characters: DBA.
Fri, Oct 6, 2:50 PM · Release-Engineering-Team (Backlog), Patch-For-Review, Operations, Gerrit
Dzahn added a comment to T145832: Create Trusted Contributors project?.

Membership should be maximally-inclusive, essentially anyone who has demonstrated constructive contributions in any area of the wikimedia community should be added

Fri, Oct 6, 12:05 AM · Project-Admins

Thu, Oct 5

Dzahn added a comment to T99531: [Task] move wikiba.se webhosting to wikimedia misc-cluster.

By the way i couldn't just merge Ladsgroup's change in the new content repo as i just have +1 rights there, but not +2.

Thu, Oct 5, 11:53 PM · Patch-For-Review, Traffic, wikiba.se, Operations, Wikidata-Sprint-2016-11-08, Wikidata
Dzahn added a comment to T99531: [Task] move wikiba.se webhosting to wikimedia misc-cluster.

Next we should figure out:

Thu, Oct 5, 11:51 PM · Patch-For-Review, Traffic, wikiba.se, Operations, Wikidata-Sprint-2016-11-08, Wikidata
Dzahn awarded T177493: IRC operator request for Freenode #wikimedia-operations for @Dereckson a Like token.
Thu, Oct 5, 10:56 PM · Ops-Access-Requests, Operations
Dzahn updated the task description for T177537: make addwiki.php use 'CREATE IF NOT EXISTS' consistently.
Thu, Oct 5, 9:59 PM · MW-1.31-release-notes (WMF-deploy-2017-10-10 (1.31.0-wmf.3)), MediaWiki-extensions-WikimediaMaintenance, MediaWiki-Database
Dzahn updated subscribers of T177537: make addwiki.php use 'CREATE IF NOT EXISTS' consistently.
Thu, Oct 5, 8:50 PM · MW-1.31-release-notes (WMF-deploy-2017-10-10 (1.31.0-wmf.3)), MediaWiki-extensions-WikimediaMaintenance, MediaWiki-Database
Dzahn added a project to T177537: make addwiki.php use 'CREATE IF NOT EXISTS' consistently: MediaWiki-extensions-Maintenance.
Thu, Oct 5, 8:48 PM · MW-1.31-release-notes (WMF-deploy-2017-10-10 (1.31.0-wmf.3)), MediaWiki-extensions-WikimediaMaintenance, MediaWiki-Database
Dzahn created T177537: make addwiki.php use 'CREATE IF NOT EXISTS' consistently.
Thu, Oct 5, 8:46 PM · MW-1.31-release-notes (WMF-deploy-2017-10-10 (1.31.0-wmf.3)), MediaWiki-extensions-WikimediaMaintenance, MediaWiki-Database
Dzahn updated subscribers of T177493: IRC operator request for Freenode #wikimedia-operations for @Dereckson.
Thu, Oct 5, 2:21 PM · Ops-Access-Requests, Operations
Dzahn added a comment to T99531: [Task] move wikiba.se webhosting to wikimedia misc-cluster.

We have the repo, and i see your change Amir, thank you! Before i merge the content itself i started with puppet code to install Apache site and git pull the static files. I was thinking we test that role class first on a cloud VPS with something like wikibase.wmflabs.org or whatever until we are happy to move it to prod. I also started it as a profile following the new puppet style guide lines, so that jenkins-bot likes it now also with the new style check. I made the server name and admin email variable and in templates/Hiera, so it will be easy to change for labs (just a wiki page edit in Hiera: namespace, not even a commit) and also to move it to prod and the actual domain name.

Thu, Oct 5, 4:47 AM · Patch-For-Review, Traffic, wikiba.se, Operations, Wikidata-Sprint-2016-11-08, Wikidata

Wed, Oct 4

Dzahn added a comment to T164030: setup releases1001.eqiad.wmnet (was: setup mwreleases1001).

https://releases-jenkins.wikimedia.org works now without the /ci/ prefix, is eqiad-only (while releases.wm.org is still active/active on both), Apache syntax isn't broken anymore, puppet run is fine and jenkins says we removed 4 violations.

Wed, Oct 4, 9:28 PM · Release-Engineering-Team (Kanban), RelEng-Archive-FY201718-Q1, Patch-For-Review, Operations, Security-General
Dzahn added a comment to T176841: Create wikibase/wikiba.se-deploy repo.

Heh, that worked ! Thanks @QChris :)

Wed, Oct 4, 6:50 PM · Wikidata-Sprint, wikiba.se, Operations, Wikidata-Sprint-2016-11-08, Wikidata
Dzahn added a comment to T164030: setup releases1001.eqiad.wmnet (was: setup mwreleases1001).

Thanks @elukey , that issue was known to me and the fix was pending in Gerrit at https://gerrit.wikimedia.org/r/#/c/382098/ . I didn't expect cronspam from it though. Service isn't in use yet.

Wed, Oct 4, 5:59 PM · Release-Engineering-Team (Kanban), RelEng-Archive-FY201718-Q1, Patch-For-Review, Operations, Security-General
Dzahn closed T165348: Check long-running screen/tmux sessions as Resolved.
Wed, Oct 4, 5:54 PM · Patch-For-Review, monitoring, Operations
Dzahn added a comment to T165348: Check long-running screen/tmux sessions.

Yes, eh.. tentatively closing :) Of course we can still comment here and reopen if necessary.

Wed, Oct 4, 5:54 PM · Patch-For-Review, monitoring, Operations
Dzahn updated subscribers of T176841: Create wikibase/wikiba.se-deploy repo.

Looks good, thanks. When i needed one in the past i always got a response from that wiki page and usually @QChris did them.

Wed, Oct 4, 5:52 PM · Wikidata-Sprint, wikiba.se, Operations, Wikidata-Sprint-2016-11-08, Wikidata
Dzahn added a comment to T176532: Gerrit is failing to connect to db on gerrit2001 thus preventing systemd from working.

Yea, this should just wait for the proper setup in codfw. I don't see an emergency here.

Wed, Oct 4, 5:50 PM · DBA, Patch-For-Review, Release-Engineering-Team (Next), Operations, Gerrit
Dzahn added a comment to T176754: Regularly purge expired temporary userrights from DB tables.

..of course pending that DBAs are ok with this running as cron.

Wed, Oct 4, 12:24 AM · MW-1.31-release-notes (WMF-deploy-2017-10-10 (1.31.0-wmf.3)), Patch-For-Review, DBA, Operations, Community-Tech, MediaWiki-General-or-Unknown, Stewards-and-global-tools (Temporary-UserRights)
Dzahn added a comment to T176754: Regularly purge expired temporary userrights from DB tables.

Yep, once you know what command exactly you want to run you can ping me about adding that to a puppetized cron job. And yes, modules/mediawiki/manifests/maintenance/ is the right place to look at to see existing examples.

Wed, Oct 4, 12:23 AM · MW-1.31-release-notes (WMF-deploy-2017-10-10 (1.31.0-wmf.3)), Patch-For-Review, DBA, Operations, Community-Tech, MediaWiki-General-or-Unknown, Stewards-and-global-tools (Temporary-UserRights)

Tue, Oct 3

Dzahn claimed T164030: setup releases1001.eqiad.wmnet (was: setup mwreleases1001).
Tue, Oct 3, 8:08 PM · Release-Engineering-Team (Kanban), RelEng-Archive-FY201718-Q1, Patch-For-Review, Operations, Security-General
Dzahn added a comment to T165348: Check long-running screen/tmux sessions.

There are no more alerts now. All the screens/tmux on puppetmaster are closed now. Deployment servers and rhenium are whitelisted. So right now Icinga is clean.

Tue, Oct 3, 7:56 PM · Patch-For-Review, monitoring, Operations

Mon, Oct 2

Dzahn added a comment to T169534: Expired certification with HSTS making wikiapiary.com inaccessible.

Yea, so the fix here is to just wait. I think there is not much else that can be done (besides buying a new cert just for these few days). Not sure what this means about "UBN" status. It is what it is.

Mon, Oct 2, 8:03 PM · WikiApiary
Dzahn added a project to T176364: Request access to logstash (nda group) for @framawiki: WMF-Legal.
Mon, Oct 2, 6:43 PM · WMF-Legal, LDAP-Access-Requests, WMF-NDA-Requests, Operations
Dzahn added a comment to T176364: Request access to logstash (nda group) for @framawiki.

Thanks Victoria!

Mon, Oct 2, 6:41 PM · WMF-Legal, LDAP-Access-Requests, WMF-NDA-Requests, Operations
Dzahn removed a member for WMF-NDA-Requests: Framawiki.
Mon, Oct 2, 6:40 PM
Dzahn added a member for WMF-NDA-Requests: Framawiki.
Mon, Oct 2, 6:37 PM
Dzahn added a comment to T136735: create endowment.wm.org microsite.

@kaythaney Thank you for the update. It's appreciated!:)

Mon, Oct 2, 5:01 PM · Patch-For-Review, Operations

Sat, Sep 30

Dzahn closed T176535: Miraheze wikistats new wikis not updating as Resolved.

should be good now, not entirely sure what actually stopped it before though. please reopen if it doesn't update again in a week

Sat, Sep 30, 6:45 AM · VPS-project-Wikistats
Dzahn added a comment to T176535: Miraheze wikistats new wikis not updating.

And now, double checking why the cron job might have failed, i ran the import command again which wiped the data, :p I will have to let it run once again, but nice way to confirm it. I see no reason for it failing. Running it as the same user with the same permissions.

Sat, Sep 30, 5:07 AM · VPS-project-Wikistats
Dzahn added a comment to T176535: Miraheze wikistats new wikis not updating.

FYI, the 15 either deny access (readapidenied) or have certificate issues or something else

Sat, Sep 30, 4:58 AM · VPS-project-Wikistats