Page MenuHomePhabricator

Dzahn (Daniel Zahn)
Operations EngineerAdministrator

Projects (28)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Friday

  • Clear sailing ahead.

User Details

User Since
Sep 30 2014, 4:39 PM (350 w, 19 h)
Roles
Administrator
Availability
Available
IRC Nick
mutante
LDAP User
Dzahn
MediaWiki User
Unknown

Recent Activity

Yesterday

Dzahn added a comment to T283582: Flapping codfw management alarm ( contint2001.mgmt/SSH is CRITICAL ).

I can confirm since a while these have been happening. The pattern is always:

Tue, Jun 15, 5:31 PM · DC-Ops, netops, serviceops, Continuous-Integration-Infrastructure
Dzahn added a comment to T274463: Backups for GitLab.
[backup1001:~] $ sudo bconsole
Connecting to Director backup1001.eqiad.wmnet:9101
1000 OK: 103 backup1001.eqiad.wmnet Version: 9.4.2 (04 February 2019)
Enter a period to cancel a command.
*restore
..
     5: Select the most recent backup for a client
..
    96: gitlab1001.wikimedia.org-fd
..
Select the Client (1-228): 96
The defined FileSet resources are:
Selection list for "FileSet" is empty!
No FileSet found for client "gitlab1001.wikimedia.org-fd".
Tue, Jun 15, 5:25 PM · Data-Persistence-Backup, Patch-For-Review, User-brennen, GitLab (Initialization)
Dzahn added a comment to T274463: Backups for GitLab.

Let's call it Done once we have confirmed on the Bacula side that there is a full backup and it's restorable.

Tue, Jun 15, 5:22 PM · Data-Persistence-Backup, Patch-For-Review, User-brennen, GitLab (Initialization)
Dzahn added a comment to T284885: Create Wikipedia Tachelhit.

added to DNS

Tue, Jun 15, 5:22 PM · MW-1.37-notes (1.37.0-wmf.11; 2021-06-21), Patch-For-Review, User-Urbanecm, Wiki-Setup (Create)
Dzahn added a comment to T284450: Create Wikipedia Dagbani.

host dag.wikipedia.org
dag.wikipedia.org is an alias for dyna.wikimedia.org.

Tue, Jun 15, 5:19 PM · MW-1.37-notes (1.37.0-wmf.9; 2021-06-07), Patch-For-Review, User-Urbanecm, Wiki-Setup (Create)

Fri, Jun 11

Dzahn added a comment to T283771: Allow idrac tftp fetching of firmware updates (either to existing tftp or new solution).

I reverted the firewall (ferm) change that allowed mgmt to connect to install since as comments above say it couldn't be used for firmware upgrades.

Fri, Jun 11, 11:39 PM · Patch-For-Review, SRE-tools, SRE, netops, DC-Ops
Dzahn added a comment to T243057: Move Prometheus off eqsin/ulsfo/esams bastions.

I agree, pretty sure this only happens when we add/remove disks, never happened randomly on just a reboot to me.

Fri, Jun 11, 7:12 PM · Patch-For-Review, SRE, observability
Dzahn added a comment to T243057: Move Prometheus off eqsin/ulsfo/esams bastions.

The 150G secondary disk has been removed from the prometheus3001 VM.

Fri, Jun 11, 6:59 PM · Patch-For-Review, SRE, observability
Dzahn added a comment to T243057: Move Prometheus off eqsin/ulsfo/esams bastions.

secondary disk has been removed
Strangely .. network interface was renamed. Interface was ens14 before shutdown, and after rebooting it is ens13.

Fri, Jun 11, 6:55 PM · Patch-For-Review, SRE, observability
Dzahn committed rOCMWf8dfbfd94140: rename test file to correct name, add w3m browser for tests (authored by Dzahn).
rename test file to correct name, add w3m browser for tests
Fri, Jun 11, 2:21 AM
Dzahn committed rOCMW79e353f76825: load mod_rewrite and mod_headers, add content-encoding config (authored by Dzahn).
load mod_rewrite and mod_headers, add content-encoding config
Fri, Jun 11, 2:20 AM
Dzahn committed rOCMWa0e53b5a2ff0: static-bugzilla: add config to serve gzipped HTML and a test file (authored by Dzahn).
static-bugzilla: add config to serve gzipped HTML and a test file
Fri, Jun 11, 2:15 AM

Wed, Jun 9

Dzahn added a comment to T284348: Please create two Ganeti VMs for Wikidough in eqiad.

Here are the MAC addresses, but we still need the DHCP change, I could not upload right now:

Wed, Jun 9, 11:07 PM · Patch-For-Review, SRE, Traffic, vm-requests
Dzahn added a comment to T284348: Please create two Ganeti VMs for Wikidough in eqiad.

dzahn@cumin1001:~$ sudo cookbook sre.ganeti.makevm --vcpus 2 --memory 8 --disk 10 --network public eqiad_D doh1002
Ready to create Ganeti VM doh1002.wikimedia.org in the ganeti01.svc.eqiad.wmnet cluster on row D with 2 vCPUs, 8GB of RAM, 10GB of disk in the public network.

Wed, Jun 9, 9:53 PM · Patch-For-Review, SRE, Traffic, vm-requests
Dzahn updated the task description for T284193: put static-bugzilla HTML dump on dumps servers.
Wed, Jun 9, 9:51 PM · Datasets-Archiving, serviceops-radar
Dzahn added a comment to T284193: put static-bugzilla HTML dump on dumps servers.

I copied the data to @deploy1002:/srv/miscweb (using gerrit:699064 and compressed it over there and it's 440MB .tar.gz with default settings.

Wed, Jun 9, 9:51 PM · Datasets-Archiving, serviceops-radar
Dzahn awarded T283241: Update Freenode IRC related documentation on wikis a Barnstar token.
Wed, Jun 9, 9:47 PM · Documentation, wikimedia-irc-libera
Dzahn added a comment to T284348: Please create two Ganeti VMs for Wikidough in eqiad.

dzahn@cumin1001:~$ sudo cookbook sre.ganeti.makevm --vcpus 2 --memory 8 --disk 10 --network public eqiad_C doh1001
Ready to create Ganeti VM doh1001.wikimedia.org in the ganeti01.svc.eqiad.wmnet cluster on row C with 2 vCPUs, 8GB of RAM, 10GB of disk in the public network.

Wed, Jun 9, 9:43 PM · Patch-For-Review, SRE, Traffic, vm-requests
Dzahn merged task T284682: Degraded RAID on ms-be2038 into T283401: Degraded RAID on ms-be2038.
Wed, Jun 9, 7:22 PM · SRE, ops-codfw
Dzahn merged task T284690: Degraded RAID on ms-be2038 into T283401: Degraded RAID on ms-be2038.
Wed, Jun 9, 7:22 PM · SRE, ops-codfw
Dzahn merged tasks T284690: Degraded RAID on ms-be2038, T284682: Degraded RAID on ms-be2038 into T283401: Degraded RAID on ms-be2038.
Wed, Jun 9, 7:22 PM · User-fgiunchedi, SRE, ops-codfw
Dzahn added a comment to T284193: put static-bugzilla HTML dump on dumps servers.

The size I want to add here is 3.6G _uncompressed_ so it should be only a couple hundred MB in a single .tar.gz that I actually want to add. Given that other discussion there talks about Terabytes.. seems like a drop in the ocean though, maybe ?

Wed, Jun 9, 6:06 PM · Datasets-Archiving, serviceops-radar
Dzahn updated the task description for T284193: put static-bugzilla HTML dump on dumps servers.
Wed, Jun 9, 6:04 PM · Datasets-Archiving, serviceops-radar

Tue, Jun 8

Dzahn updated the task description for T281538: New Service Request miscweb.
Tue, Jun 8, 9:41 PM · Patch-For-Review, Service-deployment-requests, Services, SRE
Dzahn added a comment to T281538: New Service Request miscweb.

reserved port 4111 as public TLS service port for miscweb

Tue, Jun 8, 9:41 PM · Patch-For-Review, Service-deployment-requests, Services, SRE
Dzahn added a project to T284588: Need to ssh with my new laptop: SRE-Access-Requests.
Tue, Jun 8, 7:52 PM · SRE-Access-Requests, SRE
Dzahn closed T266249: Requesting access to production shell groups for JAnstee as Resolved.

Tentatively closing since this sounds like issues are resolved.

Tue, Jun 8, 7:39 PM · Analytics, SRE, SRE-Access-Requests

Mon, Jun 7

Dzahn claimed T284460: Add dagwiki to wikistats.
Mon, Jun 7, 2:37 PM · VPS-project-Wikistats

Sat, Jun 5

Dzahn added a comment to T274463: Backups for GitLab.
  1. You can create private repos as an individual user, by default. (Thinking about it, I guess we probably only want that - if we want it - for trusted contributors.)
Sat, Jun 5, 9:33 PM · Data-Persistence-Backup, Patch-For-Review, User-brennen, GitLab (Initialization)
Dzahn added a comment to T274463: Backups for GitLab.

Alright, so if we can't make public dumps then I don't see a difference between the 2 things, both contain secrets. right?

Sat, Jun 5, 6:18 PM · Data-Persistence-Backup, Patch-For-Review, User-brennen, GitLab (Initialization)
Dzahn committed rOCMW61a0b09791a0: httpd: only load modules actually needed, further simplify config, add links (authored by Dzahn).
httpd: only load modules actually needed, further simplify config, add links
Sat, Jun 5, 12:53 AM

Fri, Jun 4

Dzahn added a comment to T274463: Backups for GitLab.

I think all backups are treated as if they contain secrets, it's not like they are made public. I am not aware of 2 levels of security when it comes to that.

Fri, Jun 4, 11:57 PM · Data-Persistence-Backup, Patch-For-Review, User-brennen, GitLab (Initialization)
Dzahn reassigned T266249: Requesting access to production shell groups for JAnstee from colewhite to JAnstee_WMF.
Fri, Jun 4, 11:54 PM · Analytics, SRE, SRE-Access-Requests
Dzahn added a comment to T274463: Backups for GitLab.

I added /etc/gitlab/config_backup to the suggested backup::set, so it would consist of /srv/gitlab-backup and /etc/gitlab/config_backup as part of a single set called "gitlab".

Fri, Jun 4, 8:09 PM · Data-Persistence-Backup, Patch-For-Review, User-brennen, GitLab (Initialization)
Dzahn closed T284197: `Failed to fetch http://mirrors.wikimedia.org/debian/pool/main/z/zvbi/libzvbi0_0.2.35-18_amd64.deb Error reading from server. Remote end closed connection [IP: 208.80.154.15 80]` on build of a bullseye CI image as Invalid.

I can also wget http://mirrors.wikimedia.org/debian/pool/main/f/ffmpeg/libavcodec58_4.3.2-0+deb11u1_amd64.deb without problems now.

Fri, Jun 4, 7:21 PM · serviceops
Dzahn reassigned T266249: Requesting access to production shell groups for JAnstee from JAnstee_WMF to colewhite.
Fri, Jun 4, 7:08 PM · Analytics, SRE, SRE-Access-Requests
Dzahn added a comment to T278729: cp1087 down with hardware issues.

18:38 < icinga-wm> PROBLEM - Check systemd state on cp1087 is CRITICAL: CRITICAL - degraded: The following units failed: rsyslog.service,syslog.socket

https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state
Fri, Jun 4, 6:41 PM · ops-eqiad, SRE, Traffic
Dzahn added a comment to T276148: SSH Access of Git data in GitLab.

Sure you want to open ssh to the public before backups and logging tasks are done?

Fri, Jun 4, 6:28 PM · Patch-For-Review, Release-Engineering-Team (Doing), SRE, User-brennen, GitLab (Initialization)
Dzahn closed T274460: Shell access for VMs for Speed & Function contractors, a subtask of T274458: Remove Speed & Function blockers for GitLab work, as Resolved.
Fri, Jun 4, 6:27 PM · Release-Engineering-Team (Doing), User-brennen, GitLab (Initialization)
Dzahn closed T274460: Shell access for VMs for Speed & Function contractors as Resolved.

boldly closing this - shell access was done quite some time ago and there have been no further issues that we heard of

Fri, Jun 4, 6:27 PM · GitLab (Initialization)
Dzahn closed T274458: Remove Speed & Function blockers for GitLab work as Resolved.

boldly closing this since there obviously aren't blockers anymore or contractors wouldn't have been able to work all this time

Fri, Jun 4, 6:26 PM · Release-Engineering-Team (Doing), User-brennen, GitLab (Initialization)
Dzahn closed T276144: open firewall ports on gitlab1001.wikimedia.org (was: Port map of how Gitlab is accessed) as Resolved.
ACCEPT     tcp  --  anywhere             gitlab.wikimedia.org  tcp dpt:http
ACCEPT     tcp  --  anywhere             gitlab.wikimedia.org  tcp dpt:https
ACCEPT     tcp  --  anywhere             gitlab.wikimedia.org  tcp dpt:ssh
Fri, Jun 4, 6:25 PM · SRE, Traffic, Patch-For-Review, User-brennen, GitLab (Initialization), Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3))
Dzahn closed T276144: open firewall ports on gitlab1001.wikimedia.org (was: Port map of how Gitlab is accessed), a subtask of T274458: Remove Speed & Function blockers for GitLab work, as Resolved.
Fri, Jun 4, 6:24 PM · Release-Engineering-Team (Doing), User-brennen, GitLab (Initialization)
Dzahn closed T276144: open firewall ports on gitlab1001.wikimedia.org (was: Port map of how Gitlab is accessed), a subtask of T276170: DNS for GitLab, as Resolved.
Fri, Jun 4, 6:24 PM · SRE, Traffic, DNS, serviceops, Release-Engineering-Team-TODO, User-brennen, GitLab (Initialization)
Dzahn added a comment to T276144: open firewall ports on gitlab1001.wikimedia.org (was: Port map of how Gitlab is accessed).

Thanks! I think the SSH part was T276148 (shouldnt that be closed now? We can't have ssh port open before the ssh setup is done, surely?) and separate from this ticket though.

Fri, Jun 4, 6:22 PM · SRE, Traffic, Patch-For-Review, User-brennen, GitLab (Initialization), Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3))
Dzahn added a comment to T276148: SSH Access of Git data in GitLab.

per T276144#7133694 the SSH port is open to the world now.

Fri, Jun 4, 6:20 PM · Patch-For-Review, Release-Engineering-Team (Doing), SRE, User-brennen, GitLab (Initialization)
Dzahn added a comment to T274463: Backups for GitLab.

Alright, thanks! That's not even on a different note but exactly the kind of feedback needed. So we'll have to add /etc/gitlab/config_backup to our new backup::set. It would't be separate though. In which way are they supposed to be separate?

Fri, Jun 4, 6:13 PM · Data-Persistence-Backup, Patch-For-Review, User-brennen, GitLab (Initialization)
Dzahn added a comment to T265936: make noc.wikimedia.org active/active (was: improve mw maintenance server switch over and discovery names).

.

Fri, Jun 4, 3:00 AM · Patch-For-Review, serviceops, SRE
Dzahn added a comment to T284246: Please create two Ganeti VMs for Wikidough in eqsin.

@ssingh doh5001.wikimedia.org is ready for you now. doh5002 on hold for lack of IP in that subnet.

Fri, Jun 4, 12:04 AM · SRE, Traffic, vm-requests

Thu, Jun 3

Dzahn updated subscribers of T266249: Requesting access to production shell groups for JAnstee.

@JAnstee_WMF Alright, I made a patch to replace your key and uploaded it to code review.

Thu, Jun 3, 11:18 PM · Analytics, SRE, SRE-Access-Requests
Dzahn claimed T284246: Please create two Ganeti VMs for Wikidough in eqsin.
Thu, Jun 3, 11:07 PM · SRE, Traffic, vm-requests
Dzahn added a comment to T266249: Requesting access to production shell groups for JAnstee.

If it turns out you need to make new keys, just paste the public part here and ask us to update the repository.

Thu, Jun 3, 8:33 PM · Analytics, SRE, SRE-Access-Requests
Dzahn reopened T266249: Requesting access to production shell groups for JAnstee as "Open".
Thu, Jun 3, 8:33 PM · Analytics, SRE, SRE-Access-Requests
Dzahn added a comment to T266249: Requesting access to production shell groups for JAnstee.

Ok, thank you. Hmm.. Let's try this:

Thu, Jun 3, 8:32 PM · Analytics, SRE, SRE-Access-Requests
Dzahn added a comment to T277163: Prometheus PoPs disk space utilization.

ACK, thanks, @fgiunchedi As commented on the linked ticket, multiple people have mentioned we have metal there that prometheus could move to. Seems to me that is what we should probably do (instead of using that hardware as another ganeti server).

Thu, Jun 3, 8:17 PM · SRE, serviceops-radar, User-fgiunchedi, observability
Dzahn added a comment to T243057: Move Prometheus off eqsin/ulsfo/esams bastions.

ACK. thanks all. Several people have mentioned the part that we have metal there this could move to. It seems to make sense to me to move prometheus to that and outside of ganeti (rather than adding another ganeti server).

Thu, Jun 3, 8:15 PM · Patch-For-Review, SRE, observability
Dzahn added a comment to T284246: Please create two Ganeti VMs for Wikidough in eqsin.

Thu, Jun 3, 7:53 PM · SRE, Traffic, vm-requests
Dzahn updated subscribers of T284246: Please create two Ganeti VMs for Wikidough in eqsin.

@ssingh @BBlack Our issue over here is lack of the resource of .. public IPs, it looks:

Thu, Jun 3, 7:49 PM · SRE, Traffic, vm-requests
Dzahn added a comment to T284246: Please create two Ganeti VMs for Wikidough in eqsin.

doh5001 has been created but doh5002 hit resource limits here as well, even though we just used 10G disk, it is maybe another resource:

Thu, Jun 3, 7:45 PM · SRE, Traffic, vm-requests
Dzahn added a comment to T251918: docker-reporter-releng-images failed on deneb.

May 31 16:02:36 deneb docker-report-releng[31493]: ERROR[docker-report] Debmonitor report for image docker-registry.wikimedia.org/releng/quibble-jessie-php56:0.0.31-1 failed

Thu, Jun 3, 7:39 PM · User-jbond, Patch-For-Review, serviceops, SRE
Dzahn added a comment to T251918: docker-reporter-releng-images failed on deneb.

2021-06-03 19:19:30 3d 2h 47m 35s 3/3 CRITICAL - degraded: The following units failed: docker-reporter-releng-images.service

Thu, Jun 3, 7:31 PM · User-jbond, Patch-For-Review, serviceops, SRE
Dzahn added a comment to T164456: Migrate to nginx-light.

checked on install* that nginx-full is gone, nginx-light is there and restarted nginx to be sure

Thu, Jun 3, 7:22 PM · Patch-For-Review, User-jbond, User-MoritzMuehlenhoff, User-ArielGlenn, Traffic, SRE
Dzahn added a comment to T266249: Requesting access to production shell groups for JAnstee.
440601 Jun  2 23:05:38 bast4003 sshd[6968]: Accepted key ED25519 SHA256:plaVmNDA1Ug/00RQCUV2WfIKRDNwP7GLq9NouyMKMJM found at /etc/ssh/userkeys/janstee:1
440602 Jun  2 23:05:38 bast4003 sshd[6968]: Postponed publickey for janstee from 198.27.180.90 port 60707 ssh2 [preauth]
Thu, Jun 3, 7:01 PM · Analytics, SRE, SRE-Access-Requests
Dzahn added a comment to T266249: Requesting access to production shell groups for JAnstee.

Would you mind pasting the contents of /Users/janstee/.ssh/config, Jaime?

Thu, Jun 3, 6:49 PM · Analytics, SRE, SRE-Access-Requests
Dzahn added a comment to T278729: cp1087 down with hardware issues.

END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on cp1087.eqiad.wmnet with reason: replaced DIMM https://phabricator.wikimedia.org/T278729

Thu, Jun 3, 6:38 PM · ops-eqiad, SRE, Traffic
Dzahn reopened T278729: cp1087 down with hardware issues as "Open".

https://icinga.wikimedia.org/cgi-bin/icinga/status.cgi?host=cp1087

Thu, Jun 3, 6:34 PM · ops-eqiad, SRE, Traffic
Dzahn added a comment to T266249: Requesting access to production shell groups for JAnstee.

debug1: Executing proxy command: exec ssh -a -W stat1006.ulsfo.wmnet:22 janstee@bast4003.wikimedia.org

Thu, Jun 3, 4:15 PM · Analytics, SRE, SRE-Access-Requests

Wed, Jun 2

Dzahn updated the task description for T283852: Please create two Ganeti VMs for Wikidough in esams.
Wed, Jun 2, 11:11 PM · SRE, Traffic, vm-requests
Dzahn closed T283852: Please create two Ganeti VMs for Wikidough in esams as Resolved.
Wed, Jun 2, 11:10 PM · SRE, Traffic, vm-requests
Dzahn added a comment to T283852: Please create two Ganeti VMs for Wikidough in esams.
[doh3001:~] $ df -h
Filesystem      Size  Used Avail Use% Mounted on
..
/dev/vda1       8.9G  1.6G  6.8G  19% /
Wed, Jun 2, 11:06 PM · SRE, Traffic, vm-requests
Dzahn added a comment to T283852: Please create two Ganeti VMs for Wikidough in esams.

VMs have been created.

Wed, Jun 2, 11:05 PM · SRE, Traffic, vm-requests
Dzahn added a comment to T283852: Please create two Ganeti VMs for Wikidough in esams.

We ran out of resources on esams ganeti (disk). Creating machines with just 10G instead after discussion.

Wed, Jun 2, 10:41 PM · SRE, Traffic, vm-requests
Dzahn added projects to T277163: Prometheus PoPs disk space utilization: serviceops-radar, SRE.
Wed, Jun 2, 9:40 PM · SRE, serviceops-radar, User-fgiunchedi, observability
Dzahn updated subscribers of T277163: Prometheus PoPs disk space utilization.
Wed, Jun 2, 9:39 PM · SRE, serviceops-radar, User-fgiunchedi, observability
Dzahn added a comment to T277163: Prometheus PoPs disk space utilization.

Today we tried to create 2 new VMs in the ganeti esams cluster and actually ran out of resources.

Wed, Jun 2, 9:38 PM · SRE, serviceops-radar, User-fgiunchedi, observability
Dzahn added a comment to T243057: Move Prometheus off eqsin/ulsfo/esams bastions.

When we recreate the bastions without prometheus, we don't need to use 40GB disk anymore, right?

Wed, Jun 2, 9:31 PM · Patch-For-Review, SRE, observability
Dzahn added a subtask for T281538: New Service Request miscweb: T284193: put static-bugzilla HTML dump on dumps servers.
Wed, Jun 2, 9:28 PM · Patch-For-Review, Service-deployment-requests, Services, SRE
Dzahn added a parent task for T284193: put static-bugzilla HTML dump on dumps servers: T281538: New Service Request miscweb.
Wed, Jun 2, 9:28 PM · Datasets-Archiving, serviceops-radar
Dzahn updated subscribers of T284193: put static-bugzilla HTML dump on dumps servers.
Wed, Jun 2, 9:26 PM · Datasets-Archiving, serviceops-radar
Dzahn created T284193: put static-bugzilla HTML dump on dumps servers.
Wed, Jun 2, 9:26 PM · Datasets-Archiving, serviceops-radar
Dzahn raised the priority of T283852: Please create two Ganeti VMs for Wikidough in esams from Medium to High.
Wed, Jun 2, 7:37 PM · SRE, Traffic, vm-requests
Dzahn claimed T283852: Please create two Ganeti VMs for Wikidough in esams.
Wed, Jun 2, 7:37 PM · SRE, Traffic, vm-requests
Dzahn added a comment to T274463: Backups for GitLab.

all interested, please take a look at the latest Gerrit link above and feel free to review/comment there.

Wed, Jun 2, 7:24 PM · Data-Persistence-Backup, Patch-For-Review, User-brennen, GitLab (Initialization)
Dzahn added a comment to T274463: Backups for GitLab.

We now have the Bacula client (fd for file daemon) running on gitlab1001. It

[gitlab1001:~] $ ps aux | grep bacula
root     11735  0.0  0.0  26140  7684 ?        Ssl  19:00   0:00 /usr/sbin/bacula-fd -fP -c /etc/bacula/bacula-fd.conf
Wed, Jun 2, 7:04 PM · Data-Persistence-Backup, Patch-For-Review, User-brennen, GitLab (Initialization)
Dzahn added a comment to T274463: Backups for GitLab.

https://puppet-compiler.wmflabs.org/compiler1002/29784/gitlab1001.wikimedia.org/index.html

Wed, Jun 2, 6:56 PM · Data-Persistence-Backup, Patch-For-Review, User-brennen, GitLab (Initialization)
Dzahn added a comment to T274463: Backups for GitLab.

There are 2 things needed to make Bacula backups work on a given host.

Wed, Jun 2, 6:42 PM · Data-Persistence-Backup, Patch-For-Review, User-brennen, GitLab (Initialization)
Dzahn updated subscribers of T281538: New Service Request miscweb.
git clone "https://gerrit.wikimedia.org/r/operations/container/miscweb"
Wed, Jun 2, 1:06 AM · Patch-For-Review, Service-deployment-requests, Services, SRE
Dzahn committed rOCMW7c1cfe586df6: add httpd site config and index.html for static-bugzilla (authored by Dzahn).
add httpd site config and index.html for static-bugzilla
Wed, Jun 2, 1:00 AM

Tue, Jun 1

Dzahn committed rOCMWf2c8c9b1fd68: blubber: use a single html dir for both staging and prod variants (authored by Dzahn).
blubber: use a single html dir for both staging and prod variants
Tue, Jun 1, 11:53 PM
Dzahn committed rOCMWb11fdb9cc250: blubber: install curl in container for testing httpd (authored by Dzahn).
blubber: install curl in container for testing httpd
Tue, Jun 1, 9:19 PM
Dzahn committed rOCMW4e3986513a52: blubber: simplify path to HTML doc roots (authored by Dzahn).
blubber: simplify path to HTML doc roots
Tue, Jun 1, 9:18 PM
Dzahn committed rOCMW58c7d528c6fb: blubber: set runs: { insecurely: false } (authored by Dzahn).
blubber: set runs: { insecurely: false }
Tue, Jun 1, 9:07 PM
Dzahn committed rOCMW9310338f19f2: install vim in container for debugging (authored by Dzahn).
install vim in container for debugging
Tue, Jun 1, 8:26 PM
Dzahn committed rOCMW5acd6b9966bc: greatly simplify httpd.conf (authored by Dzahn).
greatly simplify httpd.conf
Tue, Jun 1, 8:25 PM
Dzahn added a comment to T280540: (Need By: TBD) rack/setup/install phab1004 (was: phab1002).

thanks, Rob! the continuation of this will be T280597

Tue, Jun 1, 6:31 PM · SRE, ops-eqiad, DC-Ops
Dzahn added a comment to T283975: Create cron job to update query page Special:OrphanedTimedText on wikis with $wgEnableLocalTimedText = true.

note: if a 'periodic job' is needed these are not literally cron jobs anymore, they are nowadays systemd timers and in other, non-mediawiki, places we are still in the process of converting from cron to systemd. The puppet class 'profile::mediawiki::periodic_job' should be used for this, that will create a service and a timer on the mwmaint hosts.

Tue, Jun 1, 6:29 PM · MW-1.37-notes (1.37.0-wmf.9; 2021-06-07), TimedMediaHandler-TimedText, serviceops

Sun, May 30

Dzahn added a comment to T283930: Add WikiDex and SmashPedia.

Ok, thanks for additional info! cheers

Sun, May 30, 3:24 PM · VPS-project-Wikistats

Sat, May 29

Dzahn closed T283903: Add JoJoWiki to wikistats as Resolved.
Sat, May 29, 5:39 PM · VPS-project-Wikistats
Dzahn added a comment to T283903: Add JoJoWiki to wikistats.

MariaDB [wikistats]> insert into mediawikis (statsurl, method) values ("https://jojowiki.com/api.php",8);

Sat, May 29, 5:38 PM · VPS-project-Wikistats
Dzahn closed T283930: Add WikiDex and SmashPedia as Resolved.
Sat, May 29, 5:37 PM · VPS-project-Wikistats