Page MenuHomePhabricator

LSobanski (Lukasz Sobanski)
Woo$

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Wednesday

  • Clear sailing ahead.

User Details

User Since
Aug 31 2020, 5:40 PM (132 w, 6 d)
Availability
Available
LDAP User
LSobanski
MediaWiki User
LSobanski (WMF) [ Global Accounts ]

Recent Activity

Yesterday

LSobanski updated the task description for T332391: Training checklist runbook review (Sprint Week 2023-03).
Sun, Mar 19, 7:48 PM · SRE

Fri, Mar 17

LSobanski created T332391: Training checklist runbook review (Sprint Week 2023-03).
Fri, Mar 17, 11:50 AM · SRE

Thu, Mar 16

LSobanski created SRE-Unowned.
Thu, Mar 16, 9:03 AM

Wed, Mar 15

LSobanski closed T37715: Attachments not shown inline but need downloading: Change "AttachmentDownloadType" setting in configuration as Resolved.
Wed, Mar 15, 3:56 PM · Znuny
LSobanski added a comment to T37715: Attachments not shown inline but need downloading: Change "AttachmentDownloadType" setting in configuration.

Given the age of this request and the recent comments, I'll resolve this task. Please reopen it if you think this still needs to be addressed.

Wed, Mar 15, 3:56 PM · Znuny
LSobanski added a comment to T266965: In Firefox, "Ticket #" field is not clickable.

@Urbanecm could you add a bit more detail on what the "works as intended" state would be here? Right now the experience seems to be the same on both Chrome and Firefox - none of the fields in the top row of the table causes any changes to the table sorting.

Wed, Mar 15, 3:41 PM · Browser-Support-Firefox, Regression, Znuny

Tue, Mar 14

LSobanski added a comment to T330960: switch releases.wikimedia.org from eqiad to codfw.

@hashar A follow up question, is there a plan to configure releases2002 so that the switchover is possible? I'm asking both in terms of testing this within the DC switchover period and longer term operational expectations.

Tue, Mar 14, 5:54 PM · Continuous-Integration-Infrastructure, Datacenter-Switchover, Release-Engineering-Team, serviceops-collab
LSobanski added a project to T331969: Migrate chartmuseum to bullseye: serviceops.
Tue, Mar 14, 12:15 PM · serviceops, SRE

Thu, Mar 9

LSobanski added a comment to T330960: switch releases.wikimedia.org from eqiad to codfw.

@hashar A follow up question, is there a plan to configure releases2002 so that the switchover is possible? I'm asking both in terms of testing this within the DC switchover period and longer term operational expectations.

Thu, Mar 9, 2:17 PM · Continuous-Integration-Infrastructure, Datacenter-Switchover, Release-Engineering-Team, serviceops-collab

Wed, Mar 8

LSobanski moved T330864: Gerrit team dashboard for Collab from Work in Progress to Backlog on the serviceops-collab board.
Wed, Mar 8, 1:05 PM · serviceops-collab

Thu, Mar 2

LSobanski added a comment to T310922: Install NVMe SSDs into moss-be100[1|2] & thanos-be100?.

T279621 is the best place to track the progress of this.

Thu, Mar 2, 3:08 PM · SRE, ops-eqiad, SRE-swift-storage, DC-Ops
LSobanski added a parent task for T310923: Install NVMe SSDs into moss-be200[1|2] & thanos-be200?: T279621: Set up Misc Object Storage Service (moss).
Thu, Mar 2, 3:07 PM · SRE, ops-codfw, SRE-swift-storage, DC-Ops
LSobanski added a subtask for T279621: Set up Misc Object Storage Service (moss): T310923: Install NVMe SSDs into moss-be200[1|2] & thanos-be200?.
Thu, Mar 2, 3:07 PM · SRE-swift-storage
LSobanski added a parent task for T310922: Install NVMe SSDs into moss-be100[1|2] & thanos-be100?: T279621: Set up Misc Object Storage Service (moss).
Thu, Mar 2, 3:07 PM · SRE, ops-eqiad, SRE-swift-storage, DC-Ops
LSobanski added a subtask for T279621: Set up Misc Object Storage Service (moss): T310922: Install NVMe SSDs into moss-be100[1|2] & thanos-be100?.
Thu, Mar 2, 3:07 PM · SRE-swift-storage
LSobanski updated the task description for T330864: Gerrit team dashboard for Collab.
Thu, Mar 2, 3:05 PM · serviceops-collab
LSobanski added a comment to T310922: Install NVMe SSDs into moss-be100[1|2] & thanos-be100?.

@Jclark-ctr the Thanos hosts had to be returned to the Thanos cluster so this request is in limbo until new hosts are procured (most likely next FY). Let me know if any follow up i needed.

Thu, Mar 2, 3:04 PM · SRE, ops-eqiad, SRE-swift-storage, DC-Ops
LSobanski added a comment to T330864: Gerrit team dashboard for Collab.

Instructions how to add custom dashboards to the "Your" menu in Gerrit are here: https://www.mediawiki.org/wiki/Gerrit/Advanced_usage#Add_team_dashboard_to_%22Your%22_menu

Thu, Mar 2, 2:46 PM · serviceops-collab
LSobanski added a comment to T330864: Gerrit team dashboard for Collab.

Dashboard is now ready for visual inspection at https://gerrit.wikimedia.org/r/p/wikimedia/+/dashboard/teams:sre-collab.

Thu, Mar 2, 2:43 PM · serviceops-collab

Wed, Mar 1

LSobanski updated the task description for T330172: Define future design of GitLab backups.
Wed, Mar 1, 1:56 PM · Patch-For-Review, serviceops-collab, GitLab (Infrastructure)
LSobanski updated subscribers of T330864: Gerrit team dashboard for Collab.
Wed, Mar 1, 1:03 PM · serviceops-collab
LSobanski updated the task description for T330864: Gerrit team dashboard for Collab.
Wed, Mar 1, 12:53 PM · serviceops-collab
LSobanski updated subscribers of T330864: Gerrit team dashboard for Collab.

Gerrit group created (thanks to @Ladsgroup)

Wed, Mar 1, 12:53 PM · serviceops-collab
LSobanski moved T330864: Gerrit team dashboard for Collab from Incoming to Work in Progress on the serviceops-collab board.
Wed, Mar 1, 12:52 PM · serviceops-collab
LSobanski triaged T330864: Gerrit team dashboard for Collab as Low priority.
Wed, Mar 1, 12:52 PM · serviceops-collab
LSobanski created T330864: Gerrit team dashboard for Collab.
Wed, Mar 1, 12:52 PM · serviceops-collab

Mon, Feb 27

LSobanski moved T330639: Clarify the process of logging into gitlab-replica using 2FA from Incoming to Backlog on the serviceops-collab board.
Mon, Feb 27, 10:30 AM · GitLab (Auth & Access), serviceops-collab
LSobanski triaged T330639: Clarify the process of logging into gitlab-replica using 2FA as Low priority.
Mon, Feb 27, 10:30 AM · GitLab (Auth & Access), serviceops-collab
LSobanski created T330639: Clarify the process of logging into gitlab-replica using 2FA.
Mon, Feb 27, 10:28 AM · GitLab (Auth & Access), serviceops-collab

Mon, Feb 20

LSobanski added a comment to T327972: create blackbox::http monitoring for contint (integration.wikimedia.org).

This alert is currently firing, see https://logstash.wikimedia.org/app/dashboards#/view/f3e709c0-a5f8-11ec-bf8e-43f1807d5bc2?_g=h@28cc80f&_a=h@5bbe4f8

Mon, Feb 20, 4:33 PM · serviceops-collab
LSobanski moved T330091: Switchover People and Planet services to codfw from Incoming to Backlog on the serviceops-collab board.
Mon, Feb 20, 4:24 PM · Patch-For-Review, Datacenter-Switchover, serviceops-collab
LSobanski moved T329908: Deploy phabricator to aphlict2001.codfw.wmnet from Incoming to Consultation on the serviceops-collab board.
Mon, Feb 20, 4:23 PM · Patch-For-Review, Phabricator, serviceops-collab, Release-Engineering-Team
LSobanski triaged T329931: Switchover gitlab (gitlab1004 -> gitlab2002) as High priority.
Mon, Feb 20, 4:23 PM · Patch-For-Review, GitLab (Infrastructure), serviceops-collab, Datacenter-Switchover
LSobanski moved T329931: Switchover gitlab (gitlab1004 -> gitlab2002) from Incoming to Backlog on the serviceops-collab board.
Mon, Feb 20, 4:23 PM · Patch-For-Review, GitLab (Infrastructure), serviceops-collab, Datacenter-Switchover
LSobanski moved T330090: Switchover static miscweb services to codfw from Incoming to Backlog on the serviceops-collab board.
Mon, Feb 20, 4:23 PM · Patch-For-Review, Datacenter-Switchover, serviceops-collab
LSobanski triaged T330090: Switchover static miscweb services to codfw as High priority.
Mon, Feb 20, 4:23 PM · Patch-For-Review, Datacenter-Switchover, serviceops-collab
LSobanski triaged T330091: Switchover People and Planet services to codfw as High priority.
Mon, Feb 20, 4:23 PM · Patch-For-Review, Datacenter-Switchover, serviceops-collab
LSobanski added a project to T330090: Switchover static miscweb services to codfw: Datacenter-Switchover.
Mon, Feb 20, 3:58 PM · Patch-For-Review, Datacenter-Switchover, serviceops-collab
LSobanski added a project to T330091: Switchover People and Planet services to codfw: Datacenter-Switchover.
Mon, Feb 20, 3:58 PM · Patch-For-Review, Datacenter-Switchover, serviceops-collab
LSobanski created T330091: Switchover People and Planet services to codfw.
Mon, Feb 20, 3:14 PM · Patch-For-Review, Datacenter-Switchover, serviceops-collab
LSobanski renamed T330090: Switchover static miscweb services to codfw from Failover static miscweb services to codfw to Switchover static miscweb services to codfw.
Mon, Feb 20, 3:14 PM · Patch-For-Review, Datacenter-Switchover, serviceops-collab
LSobanski created T330090: Switchover static miscweb services to codfw.
Mon, Feb 20, 3:11 PM · Patch-For-Review, Datacenter-Switchover, serviceops-collab

Feb 17 2023

LSobanski removed a project from T199431: Consider the possibility of separating ChangeProp and JobQueue on Kafka level: SRE.

I don't see any specific actions for SRE, removing the tag, please add us back when needed.

Feb 17 2023, 1:52 PM · Data-Engineering-Icebox, Analytics-Radar, Sustainability (Incident Followup), Platform Engineering (Icebox), ChangeProp, MediaWiki-Core-JobQueue

Feb 16 2023

LSobanski removed a project from T277467: Upgrade to Kafka MirrorMaker 2: SRE.

I don't see a clear action for SRE so I'm removing the tag. Please add us back when needed.

Feb 16 2023, 2:44 PM · Data-Engineering-Icebox, Analytics-Radar
LSobanski removed a project from T228591: Document how to request installing additional fonts for SVG thumbnails and generated PDF files on Wikimedia servers: SRE.

I don't see a clear action for SRE so I'm removing the tag. Please add us back when needed.

Feb 16 2023, 2:43 PM · User-AKlapper, WMF-General-or-Unknown, Documentation, Wikimedia-SVG-rendering
LSobanski removed a project from T266886: Augment NEL reports with a computed timestamp-of-generation: SRE.
Feb 16 2023, 2:38 PM · Data-Engineering-Icebox, Analytics
LSobanski added a project to T125015: Requests to (hard) redirect pages return their target's contents but are counted as pageviews to the redirect page: Traffic.
Feb 16 2023, 2:36 PM · Traffic, Data-Engineering-Icebox, Analytics-Radar, SRE
LSobanski removed a project from T257354: Add backend field to webrequest Hive table: SRE.

I don't see anything actionable for SRE, untagging.

Feb 16 2023, 12:56 PM · Data-Engineering
LSobanski added a project to T329218: LDAP connections use TLSv1.0 and TLSv1.1: Infrastructure-Foundations.
Feb 16 2023, 12:51 PM · Infrastructure-Foundations, SRE, LDAP
LSobanski added a project to T328941: followups to unactionable NELHigh pages due to Telecom Italia outage, 2023-02-05: SRE-OnFire.
Feb 16 2023, 12:50 PM · SRE-Sprint-Week-Sustainability-March2023, Patch-For-Review, SRE-OnFire, Sustainability (Incident Followup)
LSobanski renamed T329587: let all serviceops-collab alertmanager alerts create tickets from let all serviceops-collabd alertmanager alerts create tickets to let all serviceops-collab alertmanager alerts create tickets.
Feb 16 2023, 12:32 PM · Patch-For-Review, serviceops-collab
LSobanski moved T329587: let all serviceops-collab alertmanager alerts create tickets from Incoming to Work in Progress on the serviceops-collab board.
Feb 16 2023, 12:32 PM · Patch-For-Review, serviceops-collab
LSobanski assigned T329587: let all serviceops-collab alertmanager alerts create tickets to Dzahn.
Feb 16 2023, 12:32 PM · Patch-For-Review, serviceops-collab

Feb 15 2023

LSobanski updated the task description for T327404: (Need By:TBD) rack/setup/install rack A1 and A8 new PDUs .
Feb 15 2023, 3:42 PM · SRE, ops-codfw

Feb 13 2023

LSobanski updated the task description for T329073: eqiad row A switches upgrade.
Feb 13 2023, 4:33 PM · Patch-For-Review, Discovery-Search (Current work), Shared-Data-Infrastructure, Data-Engineering-Planning, DBA, SRE, Platform Engineering, Infrastructure-Foundations, Traffic, serviceops, Machine-Learning-Team, cloud-services-team, Data-Persistence, SRE Observability, serviceops-collab

Feb 9 2023

LSobanski added a comment to T328287: CommRel support for March 2023 Datacenter Switchover.
  • GitLab failover requires a ~1.5h maintenance window during which GitLab will be unavailable.
  • We won't be switching Phabricator or Etherpad over.
Feb 9 2023, 1:23 PM · CommRel-Specialists-Support (Jan-Mar-2023), serviceops, Datacenter-Switchover, SRE

Feb 6 2023

LSobanski moved T324659: contint2002 service implementation tracking from Work in Progress to Backlog on the serviceops-collab board.
Feb 6 2023, 4:40 PM · Patch-For-Review, Continuous-Integration-Infrastructure, serviceops-collab, SRE
LSobanski moved T328595: create a phorge instance in the devtools project in wmcs from Backlog to Work in Progress on the serviceops-collab board.
Feb 6 2023, 4:32 PM · serviceops-collab

Jan 30 2023

LSobanski moved T327991: codfw row B switches upgrade from Incoming to Consultation on the serviceops-collab board.
Jan 30 2023, 4:56 PM · Discovery-Search (Current work), SRE, Platform Engineering, serviceops-collab, Infrastructure-Foundations, SRE Observability, Traffic, Machine-Learning-Team, cloud-services-team, Data-Persistence, DBA, serviceops, netops
LSobanski updated the task description for T327991: codfw row B switches upgrade.
Jan 30 2023, 4:56 PM · Discovery-Search (Current work), SRE, Platform Engineering, serviceops-collab, Infrastructure-Foundations, SRE Observability, Traffic, Machine-Learning-Team, cloud-services-team, Data-Persistence, DBA, serviceops, netops
LSobanski moved T327925: codfw row A switches upgrade from Incoming to Consultation on the serviceops-collab board.
Jan 30 2023, 4:49 PM · Shared-Data-Infrastructure, Data-Engineering-Planning, Discovery-Search (Current work), DBA, serviceops, Traffic, Machine-Learning-Team, serviceops-collab, cloud-services-team, Platform Engineering, SRE Observability, Data-Persistence, SRE, netops, Infrastructure-Foundations
LSobanski updated the task description for T327925: codfw row A switches upgrade.
Jan 30 2023, 4:48 PM · Shared-Data-Infrastructure, Data-Engineering-Planning, Discovery-Search (Current work), DBA, serviceops, Traffic, Machine-Learning-Team, serviceops-collab, cloud-services-team, Platform Engineering, SRE Observability, Data-Persistence, SRE, netops, Infrastructure-Foundations
LSobanski moved T327435: add disk space usage to grafana dashboard for gitlab-runners from Incoming to Work in Progress on the serviceops-collab board.
Jan 30 2023, 4:41 PM · GitLab (CI & Job Runners), serviceops-collab
LSobanski triaged T327435: add disk space usage to grafana dashboard for gitlab-runners as Medium priority.
Jan 30 2023, 4:41 PM · GitLab (CI & Job Runners), serviceops-collab
LSobanski moved T327771: Create an internal Znuny DEB package from Incoming to Backlog on the serviceops-collab board.
Jan 30 2023, 4:39 PM · serviceops-collab
LSobanski triaged T327771: Create an internal Znuny DEB package as Medium priority.
Jan 30 2023, 4:39 PM · serviceops-collab
LSobanski changed the status of T327771: Create an internal Znuny DEB package from Open to Stalled.

Stalled until we run Bullseye.

Jan 30 2023, 4:38 PM · serviceops-collab
LSobanski moved T328283: Cleanup and attach volumes for gitlab-runners WMCS project from Incoming to Backlog on the serviceops-collab board.
Jan 30 2023, 4:35 PM · Release-Engineering-Team (Radar), GitLab (CI & Job Runners), serviceops-collab
LSobanski triaged T328283: Cleanup and attach volumes for gitlab-runners WMCS project as Low priority.
Jan 30 2023, 4:35 PM · Release-Engineering-Team (Radar), GitLab (CI & Job Runners), serviceops-collab
LSobanski moved T327972: create blackbox::http monitoring for contint (integration.wikimedia.org) from Incoming to Backlog on the serviceops-collab board.
Jan 30 2023, 4:33 PM · serviceops-collab
LSobanski moved T327974: create blackbox::http monitoring for etherpad.wikimedia.org from Incoming to Backlog on the serviceops-collab board.
Jan 30 2023, 4:33 PM · serviceops-collab
LSobanski moved T327975: create blackbox::http monitoring for releases.wikimedia.org from Incoming to Backlog on the serviceops-collab board.
Jan 30 2023, 4:33 PM · serviceops-collab
LSobanski triaged T327977: create blackbox::http monitoring for planet as Medium priority.
Jan 30 2023, 4:33 PM · serviceops-collab
LSobanski moved T327977: create blackbox::http monitoring for planet from Incoming to Backlog on the serviceops-collab board.
Jan 30 2023, 4:33 PM · serviceops-collab
LSobanski moved T327976: create blackbox::http monitoring for miscweb from Incoming to Backlog on the serviceops-collab board.
Jan 30 2023, 4:33 PM · serviceops-collab
LSobanski moved T327973: create blackbox::http monitoring for doc.wikimedia.org from Incoming to Backlog on the serviceops-collab board.
Jan 30 2023, 4:33 PM · serviceops-collab
LSobanski moved T327978: create blackbox::http monitoring for RT from Incoming to Backlog on the serviceops-collab board.
Jan 30 2023, 4:33 PM · serviceops-collab
LSobanski triaged T327973: create blackbox::http monitoring for doc.wikimedia.org as Medium priority.
Jan 30 2023, 4:33 PM · serviceops-collab
LSobanski triaged T327974: create blackbox::http monitoring for etherpad.wikimedia.org as Medium priority.
Jan 30 2023, 4:33 PM · serviceops-collab
LSobanski triaged T327978: create blackbox::http monitoring for RT as Medium priority.
Jan 30 2023, 4:33 PM · serviceops-collab
LSobanski triaged T327975: create blackbox::http monitoring for releases.wikimedia.org as Medium priority.
Jan 30 2023, 4:33 PM · serviceops-collab
LSobanski triaged T327976: create blackbox::http monitoring for miscweb as Medium priority.
Jan 30 2023, 4:32 PM · serviceops-collab
LSobanski triaged T327972: create blackbox::http monitoring for contint (integration.wikimedia.org) as Medium priority.
Jan 30 2023, 4:28 PM · serviceops-collab
LSobanski moved T321758: create logstash dashboards for serviceops-collab services from Backlog to Work in Progress on the serviceops-collab board.
Jan 30 2023, 4:14 PM · serviceops-collab

Jan 25 2023

LSobanski updated the task description for T327925: codfw row A switches upgrade.
Jan 25 2023, 6:50 PM · Shared-Data-Infrastructure, Data-Engineering-Planning, Discovery-Search (Current work), DBA, serviceops, Traffic, Machine-Learning-Team, serviceops-collab, cloud-services-team, Platform Engineering, SRE Observability, Data-Persistence, SRE, netops, Infrastructure-Foundations

Jan 24 2023

LSobanski created T327772: Automatically created ProbeDown tasks for Collab services.
Jan 24 2023, 11:51 AM · serviceops-collab
LSobanski created T327771: Create an internal Znuny DEB package.
Jan 24 2023, 11:44 AM · serviceops-collab

Jan 23 2023

LSobanski moved T191182: Migrate active repositories in Phabricator Differential to GitLab from Work in Progress to Backlog on the serviceops-collab board.
Jan 23 2023, 4:24 PM · Release-Engineering-Team (Priority Backlog 📥), serviceops-collab, User-AKlapper, GitLab (Project Migration), Epic, Technical-Debt, Phabricator
LSobanski moved T324171: Audit Diffusion-Repository-Administrators group membership and rights from Work in Progress to Backlog on the serviceops-collab board.
Jan 23 2023, 4:24 PM · Release-Engineering-Team, serviceops-collab, Phabricator, Diffusion-Repository-Administrators
LSobanski removed a project from T273738: Get platform engineering team green light for Cloud NAT to wikis change: SRE.
Jan 23 2023, 12:30 PM · cloud-services-team, Platform Engineering
LSobanski edited Description on SRE.
Jan 23 2023, 11:18 AM

Jan 20 2023

LSobanski added a comment to T327068: Bullseye upgrade for remaining Collab hosts.

Comment from Moritz about requesttracker: Since RT is packaged natively in Debian (https://tracker.debian.org/pkg/request-tracker4) and is read-only anyway, I'd recommend simply upgrading it in place to Bullseye. This extends its life time by two years and then in 2025 we can revisit the decommission conversation.

Jan 20 2023, 1:06 PM · Patch-For-Review, serviceops-collab
LSobanski moved T327405: Decommission Racktables from Incoming to Work in Progress on the serviceops-collab board.
Jan 20 2023, 1:03 PM · Data-Persistence (work done), serviceops-collab

Jan 18 2023

LSobanski closed T327242: ProbeDown as Resolved.

This was an intentional reboot in T326815, resolving.

Jan 18 2023, 12:25 PM · serviceops-collab

Jan 17 2023

LSobanski assigned T326888: rsync server on people2002 to Dzahn.
Jan 17 2023, 4:46 PM · serviceops-collab, SRE
LSobanski assigned T324659: contint2002 service implementation tracking to Dzahn.
Jan 17 2023, 4:45 PM · Patch-For-Review, Continuous-Integration-Infrastructure, serviceops-collab, SRE
LSobanski closed T325385: Trusted gitlab runner containers need access to staging k8s cluster as Resolved.

Resolving as it seems like the original request was addressed. If there's follow up discussion then let's create a separate task for that.

Jan 17 2023, 4:45 PM · Kubernetes, serviceops, serviceops-collab, GitLab
LSobanski assigned T326125: apache2 fails to start after gerrit hosts are rebooted to Dzahn.
Jan 17 2023, 4:42 PM · Release-Engineering-Team, Gerrit, serviceops-collab
LSobanski merged T323312: make Phabricator work with readonly DB into T313879: Phabricator: Unable to view tasks in DB read-only mode.
Jan 17 2023, 4:41 PM · serviceops-collab, Release-Engineering-Team (Radar), serviceops-radar, Sustainability (Incident Followup), Phabricator
LSobanski merged task T323312: make Phabricator work with readonly DB into T313879: Phabricator: Unable to view tasks in DB read-only mode.
Jan 17 2023, 4:40 PM · Phabricator, serviceops-collab