Page MenuHomePhabricator

LSobanski (Lukasz Sobanski)
Woo$

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Tuesday

  • Clear sailing ahead.

User Details

User Since
Aug 31 2020, 5:40 PM (20 w, 5 d)
Availability
Available
LDAP User
LSobanski
MediaWiki User
LSobanski (WMF) [ Global Accounts ]

Recent Activity

Fri, Jan 22

LSobanski triaged T272686: print a list of backed up directories in the MOTD of production servers as Medium priority.

Sounds like a good idea. Is this to address a specific concern that came up? One thing that comes to mind is the amount, readability and relevance of the information, e.g. maybe it would make sense to only limit this to directories the account has write access to (tricky for root)? Some UX testing would be nice :)

Fri, Jan 22, 11:36 AM · Data-Persistence-Backup, SRE

Thu, Jan 21

LSobanski triaged T272614: m2 codfw master crashed as High priority.
Thu, Jan 21, 4:52 PM · DBA

Wed, Jan 20

LSobanski triaged T272533: Grant "sockpuppet_import" user INDEX on "sockpuppet" database as Medium priority.
Wed, Jan 20, 7:44 PM · DBA

Tue, Jan 19

LSobanski moved T271261: Prepare and check storage layer for trwikivoyage from Triage to Ready on the DBA board.
Tue, Jan 19, 1:54 PM · cloud-services-team (Kanban), Data-Services, DBA
LSobanski moved T266106: orchestrator: Support SSO from Ready to In progress on the DBA board.
Tue, Jan 19, 12:06 PM · Patch-For-Review, Orchestrator, CAS-SSO, SRE, User-Kormat, DBA
LSobanski moved T272347: orchestrator: Monitor for non-fqdns in the host resolve cache from Triage to Refine on the DBA board.
Tue, Jan 19, 12:05 PM · Orchestrator, DBA
LSobanski moved T272344: Matomo database backup size doubled, we should check this is normal operation from Triage to Refine on the Data-Persistence-Backup board.
Tue, Jan 19, 10:30 AM · Data-Persistence-Backup, Analytics

Mon, Jan 18

LSobanski moved T167973: Move database for wikitech (labswiki) to a main cluster section from Ready to Refine on the DBA board.
Mon, Jan 18, 5:13 PM · wikitech.wikimedia.org, DBA
LSobanski added a comment to T267365: Develop maintenance script for enumerating Swift media files from MediaWiki (for backup processing).

To clarify, are we talking about urls of the form upload.wikimedia.org/something (or lang.project.org/something.php?somethingelse... or swift urls?

Mon, Jan 18, 5:09 PM · User-ArielGlenn, Platform Team Workboards (Clinic Duty Team), Structured-Data-Backlog, Structured Data Engineering, Commons, MediaWiki-File-management

Thu, Jan 14

LSobanski triaged T271982: Prepare and check storage layer for altwiki as Medium priority.

Thanks, let us know when the database is created, so we can sanitize it.

Thu, Jan 14, 10:52 AM · Data-Services, DBA

Wed, Jan 13

LSobanski added a project to T271268: Seek DBA approval for new logging table in SecurePoll: Data-Persistence (Consulted).
Wed, Jan 13, 6:43 PM · Data-Persistence (Consulted), Patch-For-Review, Anti-Harassment (The Letter Song), MediaWiki-extensions-SecurePoll
LSobanski updated subscribers of T267404: Add reply links to the parser cache.

@Esanders could you help me understand what impact this is expected to have on the parser cache databases?

Wed, Jan 13, 6:40 PM · Verified, MW-1.36-notes (1.36.0-wmf.28; 2021-01-26), Editing-team (FY2020-21 Kanban Board), Technical-Debt, Performance Issue, DiscussionTools
LSobanski moved T271821: export_smart_data_dump.service failed on dbprov2001 because of a timeout in the raid facter from Triage to Ready on the Data-Persistence-Backup board.
Wed, Jan 13, 5:19 PM · Data-Persistence-Backup

Mon, Jan 11

LSobanski added a comment to T267043: (Need By: 2020-11-29) rack/setup/install db11[51-76].

Thanks!

Mon, Jan 11, 1:31 PM · DBA, ops-eqiad, DC-Ops, SRE
LSobanski added a comment to T267043: (Need By: 2020-11-29) rack/setup/install db11[51-76].

@Jclark-ctr @Cmjohnson, what's a realistic ETA for completing the work on these servers? It would help us plan the next steps for this quarter.

Mon, Jan 11, 1:18 PM · DBA, ops-eqiad, DC-Ops, SRE

Wed, Jan 6

LSobanski added a comment to T268505: New database request: sockpuppet.

@gmodena Manuel will be around tomorrow, we can sync up on exact timing then. Will that work?

Wed, Jan 6, 5:31 PM · DBA
LSobanski removed a project from T249683: Redefine mysql GRANTs for wikiadmin: SRE.
Wed, Jan 6, 4:56 PM · Sustainability (Incident Followup), DBA
LSobanski added a comment to T268505: New database request: sockpuppet.

@gmodena Manuel is not around today, could we postpone this?

Wed, Jan 6, 3:02 PM · DBA
LSobanski triaged T271261: Prepare and check storage layer for trwikivoyage as Medium priority.

Thanks, let us know when the database is created, so we can sanitize it.

Wed, Jan 6, 1:20 PM · cloud-services-team (Kanban), Data-Services, DBA

Mon, Jan 4

LSobanski renamed T271140: Some Data Persistence DB clusters apparently do not support IPv6 from Some Data Persistence clusters apparently do not support IPv6 to Some Data Persistence DB clusters apparently do not support IPv6.
Mon, Jan 4, 7:48 PM · IPv6, DBA, SRE-tools
LSobanski triaged T271148: Some Data Persistence Backup clusters apparently do not support IPv6 as Medium priority.
Mon, Jan 4, 7:47 PM · Data-Persistence-Backup, SRE-tools
LSobanski triaged T271140: Some Data Persistence DB clusters apparently do not support IPv6 as Medium priority.
Mon, Jan 4, 7:47 PM · IPv6, DBA, SRE-tools
LSobanski moved T271148: Some Data Persistence Backup clusters apparently do not support IPv6 from Triage to Refine on the Data-Persistence-Backup board.
Mon, Jan 4, 7:47 PM · Data-Persistence-Backup, SRE-tools
LSobanski updated the task description for T271140: Some Data Persistence DB clusters apparently do not support IPv6.
Mon, Jan 4, 7:44 PM · IPv6, DBA, SRE-tools
LSobanski updated the task description for T271148: Some Data Persistence Backup clusters apparently do not support IPv6.
Mon, Jan 4, 7:43 PM · Data-Persistence-Backup, SRE-tools
LSobanski added a subtask for T253173: Some clusters do not have DNS for IPv6 addresses (TRACKING TASK): T271148: Some Data Persistence Backup clusters apparently do not support IPv6.
Mon, Jan 4, 7:43 PM · IPv6, User-jbond, netbox
LSobanski added a parent task for T271148: Some Data Persistence Backup clusters apparently do not support IPv6: T253173: Some clusters do not have DNS for IPv6 addresses (TRACKING TASK).
Mon, Jan 4, 7:43 PM · Data-Persistence-Backup, SRE-tools
LSobanski added a comment to T271140: Some Data Persistence DB clusters apparently do not support IPv6.

I moved the backup hosts to https://phabricator.wikimedia.org/T271148.

Mon, Jan 4, 7:43 PM · IPv6, DBA, SRE-tools
LSobanski created T271148: Some Data Persistence Backup clusters apparently do not support IPv6.
Mon, Jan 4, 7:42 PM · Data-Persistence-Backup, SRE-tools
LSobanski updated the task description for T271140: Some Data Persistence DB clusters apparently do not support IPv6.
Mon, Jan 4, 7:34 PM · IPv6, DBA, SRE-tools
LSobanski updated subscribers of T271140: Some Data Persistence DB clusters apparently do not support IPv6.
Mon, Jan 4, 7:32 PM · IPv6, DBA, SRE-tools
LSobanski updated subscribers of T271140: Some Data Persistence DB clusters apparently do not support IPv6.
Mon, Jan 4, 7:06 PM · IPv6, DBA, SRE-tools
LSobanski moved T271140: Some Data Persistence DB clusters apparently do not support IPv6 from Triage to Refine on the DBA board.
Mon, Jan 4, 7:05 PM · IPv6, DBA, SRE-tools
LSobanski edited projects for T271140: Some Data Persistence DB clusters apparently do not support IPv6, added: DBA; removed Data-Persistence.
Mon, Jan 4, 7:05 PM · IPv6, DBA, SRE-tools
LSobanski updated subscribers of T271140: Some Data Persistence DB clusters apparently do not support IPv6.
  • heze.codfw.wmnet will be deprecated (planned in Q3).
  • ms-be and ms-fe are media storage and not managed by us (yet). @fgiunchedi, could you comment on these?
Mon, Jan 4, 7:02 PM · IPv6, DBA, SRE-tools
LSobanski moved T264162: IRC bot adjustments for #wikimedia-databases from Done to In Progress on the Data-Persistence-Admin board.
Mon, Jan 4, 5:14 PM · Wikibugs, Data-Persistence-Admin
LSobanski updated the task description for T264162: IRC bot adjustments for #wikimedia-databases.
Mon, Jan 4, 5:14 PM · Wikibugs, Data-Persistence-Admin

Dec 21 2020

LSobanski triaged T270410: Prepare and check storage layer for niawiktionary as Medium priority.

Thanks, let us know when the database is created, so we can sanitize it.

Dec 21 2020, 1:32 PM · cloud-services-team (Kanban), Data-Services, DBA
LSobanski triaged T270414: Prepare and check storage layer for niawiki as Medium priority.

Thanks, let us know when the database is created, so we can sanitize it.

Dec 21 2020, 1:31 PM · cloud-services-team (Kanban), Data-Services, DBA
LSobanski triaged T270621: Schema change for renaming several indexes in sites table as Medium priority.
Dec 21 2020, 12:52 PM · DBA, Blocked-on-schema-change
LSobanski triaged T270620: Schema change for renaming several indexes in logging table as Medium priority.
Dec 21 2020, 12:38 PM · DBA, Blocked-on-schema-change
LSobanski triaged T268258: transfer.py argument parsing exception as Medium priority.
Dec 21 2020, 11:47 AM · SRE-tools, Data-Persistence-Backup

Dec 17 2020

LSobanski added a comment to T267365: Develop maintenance script for enumerating Swift media files from MediaWiki (for backup processing).

@ArielGlenn will look into whether URLs can be extracted for deleted images and what the alternatives are if not.

Dec 17 2020, 4:22 PM · User-ArielGlenn, Platform Team Workboards (Clinic Duty Team), Structured-Data-Backlog, Structured Data Engineering, Commons, MediaWiki-File-management
LSobanski moved T270112: mariadb on dbstore hosts, and specifically dbstore1004, possible memory leaking from Triage to Refine on the DBA board.
Dec 17 2020, 12:56 PM · Analytics-Radar, DBA
LSobanski triaged T270276: Prepare and check storage layer for diqwiktionary as Medium priority.

Thanks, let us know when the database is created, so we can sanitize it.

Dec 17 2020, 12:55 PM · cloud-services-team (Kanban), Data-Services, DBA
LSobanski triaged T270280: Prepare and check storage layer for bclwiktionary as Medium priority.

Thanks, let us know when the database is created, so we can sanitize it.

Dec 17 2020, 12:55 PM · cloud-services-team (Kanban), Data-Services, DBA

Dec 14 2020

LSobanski moved T270053: Increase size of content_models.model_id from Triage to Ready on the DBA board.
Dec 14 2020, 10:23 AM · DBA, Blocked-on-schema-change
LSobanski moved T270054: Increase size of slot_roles.role_id from Triage to Ready on the DBA board.
Dec 14 2020, 10:23 AM · DBA, Blocked-on-schema-change
LSobanski moved T270055: Schema change for timestamp field of uploadstash from Triage to Ready on the DBA board.
Dec 14 2020, 10:23 AM · Blocked-on-schema-change, DBA
LSobanski triaged T270053: Increase size of content_models.model_id as Medium priority.
Dec 14 2020, 10:23 AM · DBA, Blocked-on-schema-change
LSobanski triaged T270054: Increase size of slot_roles.role_id as Medium priority.
Dec 14 2020, 10:23 AM · DBA, Blocked-on-schema-change
LSobanski triaged T270055: Schema change for timestamp field of uploadstash as Medium priority.
Dec 14 2020, 10:23 AM · Blocked-on-schema-change, DBA

Dec 7 2020

LSobanski added a comment to T262388: transfer.py fails when copying data between es hosts.

As discussed last week, this task needs to be estimated and scheduled. It's not currently considered to be a workflow blocker.

Dec 7 2020, 11:36 AM · DBA
LSobanski moved T269427: Prepare and check storage layer for eowikivoyage from Triage to Blocked on the DBA board.
Dec 7 2020, 11:32 AM · cloud-services-team (Kanban), Data-Services, DBA
LSobanski moved T269432: Prepare and check storage layer for wawikisource from Triage to Blocked on the DBA board.
Dec 7 2020, 11:32 AM · cloud-services-team (Kanban), Data-Services, DBA
LSobanski moved T269440: Prepare and check storage layer for madwiki from Triage to Blocked on the DBA board.
Dec 7 2020, 11:32 AM · cloud-services-team (Kanban), Data-Services, DBA
LSobanski triaged T269427: Prepare and check storage layer for eowikivoyage as Medium priority.
Dec 7 2020, 11:32 AM · cloud-services-team (Kanban), Data-Services, DBA
LSobanski triaged T269432: Prepare and check storage layer for wawikisource as Medium priority.
Dec 7 2020, 11:32 AM · cloud-services-team (Kanban), Data-Services, DBA
LSobanski triaged T269440: Prepare and check storage layer for madwiki as Medium priority.
Dec 7 2020, 11:32 AM · cloud-services-team (Kanban), Data-Services, DBA
LSobanski moved T264162: IRC bot adjustments for #wikimedia-databases from Refine to Done on the Data-Persistence-Admin board.
Dec 7 2020, 11:29 AM · Wikibugs, Data-Persistence-Admin
LSobanski reopened T253276: Normalise MW Core database language fields length as "Open".

Reopening based on @Ladsgroup's comment.

Dec 7 2020, 9:57 AM · Patch-For-Review, MW-1.35-notes (1.35.0-wmf.34; 2020-05-26), DBA, Schema-change, Technical-Debt, MediaWiki-General
LSobanski reopened T253276: Normalise MW Core database language fields length, a subtask of T243318: FY2020-2021 Q1 codfw -> eqiad switchback, as Open.
Dec 7 2020, 9:57 AM · SRE
LSobanski added a comment to T269440: Prepare and check storage layer for madwiki.

Thanks, let us know when the database is created, so we can sanitize it.

Dec 7 2020, 9:06 AM · cloud-services-team (Kanban), Data-Services, DBA
LSobanski added a comment to T269432: Prepare and check storage layer for wawikisource.

Thanks, let us know when the database is created, so we can sanitize it.

Dec 7 2020, 9:03 AM · cloud-services-team (Kanban), Data-Services, DBA
LSobanski added a comment to T269427: Prepare and check storage layer for eowikivoyage.

Thanks, let us know when the database is created, so we can sanitize it.

Dec 7 2020, 9:02 AM · cloud-services-team (Kanban), Data-Services, DBA

Dec 1 2020

LSobanski claimed T264162: IRC bot adjustments for #wikimedia-databases.
Dec 1 2020, 4:01 PM · Wikibugs, Data-Persistence-Admin

Nov 27 2020

LSobanski added a comment to T119173: RFC: Discourage use of MySQL's ENUM type.

Is the documentation merging / overhaul tracked anywhere (in a task, perhaps)? Seems like a good opportunity to ask :)

Nov 27 2020, 1:35 PM · Documentation, Data-Persistence (Consulted), Performance-Team (Radar), TechCom-RFC, MediaWiki-General

Nov 26 2020

LSobanski added a comment to T261405: db1139 memory errors on boot (issue continues after board change) 2020-08-27.

Thanks!

Nov 26 2020, 12:24 PM · SRE, DBA, ops-eqiad
LSobanski added a comment to T267043: (Need By: 2020-11-29) rack/setup/install db11[51-76].

@Cmjohnson Would it be possible to plan for racking 5 instead of 3 of the new hosts in one go? It would help us prepare fot Sanitarium host Buster/10.4 upgrades, which are currently at risk due to issues tracked in https://phabricator.wikimedia.org/T268742. Any 1G rack locations are fine for the 2 additional hosts.

Nov 26 2020, 12:24 PM · DBA, ops-eqiad, DC-Ops, SRE
LSobanski added a comment to T261405: db1139 memory errors on boot (issue continues after board change) 2020-08-27.

@Jclark-ctr based on the DC entry schedule, when do you expect you will be able to take a look at this? Knowing this would allow us to better plan next steps regarding backup source locations.

Nov 26 2020, 12:17 PM · SRE, DBA, ops-eqiad

Nov 25 2020

LSobanski edited projects for T263587: CAPEX for ParserCache for Parsoid, added: Data-Persistence (Consulted); removed DBA.

I believe the key questions to DBA have been answered so I'm removing the team tag. Please add us back if there is anything else we can help with.

Nov 25 2020, 10:33 AM · Parsoid (Tracking), Data-Persistence (Consulted), serviceops, MediaWiki-Parser

Nov 24 2020

LSobanski moved T268316: Base replication lag detection on heartbeat from Triage to Refine on the DBA board.
Nov 24 2020, 12:48 PM · Orchestrator, DBA, SRE
LSobanski moved T268320: Configure mariadb to notice/recover from replication issues quicker from Triage to Refine on the DBA board.
Nov 24 2020, 12:48 PM · Orchestrator, DBA
LSobanski moved T268336: Cleanup heartbeat.heartbeat on all production instances from Triage to Refine on the DBA board.
Nov 24 2020, 12:48 PM · Orchestrator, DBA
LSobanski moved T268458: Prepare and check storage layer for skrwiktionary from Triage to Blocked on the DBA board.
Nov 24 2020, 12:46 PM · cloud-services-team (Kanban), Data-Services, DBA
LSobanski triaged T268458: Prepare and check storage layer for skrwiktionary as Medium priority.

Thanks, let us know when the database is created, so we can sanitize it.

Nov 24 2020, 12:46 PM · cloud-services-team (Kanban), Data-Services, DBA

Nov 23 2020

LSobanski removed a project from T41013: Incomplete i18n for log entries in CheckUser: DBA.

Removing DBA, please add us back if there are any additional questions we can answer in this task. Alternatively, create a new task to monitor the table size once the change is deployed.

Nov 23 2020, 12:11 PM · Anti-Harassment, Schema-change, Patch-For-Review, Stewards-and-global-tools, I18n, CheckUser
LSobanski triaged T268412: Prepare and check storage layer for skrwiki as Medium priority.
Nov 23 2020, 11:25 AM · cloud-services-team (Kanban), Data-Services, DBA
LSobanski added a comment to T268412: Prepare and check storage layer for skrwiki.

https://phabricator.wikimedia.org/T265321 was resolved last week so the ipblocks_restrictions alter should no longer be necessary.

Nov 23 2020, 11:09 AM · cloud-services-team (Kanban), Data-Services, DBA
LSobanski moved T268412: Prepare and check storage layer for skrwiki from Triage to Blocked on the DBA board.
Nov 23 2020, 11:04 AM · cloud-services-team (Kanban), Data-Services, DBA
LSobanski added a comment to T268412: Prepare and check storage layer for skrwiki.

Thanks, let us know when the database is created, so we can sanitize it.

Nov 23 2020, 11:04 AM · cloud-services-team (Kanban), Data-Services, DBA

Nov 20 2020

LSobanski set Due Date to Wed, Dec 30, 11:00 PM on T268329: Request new database for pki.discovery.wmnet.
Nov 20 2020, 12:43 PM · User-jbond, Puppet, DBA, SRE
LSobanski set Due Date to Wed, Dec 30, 11:00 PM on T268327: Request new database for idp.wikimedia.org.
Nov 20 2020, 12:43 PM · User-jbond, DBA, CAS-SSO, SRE
LSobanski removed a project from T267818: Decide whether QBA-bot's proxy blocker is allowed: DBA.

I don't believe there is an explicit action for DBA here so I'm untagging us from this task. Please add us back if there are specific questions we can help with.

Nov 20 2020, 12:36 PM · Anti-Harassment
LSobanski moved T268327: Request new database for idp.wikimedia.org from Triage to Refine on the DBA board.
Nov 20 2020, 12:31 PM · User-jbond, DBA, CAS-SSO, SRE
LSobanski added a comment to T268327: Request new database for idp.wikimedia.org.

@jbond What is your preferred delivery date for this?

Nov 20 2020, 12:31 PM · User-jbond, DBA, CAS-SSO, SRE
LSobanski moved T268329: Request new database for pki.discovery.wmnet from Triage to Refine on the DBA board.
Nov 20 2020, 12:31 PM · User-jbond, Puppet, DBA, SRE
LSobanski added a comment to T268329: Request new database for pki.discovery.wmnet.

@jbond What is your preferred delivery date for this?

Nov 20 2020, 12:31 PM · User-jbond, Puppet, DBA, SRE
LSobanski added a comment to T268312: Deploy labsdbuser and views to new clouddb hosts.

Cheeky question, should we go with a "clouddb" user and role to be consistent or would that be too much work to untangle from the current set up?

Nov 20 2020, 9:36 AM · Patch-For-Review, DBA, Data-Services, cloud-services-team (Kanban)

Nov 16 2020

LSobanski added a comment to T247028: Database 'INSERT' query rate doubled (module_deps regression?).

Is it clear what the underlying problem was and how it was fixed? Just making sure it's not phantom recovery.

Nov 16 2020, 10:33 AM · MW-1.35-notes (1.35.0-wmf.34; 2020-05-26), Sustainability (Incident Followup), Performance Issue, MediaWiki-ResourceLoader, Performance-Team

Nov 4 2020

LSobanski added a comment to T267214: Add a link engineering: Database for link recommendation service.

@kostajh what is your preferred time horizon for this DB to be available?

Nov 4 2020, 2:11 PM · DBA

Nov 3 2020

LSobanski added a comment to T267088: decommission db1091.eqiad.wmnet.

This host is ready for DC-Ops to decommission.

Nov 3 2020, 2:40 PM · DC-Ops, SRE, ops-eqiad, decommission-hardware
LSobanski reassigned T267088: decommission db1091.eqiad.wmnet from Marostegui to wiki_willy.
Nov 3 2020, 2:40 PM · DC-Ops, SRE, ops-eqiad, decommission-hardware

Oct 28 2020

LSobanski removed a project from T260112: Remove muswiki and mhwiktionary from s3: Data-Persistence.
Oct 28 2020, 2:06 PM · DBA
LSobanski removed a project from T262388: transfer.py fails when copying data between es hosts: Data-Persistence.
Oct 28 2020, 2:05 PM · DBA
LSobanski removed a project from T265266: Create integration test env for wmfmariadbpy: Data-Persistence.
Oct 28 2020, 2:05 PM · Release-Engineering-Team, Continuous-Integration-Config, User-Kormat, DBA, SRE
LSobanski removed a project from T265901: Clean up role::mariadb::ferm and profile::mariadb::ferm: Data-Persistence.
Oct 28 2020, 2:04 PM · DBA, SRE, User-Kormat
LSobanski removed a project from T266003: orchestrator: Select backend database solution: Data-Persistence.
Oct 28 2020, 2:04 PM · Orchestrator, User-Kormat, DBA
LSobanski created Orchestrator.
Oct 28 2020, 9:16 AM

Oct 26 2020

LSobanski removed a project from T259831: Schema change to make change_tag.ct_rc_id unsigned: Data-Persistence.
Oct 26 2020, 4:04 PM · DBA, SRE, User-Kormat, Blocked-on-schema-change