Page MenuHomePhabricator

Banyek (Balazs Pocze)
Disabled

Projects

User does not belong to any projects.

User Details

User Since
Aug 27 2018, 1:02 PM (290 w, 20 h)
Roles
Disabled
LDAP User
Banyek
MediaWiki User
Unknown
This account has been disabled.

Recent Activity

Dec 3 2019

jcrespo awarded T208323: Predictive failures on disk S.M.A.R.T. status a Like token.
Dec 3 2019, 8:54 AM · SRE, DBA

Jan 10 2019

Banyek added a comment to T85757: Dropping user.user_options on wmf databases.

Thank you <3

Jan 10 2019, 4:17 PM · User-Banyek, Schema-change-in-production, DBA, Schema-change
Banyek added a comment to T85757: Dropping user.user_options on wmf databases.

on db1062 (s7 master) every database is done, except eswiki I have to retry this later. (Lock wait timeout exceeded)

Jan 10 2019, 10:42 AM · User-Banyek, Schema-change-in-production, DBA, Schema-change
Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Jan 10 2019, 10:42 AM · User-Banyek, Schema-change-in-production, DBA, Schema-change

Jan 9 2019

Banyek created P7968 (An Untitled Masterwork).
Jan 9 2019, 3:26 PM
Banyek removed a project from T107610: Setup separate logical External Store for Flow in production: User-Banyek.
Jan 9 2019, 11:29 AM · Growth-Team, DBA, SRE, Essential-Work, StructuredDiscussions
Banyek updated subscribers of T211939: Drop several views from srwikinews.

@Andrew I cleaned up the tables (T210693) which blocked using the maintain-views script

Jan 9 2019, 9:55 AM · Serbian-Sites, User-bd808, Patch-For-Review, User-Banyek, cloud-services-team (Kanban), Data-Services, User-Kizule
Banyek added a comment to T212333: Drop several views from ptwiki.

As the tables which blocked using the view management tool were cleaned up (T210693) , I think we could to the view updates normally.

Jan 9 2019, 9:54 AM · User-Banyek, cloud-services-team (Kanban), Data-Services, User-Kizule
Banyek closed T210693: Create materialized views on Wiki Replica hosts for better query performance as Resolved.

I cleaned up the tables, so I close the ticket

Jan 9 2019, 9:52 AM · Patch-For-Review, User-Banyek, Platform Team Legacy (Watching / External), Analytics-Kanban, DBA, Data-Services, Analytics
Banyek closed T210693: Create materialized views on Wiki Replica hosts for better query performance, a subtask of T209031: Not able to scoop comment table in labs for mediawiki reconstruction process [EPIC}, as Resolved.
Jan 9 2019, 9:52 AM · Platform Team Legacy (Watching / External), Analytics-Kanban, DBA, Data-Services, Analytics

Jan 8 2019

Banyek added a comment to T212333: Drop several views from ptwiki.

It seems I can clean up the materialized views tables which prevent the tool working correctly (see T210693) , so I guess this ticket will be solved right after. Thanks for patience

Jan 8 2019, 8:16 PM · User-Banyek, cloud-services-team (Kanban), Data-Services, User-Kizule
Banyek added a comment to T210693: Create materialized views on Wiki Replica hosts for better query performance.

Ok, then we are all agree, I'll drop those tables tomorrow in the morning

Jan 8 2019, 8:13 PM · Patch-For-Review, User-Banyek, Platform Team Legacy (Watching / External), Analytics-Kanban, DBA, Data-Services, Analytics
Banyek added a comment to T212333: Drop several views from ptwiki.
In T212333#4856663, @Zoranzoki21 wrote:

@Banyek You can`t just login there and drop these views via DROP VIEW IF EXISTS?

MariaDB [ptwiki_p]> SHOW TABLES like 'flagged%';
+-------------------------------+
| Tables_in_ptwiki_p (flagged%) |
+-------------------------------+
| flaggedimages                 |
| flaggedpage_config            |
| flaggedpage_pending           |
| flaggedpages                  |
| flaggedrevs                   |
| flaggedrevs_promote           |
| flaggedrevs_statistics        |
| flaggedrevs_tracking          |
| flaggedtemplates              |
+-------------------------------+
9 rows in set (0.00 sec)

MariaDB [ptwiki_p]>
Jan 8 2019, 7:54 PM · User-Banyek, cloud-services-team (Kanban), Data-Services, User-Kizule
Banyek added a comment to T210693: Create materialized views on Wiki Replica hosts for better query performance.

@Bstorm so, what do you think, should I drop these?

Jan 8 2019, 1:39 PM · Patch-For-Review, User-Banyek, Platform Team Legacy (Watching / External), Analytics-Kanban, DBA, Data-Services, Analytics
Banyek added a comment to T212333: Drop several views from ptwiki.

No, I didn't

Jan 8 2019, 1:34 PM · User-Banyek, cloud-services-team (Kanban), Data-Services, User-Kizule
Banyek added a comment to T85757: Dropping user.user_options on wmf databases.

on db1062 (s7 master) every database is done, except eswiki I have to retry this later. (Lock wait timeout exceeded)

Jan 8 2019, 11:25 AM · User-Banyek, Schema-change-in-production, DBA, Schema-change

Jan 7 2019

Banyek moved T210478: Migrate dbstore1002 to a multi instance setup on dbstore100[3-5] from In progress to FYI on the User-Banyek board.
Jan 7 2019, 5:20 PM · Analytics-Radar, Patch-For-Review, User-Banyek, Analytics-Kanban, DBA
Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Jan 7 2019, 10:40 AM · User-Banyek, Schema-change-in-production, DBA, Schema-change
Banyek added a comment to T212833: es2019 is not responsive.

The comparison finished, and the data is OK.

Jan 7 2019, 9:32 AM · ops-codfw, SRE, Patch-For-Review, DBA

Jan 4 2019

Banyek added a comment to T212833: es2019 is not responsive.

On Cumin2001 I have a comparison screen running inside of a screen in /home/banyek
The script is used the following:

Jan 4 2019, 3:45 PM · ops-codfw, SRE, Patch-For-Review, DBA
Banyek merged T212910: db1107 has CRITICAL status in power supply into T212861: Rack A2's hosts alarm for PSU broken.
Jan 4 2019, 3:21 PM · Analytics-Radar, ops-eqiad, SRE
Banyek merged task T212910: db1107 has CRITICAL status in power supply into T212861: Rack A2's hosts alarm for PSU broken.
Jan 4 2019, 3:21 PM · Analytics, SRE, MediaWiki-extensions-EventLogging, ops-eqiad
Banyek merged T212909: db1082 has CRITICAL status in power supply into T212861: Rack A2's hosts alarm for PSU broken.
Jan 4 2019, 3:20 PM · Analytics-Radar, ops-eqiad, SRE
Banyek merged task T212909: db1082 has CRITICAL status in power supply into T212861: Rack A2's hosts alarm for PSU broken.
Jan 4 2019, 3:20 PM · ops-eqiad, SRE
Banyek added a comment to T210693: Create materialized views on Wiki Replica hosts for better query performance.

On labsdb1010 this would be the quickest (with depooled host)

#!/bin/bash
Jan 4 2019, 2:12 PM · Patch-For-Review, User-Banyek, Platform Team Legacy (Watching / External), Analytics-Kanban, DBA, Data-Services, Analytics
Banyek added a comment to T210693: Create materialized views on Wiki Replica hosts for better query performance.

+1 on removing these tables as mentioned in T210693 too.

Jan 4 2019, 10:38 AM · Patch-For-Review, User-Banyek, Platform Team Legacy (Watching / External), Analytics-Kanban, DBA, Data-Services, Analytics
Banyek added a comment to T212333: Drop several views from ptwiki.

@Bstorm Yeah, that is the case, I just wanted to ping you about it in the ticket :)
So, I think I'll drop the materialized views (as their data is already outdated anyway) instead of modifying the script.
Do you agree?

Jan 4 2019, 10:37 AM · User-Banyek, cloud-services-team (Kanban), Data-Services, User-Kizule
Banyek closed T212908: dbstore1002 lagged on creating database as Resolved.

hm, then I close the ticket, thanks @Marostegui

Jan 4 2019, 10:27 AM · DBA
Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Jan 4 2019, 10:19 AM · User-Banyek, Schema-change-in-production, DBA, Schema-change
Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Jan 4 2019, 9:40 AM · User-Banyek, Schema-change-in-production, DBA, Schema-change
elukey awarded T212908: dbstore1002 lagged on creating database a Like token.
Jan 4 2019, 6:50 AM · DBA

Jan 3 2019

Banyek created T212910: db1107 has CRITICAL status in power supply.
Jan 3 2019, 10:52 PM · Analytics, SRE, MediaWiki-extensions-EventLogging, ops-eqiad
Banyek created T212909: db1082 has CRITICAL status in power supply.
Jan 3 2019, 10:50 PM · ops-eqiad, SRE
Banyek moved T212908: dbstore1002 lagged on creating database from Triage to Backlog on the DBA board.
Jan 3 2019, 10:45 PM · DBA
Banyek triaged T212908: dbstore1002 lagged on creating database as Medium priority.
Jan 3 2019, 10:44 PM · DBA
Banyek created T212908: dbstore1002 lagged on creating database.
Jan 3 2019, 10:44 PM · DBA
Banyek moved T212833: es2019 is not responsive from Triage to In progress on the DBA board.
Jan 3 2019, 10:36 PM · ops-codfw, SRE, Patch-For-Review, DBA
Banyek added a comment to T212833: es2019 is not responsive.

I'll start with this in the morning

Jan 3 2019, 9:51 PM · ops-codfw, SRE, Patch-For-Review, DBA
Banyek added a project to T211939: Drop several views from srwikinews: User-Banyek.

Probably the problem is caused as the comment_mat is not a view but a table, I'll check the maintain-views script to understand how it works

Jan 3 2019, 4:06 PM · Serbian-Sites, User-bd808, Patch-For-Review, User-Banyek, cloud-services-team (Kanban), Data-Services, User-Kizule
Banyek added a project to T212333: Drop several views from ptwiki: User-Banyek.
Jan 3 2019, 4:00 PM · User-Banyek, cloud-services-team (Kanban), Data-Services, User-Kizule
Banyek added a comment to T212333: Drop several views from ptwiki.

Ok, I guess I understand the problem now, I'll check the scripts

Jan 3 2019, 3:53 PM · User-Banyek, cloud-services-team (Kanban), Data-Services, User-Kizule
Banyek added a comment to T212333: Drop several views from ptwiki.

@Andrew as far as I know we manually have to drop the views as dropping the underlying tables doesn't clean them up. What shall I investigate exaclty?

Jan 3 2019, 3:47 PM · User-Banyek, cloud-services-team (Kanban), Data-Services, User-Kizule
Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Jan 3 2019, 2:35 PM · User-Banyek, Schema-change-in-production, DBA, Schema-change
Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Jan 3 2019, 1:39 PM · User-Banyek, Schema-change-in-production, DBA, Schema-change
Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Jan 3 2019, 12:54 PM · User-Banyek, Schema-change-in-production, DBA, Schema-change
Banyek closed T212833: es2019 is not responsive as Resolved.
Jan 3 2019, 9:27 AM · ops-codfw, SRE, Patch-For-Review, DBA
Banyek updated the task description for T212833: es2019 is not responsive.
Jan 3 2019, 9:27 AM · ops-codfw, SRE, Patch-For-Review, DBA
Banyek added a comment to T212833: es2019 is not responsive.

The mariadb started without any problem, and replication is resumed

Jan 3 2019, 9:14 AM · ops-codfw, SRE, Patch-For-Review, DBA
Banyek added a comment to T212833: es2019 is not responsive.

after hard reset, I didn't find anything in the logs
/var/log/syslog

Jan  3 07:35:01 es2019 CRON[16225]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Jan  3 07:35:01 es2019 CRON[16226]: (prometheus) CMD (/usr/local/bin/prometheus-puppet-agent-stats --outfile /var/lib/prometheus/node.d/puppet_agent.prom)
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@Jan  3 09:07:53 es2019 systemd-modules-load[1062]: Inserted module 'nf_conntrack'
Jan  3 09:07:53 es2019 systemd-modules-load[1062]: Inserted module 'ipmi_devintf'
Jan  3 09:07:53 es2019 systemd-sysctl[1081]: Couldn't write '65' to 'net/netfilter/nf_conntrack_tcp_timeout_time_wait', ignoring: No such file or directory
Jan 3 2019, 9:14 AM · ops-codfw, SRE, Patch-For-Review, DBA
Banyek added a comment to T212833: es2019 is not responsive.

according to https://wikitech.wikimedia.org/wiki/Platform-specific_documentation/Dell_PowerEdge_RN30 I reset the host with
racadm serveraction hardreset, now the console is available

Jan 3 2019, 9:07 AM · ops-codfw, SRE, Patch-For-Review, DBA
Banyek lowered the priority of T212833: es2019 is not responsive from Unbreak Now! to High.

I triage this as 'high' not unbreak, because the host wasn't in service

Jan 3 2019, 8:44 AM · ops-codfw, SRE, Patch-For-Review, DBA
Banyek updated the task description for T212833: es2019 is not responsive.
Jan 3 2019, 8:44 AM · ops-codfw, SRE, Patch-For-Review, DBA
Banyek triaged T212833: es2019 is not responsive as Unbreak Now! priority.
Jan 3 2019, 8:23 AM · ops-codfw, SRE, Patch-For-Review, DBA
Banyek created T212833: es2019 is not responsive.
Jan 3 2019, 8:23 AM · ops-codfw, SRE, Patch-For-Review, DBA

Jan 2 2019

Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Jan 2 2019, 4:28 PM · User-Banyek, Schema-change-in-production, DBA, Schema-change
Banyek added a comment to T212154: Recommendation API exceeds max_user_connections in MySQL.

That makes sense, then I'll add a +1 and and LGTM to the patchm, but I won't mind, if you'd ask for a second opinion from the more experienced wiki dba's, because there might be something I am not aware of

Jan 2 2019, 4:25 PM · Patch-For-Review, Research, Platform Team Legacy (Watching / External), DBA, Recommendation-API, Services (watching)
Banyek added a comment to T212154: Recommendation API exceeds max_user_connections in MySQL.

@Banyek thanks! For the context I requested the initial number to be 50 per T205294: Request to create database and account for recommendation API, which was an arbitrary number. Didn't realize it was created as 10.

Jan 2 2019, 4:21 PM · Patch-For-Review, Research, Platform Team Legacy (Watching / External), DBA, Recommendation-API, Services (watching)
Banyek updated subscribers of T212154: Recommendation API exceeds max_user_connections in MySQL.

@bmansurov as I see the memory usage on the m2 hosts, this could be good, but I am not sure if it is a good idea to increase it to a 40x size of the original.
I mean, as I've see the numbers this shouldn't cause problems, but I'd like to ask @Marostegui or @jcrespo about why the original number was set to 10?

Jan 2 2019, 4:06 PM · Patch-For-Review, Research, Platform Team Legacy (Watching / External), DBA, Recommendation-API, Services (watching)
Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Jan 2 2019, 3:14 PM · User-Banyek, Schema-change-in-production, DBA, Schema-change
Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Jan 2 2019, 2:19 PM · User-Banyek, Schema-change-in-production, DBA, Schema-change
Banyek added a comment to T212574: Recheck wikidatawiki.pagelinks table across all hosts in s8.

awesome!

Jan 2 2019, 8:36 AM · DBA

Dec 26 2018

Banyek added a comment to T209521: replication broken on db1124:3318 on wikidata.pagelinks.

Maybe we should reimport the table in january?

Dec 26 2018, 10:30 AM · DBA

Dec 21 2018

Banyek added a comment to T210478: Migrate dbstore1002 to a multi instance setup on dbstore100[3-5].

Apart from what @Banyek commented, I have a further question: do you mean adding the current stagingdb to each instance?
stagingdb is around 300GB, so I am not sure if we'd have enough space to place _the current_ stagingdb on each instance, @Banyek would need to look at the original math done at T210478#4839965 to see how that would affect the final sizes of each host.

As @Banyek said, even if we place the _current_ stagingdb data on each instance, data would quickly become different (which is ok from my side, and should be expected).
If your question is about if we can place an empty stagingdb like on each instance, I think there is no problem with that. And probably is needed as users would need a place to store their own tables?

Dec 21 2018, 10:05 AM · Analytics-Radar, Patch-For-Review, User-Banyek, Analytics-Kanban, DBA
Banyek added a comment to T210478: Migrate dbstore1002 to a multi instance setup on dbstore100[3-5].

Looks good, one little question - would it be possible to add a staging-like db on every instance if needed in the future, or is it a mandatory choice that we have to make it now? I don't foresee the need for it, but better safe (and ask before) than sorry :)

Dec 21 2018, 9:58 AM · Analytics-Radar, Patch-For-Review, User-Banyek, Analytics-Kanban, DBA
Banyek added a comment to T210478: Migrate dbstore1002 to a multi instance setup on dbstore100[3-5].
root@cumin1001:~# mysql.py -hdbstore1002 -e "show all slaves status\G" | egrep -i "Connection"
              Connection_name: s1
              Connection_name: s2
              Connection_name: s3
              Connection_name: s4
              Connection_name: s5
              Connection_name: s6
              Connection_name: s7
              Connection_name: s8
              Connection_name: x1
Dec 21 2018, 9:48 AM · Analytics-Radar, Patch-For-Review, User-Banyek, Analytics-Kanban, DBA

Dec 20 2018

Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Dec 20 2018, 12:28 PM · User-Banyek, Schema-change-in-production, DBA, Schema-change
Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Dec 20 2018, 12:28 PM · User-Banyek, Schema-change-in-production, DBA, Schema-change
Banyek created P7935 (An Untitled Masterwork).
Dec 20 2018, 11:12 AM
Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Dec 20 2018, 10:56 AM · User-Banyek, Schema-change-in-production, DBA, Schema-change
Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Dec 20 2018, 10:34 AM · User-Banyek, Schema-change-in-production, DBA, Schema-change
Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Dec 20 2018, 10:32 AM · User-Banyek, Schema-change-in-production, DBA, Schema-change
Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Dec 20 2018, 10:16 AM · User-Banyek, Schema-change-in-production, DBA, Schema-change
Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Dec 20 2018, 9:52 AM · User-Banyek, Schema-change-in-production, DBA, Schema-change

Dec 19 2018

Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Dec 19 2018, 11:51 AM · User-Banyek, Schema-change-in-production, DBA, Schema-change
Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Dec 19 2018, 11:35 AM · User-Banyek, Schema-change-in-production, DBA, Schema-change
Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Dec 19 2018, 10:51 AM · User-Banyek, Schema-change-in-production, DBA, Schema-change
Banyek closed T211544: Drop FlaggedRevs tables in database for ptwikipedia as Resolved.
Dec 19 2018, 10:12 AM · User-Banyek, DBA, User-Kizule
Banyek closed T211544: Drop FlaggedRevs tables in database for ptwikipedia, a subtask of T211433: Remove FlaggedRevs for ptwikipedia, as Resolved.
Dec 19 2018, 10:12 AM · User-Kizule, Wikimedia-Site-requests
Banyek added a comment to T211544: Drop FlaggedRevs tables in database for ptwikipedia.
root@db1066.eqiad.wmnet[ptwiki]> show tables like 'flagged%';
+-----------------------------+
| Tables_in_ptwiki (flagged%) |
+-----------------------------+
| flaggedimages               |
| flaggedpage_config          |
| flaggedpage_pending         |
| flaggedpages                |
| flaggedrevs                 |
| flaggedrevs_promote         |
| flaggedrevs_statistics      |
| flaggedrevs_tracking        |
| flaggedtemplates            |
+-----------------------------+
9 rows in set (0.03 sec)
Dec 19 2018, 10:11 AM · User-Banyek, DBA, User-Kizule
Banyek added a comment to T211544: Drop FlaggedRevs tables in database for ptwikipedia.
root@db1122.eqiad.wmnet[ptwiki]> SET SESSION sql_log_bin=0;
Query OK, 0 rows affected (0.03 sec)
Dec 19 2018, 9:39 AM · User-Banyek, DBA, User-Kizule
Banyek added a comment to T211544: Drop FlaggedRevs tables in database for ptwikipedia.

I've created backups from the actual tables before the drop:

Dec 19 2018, 9:20 AM · User-Banyek, DBA, User-Kizule

Dec 14 2018

Banyek added a comment to T209488: Global rename of Massimo Telò → Teseo: supervision needed.

the progress url for rename is: https://meta.wikimedia.org/wiki/Special:GlobalRenameProgress/Teseo

Dec 14 2018, 9:24 AM · DBA, Wikimedia-Site-requests
Banyek added a comment to T210693: Create materialized views on Wiki Replica hosts for better query performance.

We didn't talked about this so far, but these views doesn't ask for having proper indexes?

Dec 14 2018, 8:34 AM · Patch-For-Review, User-Banyek, Platform Team Legacy (Watching / External), Analytics-Kanban, DBA, Data-Services, Analytics
Banyek added a comment to T209488: Global rename of Massimo Telò → Teseo: supervision needed.

@1997kB I'd say we should stick to the date on the calendar event

Dec 14 2018, 8:26 AM · DBA, Wikimedia-Site-requests
Banyek added a comment to T209488: Global rename of Massimo Telò → Teseo: supervision needed.

Aye, I am here, noted

Dec 14 2018, 8:16 AM · DBA, Wikimedia-Site-requests

Dec 12 2018

Banyek moved T211804: A huge spike on read rows for commonswiki from Triage to In progress on the DBA board.
Dec 12 2018, 9:29 PM · Platform Team Workboards (Done with CPT), MW-1.33-notes (1.33.0-wmf.8; 2018-12-11), DBA
Banyek created T211804: A huge spike on read rows for commonswiki.
Dec 12 2018, 6:22 PM · Platform Team Workboards (Done with CPT), MW-1.33-notes (1.33.0-wmf.8; 2018-12-11), DBA
Banyek added a comment to T211544: Drop FlaggedRevs tables in database for ptwikipedia.

Tables were renamed on db1122 for proof:

Dec 12 2018, 2:45 PM · User-Banyek, DBA, User-Kizule
Banyek added a comment to T211544: Drop FlaggedRevs tables in database for ptwikipedia.

I'll do first renaming the tables on db1122, and if nothing breaks this week, I'll do the drops

Dec 12 2018, 1:30 PM · User-Banyek, DBA, User-Kizule
Banyek created P7908 (An Untitled Masterwork).
Dec 12 2018, 10:53 AM
Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Dec 12 2018, 10:16 AM · User-Banyek, Schema-change-in-production, DBA, Schema-change
Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Dec 12 2018, 9:35 AM · User-Banyek, Schema-change-in-production, DBA, Schema-change
Banyek closed T211537: Degraded RAID on db1063 as Resolved.

The sync finished, thank you @Cmjohnson

	Virtual Drive: 0 (Target Id: 0)
	RAID Level: Primary-1, Secondary-0, RAID Level Qualifier-0
	State: Optimal
	Number Of Drives per span: 2
	Number of Spans: 6
	Current Cache Policy: WriteBack, ReadAheadNone, Direct, No Write Cache if Bad BBU
Dec 12 2018, 9:28 AM · DBA, ops-eqiad, SRE
Banyek added a comment to T210693: Create materialized views on Wiki Replica hosts for better query performance.

The materialized view generation completed.
The total size of the materialized views are ~150 G all together.

root@labsdb1010:~# find /srv -iname comment_mat.ibd -ls | awk '{size_in_g +=$7} END {print "Total size: " size_in_g/1024/1024/1024}'
Total size: 149.266

The total time view generation take is ~18 hours (!This run excluded enwiki_p.comments_mat as it was created earlier and took ~5hrs!)

root@labsdb1010:~# cat create_mat.log | egrep  "Starting|Completed"
Tue Dec 11 15:07:11 UTC 2018 - Starting
Wed Dec 12 04:18:08 UTC 2018 - Completed
Dec 12 2018, 8:27 AM · Patch-For-Review, User-Banyek, Platform Team Legacy (Watching / External), Analytics-Kanban, DBA, Data-Services, Analytics

Dec 11 2018

Banyek added a comment to T211544: Drop FlaggedRevs tables in database for ptwikipedia.

ptwikipedia lives in the s2 section, the following hosts needs to be done:

Dec 11 2018, 7:19 PM · User-Banyek, DBA, User-Kizule
Banyek added a comment to T211537: Degraded RAID on db1063.
root@db1063:~# megacli -PDList -aall | egrep -i "Slot|Firmw"
Slot Number: 0
Firmware state: Rebuild
Device Firmware Level: 0008

awesome, thanks!

Dec 11 2018, 7:12 PM · DBA, ops-eqiad, SRE
Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Dec 11 2018, 4:57 PM · User-Banyek, Schema-change-in-production, DBA, Schema-change
Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Dec 11 2018, 4:55 PM · User-Banyek, Schema-change-in-production, DBA, Schema-change
Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Dec 11 2018, 4:45 PM · User-Banyek, Schema-change-in-production, DBA, Schema-change
Banyek updated the task description for T85757: Dropping user.user_options on wmf databases.
Dec 11 2018, 4:34 PM · User-Banyek, Schema-change-in-production, DBA, Schema-change