Maniphest T211939

Drop several views from srwikinews
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Urbanecm
	Dec 13 2018, 10:05 PM

Description

in T209761, several tables were dropped. Should be reflected on cloud too.

Details

	Subject	Repo	Branch	Lines +/-
	Revert "wikireplicas: depool labsdb1009 for updates"	operations/puppet	production	+4 -4
	wikireplicas: depool labsdb1009 for updates	operations/puppet	production	+4 -4

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		Kizule	T209251 Remove FlaggedRevs and add back rights autopatrolled, patroller (with enabled RCPatrol), rollbacker on srwikinews
		Resolved		bd808	T211939 Drop several views from srwikinews

Event Timeline

Urbanecm created this task.Dec 13 2018, 10:05 PM

Restricted Application added subscribers: • Petar.petkovic, Aklapper. · View Herald TranscriptDec 13 2018, 10:05 PM

Oh, thanks for opening of task @Urbanecm :D

Urbanecm mentioned this in T209761: Drop FlaggedRevs tables in database for srwikinews.Dec 13 2018, 10:07 PM

Krenair edited projects, added Data-Services; removed Cloud-Services.Dec 13 2018, 10:07 PM

In T209251#4741279, @Marostegui wrote:

If the extension is disabled and it is decided it won't be enabled again, there is no need to keep the tables and we should clean them up.
Once this is decided for sure, create a ticket for us to get them dropped.

Thanks

I think that someone on the DBA team needs to actually drop the tables in both prod and replicas still based on what I can see in the history of T209251: Remove FlaggedRevs and add back rights autopatrolled, patroller (with enabled RCPatrol), rollbacker on srwikinews. It looks like all that has been done at this point is disabling the extension.

That was done in T209761: Drop FlaggedRevs tables in database for srwikinews

$ ssh labsdb1009
$ sudo /usr/local/sbin/maintain-views --debug --clean --replace-all --database srwikinews
...
2018-12-13 22:21:43,708 INFO cleanup is enabled
2018-12-13 22:21:43,709 INFO cleaning 12 tables
2018-12-13 22:21:43,709 INFO Dropping view srwikinews_p.flaggedpage_config
2018-12-13 22:21:43,709 DEBUG SQL: drop view srwikinews_p.flaggedpage_config
2018-12-13 22:21:43,712 INFO Dropping view srwikinews_p.flaggedpage_pending
2018-12-13 22:21:43,712 DEBUG SQL: drop view srwikinews_p.flaggedpage_pending
2018-12-13 22:21:43,714 INFO Dropping view srwikinews_p.flaggedrevs_tracking
2018-12-13 22:21:43,714 DEBUG SQL: drop view srwikinews_p.flaggedrevs_tracking
2018-12-13 22:21:43,716 INFO Dropping view srwikinews_p.flaggedrevs_promote
2018-12-13 22:21:43,716 DEBUG SQL: drop view srwikinews_p.flaggedrevs_promote
2018-12-13 22:21:43,717 INFO Dropping view srwikinews_p.flaggedrevs_stats
2018-12-13 22:21:43,718 DEBUG SQL: drop view srwikinews_p.flaggedrevs_stats
2018-12-13 22:21:43,719 INFO Dropping view srwikinews_p.flaggedtemplates
2018-12-13 22:21:43,720 DEBUG SQL: drop view srwikinews_p.flaggedtemplates
2018-12-13 22:21:43,721 INFO Dropping view srwikinews_p.flaggedrevs_stats2
2018-12-13 22:21:43,721 DEBUG SQL: drop view srwikinews_p.flaggedrevs_stats2
2018-12-13 22:21:43,723 INFO Dropping view srwikinews_p.flaggedimages
2018-12-13 22:21:43,724 DEBUG SQL: drop view srwikinews_p.flaggedimages
2018-12-13 22:21:43,725 INFO Dropping view srwikinews_p.flaggedrevs_statistics
2018-12-13 22:21:43,726 DEBUG SQL: drop view srwikinews_p.flaggedrevs_statistics
2018-12-13 22:21:43,727 INFO Dropping view srwikinews_p.flaggedrevs
2018-12-13 22:21:43,727 DEBUG SQL: drop view srwikinews_p.flaggedrevs
2018-12-13 22:21:43,729 INFO Dropping view srwikinews_p.flaggedpages
2018-12-13 22:21:43,729 DEBUG SQL: drop view srwikinews_p.flaggedpages
2018-12-13 22:21:43,732 INFO Dropping view srwikinews_p.msg_resource_links
2018-12-13 22:21:43,732 DEBUG SQL: drop view srwikinews_p.msg_resource_links

Repeated on labsdb1010 and labsdb1011.

$ ssh tools-dev.wmflabs.org
$ sql srwikinews
(u3518@srwikinews.analytics.db.svc.eqiad.wmflabs) [srwikinews_p]> show tables like 'flagged%';
Empty set (0.00 sec)
$ sql --cluster=web srwikinews
(u3518@srwikinews.web.db.svc.eqiad.wmflabs) [srwikinews_p]> show tables like 'flagged%';
Empty set (0.00 sec)

The initial run on labsdb1010 had errors because --clean is trying to drop the srwikinews_p.comment_mat table that was built there by @Banyek for T210693: Create materialized views on Wiki Replica hosts for better query performance. Running repeatedly seems to have removed all of the views that need to be dropped leaving that raw table in-place. I'll open another task about how the error handling worked in the script; we can do better.

Everything looks good now, thanks!

bd808 mentioned this in T211940: maintain-views halt with fatal error when dropping a view fails.Dec 13 2018, 10:38 PM

Kizule mentioned this in T212333: Drop several views from ptwiki.Jan 2 2019, 8:51 PM

Probably the problem is caused as the comment_mat is not a view but a table, I'll check the maintain-views script to understand how it works

• Bstorm mentioned this in T210693: Create materialized views on Wiki Replica hosts for better query performance.Jan 4 2019, 4:19 AM

@Andrew I cleaned up the tables (T210693) which blocked using the maintain-views script

Here is everything ok.

Kizule mentioned this in T216481: Remove views on ep_* tables on the wikireplicas hosts.Feb 19 2019, 9:30 AM

Change 491454 had a related patch set uploaded (by GTirloni; owner: GTirloni):
[operations/puppet@production] wikireplicas: depool labsdb1009 for updates

https://gerrit.wikimedia.org/r/491454

Change 491454 merged by GTirloni:
[operations/puppet@production] wikireplicas: depool labsdb1009 for updates

https://gerrit.wikimedia.org/r/491454

Change 491478 had a related patch set uploaded (by GTirloni; owner: GTirloni):
[operations/puppet@production] Revert "wikireplicas: depool labsdb1009 for updates"

https://gerrit.wikimedia.org/r/491478

Change 491478 merged by GTirloni:
[operations/puppet@production] Revert "wikireplicas: depool labsdb1009 for updates"

https://gerrit.wikimedia.org/r/491478

Please ignore above commits,they were meant for T216481. Sorry, bad copy/paste.

FriedrickMILBarbarossa subscribed.Aug 3 2021, 6:26 PM

Restricted Application added a project: User-bd808. · View Herald TranscriptAug 3 2021, 6:26 PM

FriedrickMILBarbarossa added a project: Serbian-Sites.Aug 3 2021, 6:28 PM

FriedrickMILBarbarossa moved this task from Backlog to Closed on the Serbian-Sites board.Aug 3 2021, 6:48 PM