Page MenuHomePhabricator

Prepare and check storage layer for zhwikiversity
Closed, ResolvedPublic


The new wiki will be a public one.

Event Timeline

Liuxinyu970226 created this task.
Marostegui subscribed.

Let us know when this is created so we can sanitize it

Mentioned in SAL (#wikimedia-operations) [2018-08-02T12:58:48Z] <marostegui> Sanitize zhwikiversity satwiki T199599 T198401

Marostegui moved this task from Blocked external/Not db team to Done on the DBA board.
Marostegui edited projects, added Data-Services; removed Cloud-Services.
Marostegui added subscribers: Bstorm, bd808.

I have sanitized this wiki and my new user was created correctly with all the sanitization done.
It is ready for cloud-services-team to create the views.

I have run sudo /usr/local/sbin/maintain-replica-indexes --debug --database zhwikiversity on all 3 wiki replica hosts.

@Bstorm I can't finish this one due to pymysql.err.OperationalError: (1044, "Access denied for user 'maintainviews'@'localhost' to database 'satwiki\\_p'") on all 3 hosts. Your root powers are required.

I assume you meant zhwikiversity, I have granted that on all hosts. Try re-running the script.

for i in labsdb1009 labsdb1010 labsdb1011; do echo $i; -h$i -e " show grants for 'labsdbuser';" | grep -i zhwikiversity;done
GRANT SELECT, SHOW VIEW ON `zhwikiversity\\_p`.* TO 'labsdbuser'
GRANT SELECT, SHOW VIEW ON `zhwikiversity\\_p`.* TO 'labsdbuser'
GRANT SELECT, SHOW VIEW ON `zhwikiversity\\_p`.* TO 'labsdbuser'

I think the failure I'm seeing is the same upstream wildcard grant bug that we have seen before. The maintain-views script connects to the databsase server as the maintainviews user. That user is being denied the ability to operate on the zhwikiversity_p database.

2018-08-03 18:48:11,762 DEBUG SQL: GRANT SELECT, SHOW VIEW ON `zhwikiversity\_p`.* TO 'labsdbuser';
Traceback (most recent call last):
  File "/usr/local/sbin/maintain-views", line 594, in <module>
  File "/usr/local/sbin/maintain-views", line 575, in main
    ops.execute(fullviews, customviews)
  File "/usr/local/sbin/maintain-views", line 362, in execute
    self.db_p.replace('_', '\\_'))
  File "/usr/local/sbin/maintain-views", line 54, in write_execute
  File "/usr/lib/python3/dist-packages/pymysql/", line 166, in execute
    result = self._query(query)
  File "/usr/lib/python3/dist-packages/pymysql/", line 322, in _query
  File "/usr/lib/python3/dist-packages/pymysql/", line 852, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "/usr/lib/python3/dist-packages/pymysql/", line 1053, in _read_query_result
  File "/usr/lib/python3/dist-packages/pymysql/", line 1336, in read
    first_packet = self.connection._read_packet()
  File "/usr/lib/python3/dist-packages/pymysql/", line 1010, in _read_packet
  File "/usr/lib/python3/dist-packages/pymysql/", line 393, in check_error
  File "/usr/lib/python3/dist-packages/pymysql/", line 107, in raise_mysql_exception
    raise errorclass(errno, errval)
pymysql.err.OperationalError: (1044, "Access denied for user 'maintainviews'@'localhost' to database 'zhwikiversity\\_p'")

The bug is weird. The grant is already there if @Marostegui ran it, so the script doesn't need to do it. It only tries to do it if the database with the _p doesn't exist. It also often will randomly fail to create the DB. I think right now, it tries the grant first. If we create the DB, it'll run. It won't attempt to do the grant itself if the DB exists.

Yup, that did it. I created the DB manually, and it ran fine.

2018-08-03T19:51:57Z mwopenstackclients.DnsManager WARNING : Creating
2018-08-03T19:53:37Z mwopenstackclients.DnsManager WARNING : Creating
2018-08-03T19:54:44Z mwopenstackclients.DnsManager WARNING : Creating
2018-08-03T19:54:55Z mwopenstackclients.DnsManager WARNING : Creating
2018-08-03T19:56:55Z mwopenstackclients.DnsManager WARNING : Creating id_internalwikimedia.web.db.svc.eqiad.wmflabs.
2018-08-03T19:58:36Z mwopenstackclients.DnsManager WARNING : Creating satwiki.web.db.svc.eqiad.wmflabs.
2018-08-03T19:59:38Z mwopenstackclients.DnsManager WARNING : Creating wikimaniawiki.web.db.svc.eqiad.wmflabs.
2018-08-03T19:59:48Z mwopenstackclients.DnsManager WARNING : Creating zhwikiversity.web.db.svc.eqiad.wmflabs.

The dns step set up all the new wiki db names.

Bstorm claimed this task.

meta_p thing done as well.