Page MenuHomePhabricator

Prepare and check storage layer for gomwiktionary
Closed, ResolvedPublic

Description

The new wiki will be public.

Event Timeline

Urbanecm created this task.Apr 20 2020, 3:25 PM
Marostegui triaged this task as Medium priority.Apr 20 2020, 3:36 PM
Marostegui moved this task from Triage to Blocked external/Not db team on the DBA board.
Marostegui added a subscriber: Marostegui.

Thanks for the heads up - let us know when the database is created so we can sanitize it

Thanks for the heads up - let us know when the database is created so we can sanitize it

Please use your mighty hammer now :)

Marostegui assigned this task to Kormat.May 20 2020, 1:33 PM
Marostegui added a subscriber: Kormat.

Thanks - I will work with @Kormat on this!
Assigning to @Kormat so this gets blocked on us and not sent for views creation yet.

Sanitization is in place, waiting on a complete private data check before handing it over to Cloud for view creation.

Private data check passed.

I've created gomwiktionary_p, and given the labsdbuser role a grant to it. This is now ready for cloud-services-team to create the views.

Kormat removed Kormat as the assignee of this task.May 22 2020, 2:21 PM
Kormat moved this task from Blocked external/Not db team to Done on the DBA board.
Kormat added a subscriber: Bstorm.

cloud-services-team please create the views also on db1141 (temporary host replacing labsdb1011)

Andrew closed this task as Resolved.Jun 3 2020, 2:00 PM
Andrew claimed this task.
Andrew added a subscriber: Andrew.

I always feel like I'm flying blind when I do this, but I've run the steps on https://wikitech.wikimedia.org/wiki/Add_a_wiki#Cloud_Services and this DB is now visible on Quarry. I think that means we're done.

Marostegui reopened this task as Open.Jun 3 2020, 2:05 PM

This needs to be run on labsdb1011 as well. The reason why it needs to be run there is because it is being recovered from several crashes

Andrew added a comment.Jun 3 2020, 2:40 PM

I've run maintain-replica-indexes and maintain-views on labsdb1011; the maintain_meta_p command fails:

andrew@labsdb1011:~$ sudo /usr/local/sbin/maintain-meta_p --databases gomwiktionary
/usr/lib/python2.7/dist-packages/pymysql/cursors.py:166: Warning: (1007, u"Can't create database 'meta_p'; database exists")
  result = self._query(query)
/usr/lib/python2.7/dist-packages/pymysql/cursors.py:166: Warning: (1050, u"Table 'wiki' already exists")
  result = self._query(query)
Traceback (most recent call last):
  File "/usr/local/sbin/maintain-meta_p", line 375, in <module>
    main()
  File "/usr/local/sbin/maintain-meta_p", line 194, in main
    seed_schema(ops)
  File "/usr/local/sbin/maintain-meta_p", line 102, in seed_schema
    FROM meta_p.wiki;""")
  File "/usr/local/sbin/maintain-meta_p", line 44, in write_execute
    self.cursor.execute(query)
  File "/usr/lib/python2.7/dist-packages/pymysql/cursors.py", line 166, in execute
    result = self._query(query)
  File "/usr/lib/python2.7/dist-packages/pymysql/cursors.py", line 322, in _query
    conn.query(q)
  File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 852, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 1053, in _read_query_result
    result.read()
  File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 1336, in read
    first_packet = self.connection._read_packet()
  File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 1010, in _read_packet
    packet.check_error()
  File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 393, in check_error
    err.raise_mysql_exception(self._data)
  File "/usr/lib/python2.7/dist-packages/pymysql/err.py", line 107, in raise_mysql_exception
    raise errorclass(errno, errval)
pymysql.err.InternalError: (1347, u"'meta_p.legacy' is not VIEW")

labsdb1011 is still reimporting lots of data, so maybe we should wait till it is already finished - some moving pieces for that script might not be ready yet.

Andrew added a comment.Jun 3 2020, 5:11 PM

works for me! Can you ping on this task when ready? Or do you have an eta?

Yeah, ETA will be more than a week unfortunately :(
I will comment on this tag when ready. Thank you!

CptViraj removed a subscriber: CptViraj.Jun 6 2020, 4:54 PM
Marostegui closed this task as Resolved.Tue, Jun 9, 4:53 AM

This is done as labsdb1011 has been recloned from labsdb1012