Page MenuHomePhabricator

Prepare and check storage layer for gomwiktionary
Closed, ResolvedPublic

Description

The new wiki will be public.

Event Timeline

Marostegui triaged this task as Medium priority.Apr 20 2020, 3:36 PM
Marostegui moved this task from Triage to Blocked external/Not db team on the DBA board.
Marostegui added a subscriber: Marostegui.

Thanks for the heads up - let us know when the database is created so we can sanitize it

Thanks for the heads up - let us know when the database is created so we can sanitize it

Please use your mighty hammer now :)

Marostegui added a subscriber: Kormat.

Thanks - I will work with @Kormat on this!
Assigning to @Kormat so this gets blocked on us and not sent for views creation yet.

Sanitization is in place, waiting on a complete private data check before handing it over to Cloud for view creation.

Private data check passed.

I've created gomwiktionary_p, and given the labsdbuser role a grant to it. This is now ready for cloud-services-team to create the views.

Kormat moved this task from Blocked external/Not db team to Done on the DBA board.
Kormat added a subscriber: Bstorm.

cloud-services-team please create the views also on db1141 (temporary host replacing labsdb1011)

Andrew claimed this task.
Andrew added a subscriber: Andrew.

I always feel like I'm flying blind when I do this, but I've run the steps on https://wikitech.wikimedia.org/wiki/Add_a_wiki#Cloud_Services and this DB is now visible on Quarry. I think that means we're done.

This needs to be run on labsdb1011 as well. The reason why it needs to be run there is because it is being recovered from several crashes

I've run maintain-replica-indexes and maintain-views on labsdb1011; the maintain_meta_p command fails:

andrew@labsdb1011:~$ sudo /usr/local/sbin/maintain-meta_p --databases gomwiktionary
/usr/lib/python2.7/dist-packages/pymysql/cursors.py:166: Warning: (1007, u"Can't create database 'meta_p'; database exists")
  result = self._query(query)
/usr/lib/python2.7/dist-packages/pymysql/cursors.py:166: Warning: (1050, u"Table 'wiki' already exists")
  result = self._query(query)
Traceback (most recent call last):
  File "/usr/local/sbin/maintain-meta_p", line 375, in <module>
    main()
  File "/usr/local/sbin/maintain-meta_p", line 194, in main
    seed_schema(ops)
  File "/usr/local/sbin/maintain-meta_p", line 102, in seed_schema
    FROM meta_p.wiki;""")
  File "/usr/local/sbin/maintain-meta_p", line 44, in write_execute
    self.cursor.execute(query)
  File "/usr/lib/python2.7/dist-packages/pymysql/cursors.py", line 166, in execute
    result = self._query(query)
  File "/usr/lib/python2.7/dist-packages/pymysql/cursors.py", line 322, in _query
    conn.query(q)
  File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 852, in query
    self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 1053, in _read_query_result
    result.read()
  File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 1336, in read
    first_packet = self.connection._read_packet()
  File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 1010, in _read_packet
    packet.check_error()
  File "/usr/lib/python2.7/dist-packages/pymysql/connections.py", line 393, in check_error
    err.raise_mysql_exception(self._data)
  File "/usr/lib/python2.7/dist-packages/pymysql/err.py", line 107, in raise_mysql_exception
    raise errorclass(errno, errval)
pymysql.err.InternalError: (1347, u"'meta_p.legacy' is not VIEW")

labsdb1011 is still reimporting lots of data, so maybe we should wait till it is already finished - some moving pieces for that script might not be ready yet.

works for me! Can you ping on this task when ready? Or do you have an eta?

Yeah, ETA will be more than a week unfortunately :(
I will comment on this tag when ready. Thank you!

This is done as labsdb1011 has been recloned from labsdb1012