Page MenuHomePhabricator

Convert Tendril TokuDB tables to InnoDB
Closed, ResolvedPublic

Description

We no longer maintain TokuDB on 10.4 (Buster only ships 10.4) and this is a blocker to get the whole Tendril/Dbtree infra migrated to Buster.
There are several issues with it on 10.4 and the new jemalloc version:
T248957 tracks part of it and the workaround to get TokuDB enabled on 10.4 (if the package is compiled with that version, 10.4.12-1 isnt't, 10.4.12-2 (is, but not uploaded to the repo).

Upstream issues:
https://github.com/jemalloc/jemalloc/issues/937
https://jira.percona.com/browse/PS-4393
https://jira.mariadb.org/browse/MDEV-15034
https://jira.mariadb.org/browse/MDEV-16676
https://jira.mariadb.org/browse/MDEV-16183

Tendril is the last piece of infra that uses TokuDB.
There are several (big) tables there using it (and some other empty/non used):

+-----------------------+--------------+
| table_name            | table_schema |
+-----------------------+--------------+
| client_statistics_log | tendril      |
| general_log_sampled   | tendril      |
| global_status_log     | tendril      |
| global_status_log_5m  | tendril      |
| index_statistics_log  | tendril      |
| innodb_locks_log      | tendril      |
| innodb_trx_log        | tendril      |
| queries               | tendril      |
| queries_seen_log      | tendril      |
| query_footprint_log   | tendril      |
| slave_status_log      | tendril      |
| slow_log_sampled      | tendril      |
| table_statistics_log  | tendril      |
| user_statistics_log   | tendril      |
+-----------------------+--------------+
14 rows in set (0.23 sec)

Probably ToluDB is used for compression but nothing else.
Some of those tables are purged by an event, and we keep 7 days of data, which is probably not needed:

root@db1115.eqiad.wmnet[tendril]> show create event tendril_purge_global_status_log_5m\G
*************************** 1. row ***************************
               Event: tendril_purge_global_status_log_5m
            sql_mode:
           time_zone: SYSTEM
        Create Event: CREATE DEFINER=`root`@`208.80.154.151` EVENT `tendril_purge_global_status_log_5m` ON SCHEDULE EVERY 5 MINUTE STARTS '2014-05-23 00:00:00' ON COMPLETION NOT PRESERVE ENABLE DO begin

    if (get_lock('tendril_purge_global_status_log_5m', 1) = 0) then
      signal sqlstate value '45000' set message_text = 'get_lock';
    end if;

    select @stamp := now() - interval 7 day;
    delete from global_status_log_5m where stamp < @stamp limit 10000;

    do release_lock('tendril_purge_global_status_log_5m');
  end
character_set_client: utf8
collation_connection: utf8_general_ci
  Database Collation: latin1_swedish_ci
1 row in set (0.00 sec)

For global_status maybe we don't need a whole month of data:

root@db1115.eqiad.wmnet[tendril]> select * from global_status_log order by stamp asc limit 1;
+-----------+---------------------+---------+-----------+
| server_id | stamp               | name_id | value     |
+-----------+---------------------+---------+-----------+
|      1273 | 2020-03-03 00:00:00 |     421 | 180367401 |
+-----------+---------------------+---------+-----------+
1 row in set (0.73 sec)

Event Timeline

Marostegui changed the task status from Open to Stalled.Apr 1 2020, 8:01 AM
Marostegui triaged this task as Medium priority.
Marostegui moved this task from Triage to Backlog on the DBA board.

Just for the record global_status_log and global_status_log_5m are now under control in terms of growth: T252331: tendril_purge_global_status_log_5m and global_status_log needs more frequent purging
general_log_sampled is not used, I have moved it to InnoDB and removed its partitions.

I was digging into some tables in tendril, I noticed query_footprint_log is empty and running TokuDB, so I have converted it to InnoDB.

root@db1115.eqiad.wmnet[information_schema]> select table_schema,table_name,engine from tables where engine='Tokudb';
+--------------+-----------------------+--------+
| table_schema | table_name            | engine |
+--------------+-----------------------+--------+
| tendril      | client_statistics_log | TokuDB |
| tendril      | global_status_log     | TokuDB |
| tendril      | global_status_log_5m  | TokuDB |
| tendril      | index_statistics_log  | TokuDB |
| tendril      | innodb_locks_log      | TokuDB |
| tendril      | innodb_trx_log        | TokuDB |
| tendril      | queries               | TokuDB |
| tendril      | queries_seen_log      | TokuDB |
| tendril      | slave_status_log      | TokuDB |
| tendril      | slow_log_sampled      | TokuDB |
| tendril      | table_statistics_log  | TokuDB |
| tendril      | user_statistics_log   | TokuDB |
+--------------+-----------------------+--------+
12 rows in set (0.221 sec)
Marostegui claimed this task.

I am going to close this, we are not going to convert more tables than the ones we've already done. The above tables are still running tokudb, but unless we had issues we shouldn't spend time converting them to InnoDB (some of them are huge).
Let's kill tendril instead.