Page MenuHomePhabricator

maintain-views table filter not working for custom views on multiple tables
Open, Needs TriagePublic

Description

Coming from T305300 there seems to be an issue with maintain-views regenerating the view for centralauth_p.localuser.
The view in puppet is:

localuser:
  source:
    - localuser
    - globaluser
  view: >
    select lu_wiki, lu_name, lu_attached_timestamp, lu_attached_method, lu_attachment_method,
    lu_local_id, lu_global_id
  where: lu_global_id = gu_id AND gu_hidden_level=0

Please note that lu_attachment_method is there.
The current view is:

root@clouddb1021.eqiad.wmnet[centralauth_p]> show create table localuser\G
*************************** 1. row ***************************
                View: localuser
         Create View: CREATE ALGORITHM=UNDEFINED DEFINER=`viewmaster`@`%` SQL SECURITY DEFINER VIEW `localuser` AS select `centralauth`.`localuser`.`lu_wiki` AS `lu_wiki`,`centralauth`.`localuser`.`lu_name` AS `lu_name`,`centralauth`.`localuser`.`lu_attached_timestamp` AS `lu_attached_timestamp`,`centralauth`.`localuser`.`lu_attached_method` AS `lu_attached_method`,`centralauth`.`localuser`.`lu_local_id` AS `lu_local_id`,`centralauth`.`localuser`.`lu_global_id` AS `lu_global_id` from (`centralauth`.`localuser` join `centralauth`.`globaluser`) where `centralauth`.`localuser`.`lu_global_id` = `centralauth`.`globaluser`.`gu_id` and `centralauth`.`globaluser`.`gu_hidden_level` = 0
character_set_client: utf8
collation_connection: utf8_general_ci
1 row in set (0.001 sec)

Regenerating the view doesn't seem to include the new column lu_attachment_method

root@clouddb1021:~#  maintain-views --database centralauth --table localuser --replace --debug
2022-06-29 05:54:39,753 DEBUG Removing 0 dbs as sensitive
2022-06-29 05:54:39,769 INFO Full views for centralauth:
2022-06-29 05:54:39,770 INFO Custom views for centralauth:

The maintain-views script on clouddb1021 is also up-to-date and has that column:

root@clouddb1021:/etc# cat ./maintain-views.yaml | grep -w lu_attachment_method
      select lu_wiki, lu_name, lu_attached_timestamp, lu_attached_method, lu_attachment_method,

For some reason regenerating the view doesn't include the new column.

The column is there on the table under the view:

root@clouddb1021.eqiad.wmnet[centralauth_p]> show create table centralauth.localuser\G
*************************** 1. row ***************************
       Table: localuser
Create Table: CREATE TABLE `localuser` (
  `lu_wiki` varbinary(255) NOT NULL DEFAULT '',
  `lu_name` varbinary(255) NOT NULL DEFAULT '',
  `lu_attached_timestamp` binary(14) DEFAULT NULL,
  `lu_attached_method` enum('primary','empty','mail','password','admin','new','login') DEFAULT NULL,
  `lu_local_id` int(10) unsigned DEFAULT NULL,
  `lu_global_id` int(10) unsigned DEFAULT NULL,
  `lu_attachment_method` tinyint(3) unsigned DEFAULT NULL,
  PRIMARY KEY (`lu_wiki`,`lu_name`),
  KEY `lu_name` (`lu_name`,`lu_wiki`)
) ENGINE=InnoDB DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=8
1 row in set (0.001 sec)

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

I'm guessing the table filtering code

if args.table:
    fullviews = [t for t in config["fullviews"] if t == args.table]

    customviews = {}
    for view, meta in config["customviews"].items():
        if meta["source"] == args.table:
            customviews[view] = config["customviews"][view]

is getting confused because localuser is configured to have multiple source tables:

localuser:
  source:
    - localuser
    - globaluser

I guess this could be fixed with something like if isinstance(meta["source"], list) and args.table in meta["source"], but that would also make --table globaluser refresh the localuser view and I'm not sure if that's expected or not. Alternatively we could check if args.table is the first element in the list. Thoughts?

taavi renamed this task from maintain-views not regenerating centralauth views to maintain-views table filter not working for custom views on multiple tables.Jun 30 2022, 1:49 PM

I don't think refreshing the localuser view on each of those run is a problem (as it should be mostly a noop).