@jcrespo, I see that this is placed in the "Blocked external/Not db team" on the DBA board. Can you clarify who you think ought to own this task?

Maybe we should be talking to @chasemp and @madhuvishy about getting this done... ?

I believe now that it is unblocked, the tables are actually replicated, so labs team or whoever maintains the views needs to enable them to be visible for users.

I am standing by in case there is some support needed about transmission from production, but I believe my part (replication itself) was done/enabled back in the day when the tables were created.

I can confirm they are in labs:

$ mysql -h labsdb1001 wikidatawiki -e "SELECT * FROM ores_classification LIMIT 10"
+----------+-----------+-------------+-------------+-------------------+--------------------+
| oresc_id | oresc_rev | oresc_model | oresc_class | oresc_probability | oresc_is_predicted |
+----------+-----------+-------------+-------------+-------------------+--------------------+
| 11098302 | 374419683 |           7 |           1 |             0.027 |                  0 |
| 11118144 | 374462067 |           7 |           1 |             0.004 |                  0 |
| 11118145 | 374462049 |           7 |           1 |             0.024 |                  0 |
| 11118146 | 374462070 |           7 |           1 |             0.161 |                  0 |
| 11118147 | 374462080 |           7 |           1 |             0.147 |                  0 |
| 11118148 | 374462090 |           7 |           1 |             0.022 |                  0 |
| 11118149 | 374462101 |           7 |           1 |             0.033 |                  0 |
| 11118150 | 374462103 |           7 |           1 |             0.007 |                  0 |
| 11118151 | 374462107 |           7 |           1 |             0.025 |                  0 |
| 11118152 | 374462087 |           7 |           1 |             0.005 |                  0 |
+----------+-----------+-------------+-------------+-------------------+--------------------+

Great! Thanks. Resolving

Sorry I was not clear enough- the tables are replicated, but they are probably not exposed to the users yet, that is the part that someone has to figure out.

@jcrespo is right:

ladsgroup@tools-bastion-03:~$ sql fawiki "select oresm_id from ores_model limit 1;"
ERROR 1146 (42S02) at line 1: Table 'fawiki_p.ores_model' doesn't exist

@jcrespo I see. Who should figure this out?

I think labs ops will know, but they are mostly unavailable/busy this week, so I will ask you to be patient.

@Krenair may know, but it is not something he has to do (but I think he helped with the new script).

I think what they do is patching https://phabricator.wikimedia.org/diffusion/OPUP/browse/production/modules/role/templates/labsdb/maintain-views.yaml (not 100% sure that is the right file), but amending that may speed up the process?

I do the amending.

Change 320804 had a related patch set uploaded (by Ladsgroup):
labs: add ores_classification and ores_model tables

https://gerrit.wikimedia.org/r/320804

In T148561#2786440, @jcrespo wrote:

@Krenair may know, but it is not something he has to do (but I think he helped with the new script).

The code and files are different but it is the same process as with the old script. You patch the config to include the table in the (in this case) full_views list, then run the script against the affected databases (on all labsdb mysql replica servers) to have it actually create the views (along with any other changes that may have been missed by changes to the file being made without the script being run). Without this process taking place, there is little point replicating anything as no users will be able to see it.

@AlexMonk-WMF, this is not a replication or Database issue, as I just demonstrated.

Some of the subtasks you added here are not according to the description. https://phabricator.wikimedia.org/T50930#2585212 If you want to track LabsDB issues, I would kindly suggest creating a separate, specific task.

In T148561#2787065, @jcrespo wrote:

@AlexMonk-WMF, this is not a replication or Database issue, as I just demonstrated.

It's blocking users from accessing parts of the replicas, so it is relevant.

Are these https://gerrit.wikimedia.org/r/#/c/320804/2/modules/role/templates/labsdb/maintain-views.yaml views that need to be exposed in every DB or some specific DB?

Is there meant to be a table in enwiki and olowiki and on

In T148561#2788886, @chasemp wrote:

Are these https://gerrit.wikimedia.org/r/#/c/320804/2/modules/role/templates/labsdb/maintain-views.yaml views that need to be exposed in every DB or some specific DB?

Is there meant to be a table in enwiki and olowiki and on

maintain-views doesn't check that, if a database is getting public views and a defined view's source exists in that database, it should create a view. These wikis have the table(s), so will get the view(s) if replication has been set up for them:

krenair@terbium:~$ ./foreachdbs.sh "select table_schema from information_schema.tables where table_name = 'ores_classification';"
enwiki
nlwiki
plwiki
ptwiki
trwiki
wikidatawiki
ruwiki
fawiki
krenair@terbium:~$ ./foreachdbs.sh "select table_schema from information_schema.tables where table_name = 'ores_model';"
enwiki
nlwiki
plwiki
ptwiki
trwiki
wikidatawiki
ruwiki
fawiki

sure, thanks, I understand. I meant something more like: is this a table to be exposed on a single wiki one-time, on these wikis one-time, on existing wiki's starting with these as X happens to add the table, or only new wiki's from here forward. The outline here is somewhat opaque, and I don't understand what the expectation is. I can certainly run maintain-views across for these DBs/tables as a one-time thing, but I don't expect to have it running as a cron until the new labsdb setup at this time.

on existing wiki's starting with these as X happens to add the table

We have 8 wikis and we're expanding. I expect to have more wikis with ores_* tables ready soon.

In T148561#2791429, @Halfak wrote:

on existing wiki's starting with these as X happens to add the table

We have 8 wikis and we're expanding. I expect to have more wikis with ores_* tables ready soon.

Alright, I think I understand then thanks. I'll take a pass at this for the existing 8 and we can treat newcomers when they pop up in a new issue. Hopefully, soon we will not have this be as manual / adhoc.

For context there are two layers to this and neither is much good without the other. The title of Replicate ores_classification and ores_model tables in labs is probably most literally construed as completed now but we manage access to the underlying tables by creating views and have a permission model around it. Something like Expose x tables to labs users via labsdb would be more inclusive. This is not a big deal at all, just a thought on why I was momentarily confused about the scope here. The replication and views specifics are definitely inside baseball. We have a big week here and I think we can get to this wednesday 11/16 or so.

In T148561#2787065, @jcrespo wrote:

@AlexMonk-WMF, this is not a replication or Database issue, as I just demonstrated.

Some of the subtasks you added here are not according to the description. https://phabricator.wikimedia.org/T50930#2585212 If you want to track LabsDB issues, I would kindly suggest creating a separate, specific task.

If it's inconvenient for @jcrespo to have this replication tracking task as a parent for this kind of Labs DB / views tasks, I don't see why we can't break those off onto a separate parent task indeed. Let's clarify that on T50930 as well.

Change 320804 merged by Rush:
labs: add ores_classification and ores_model tables

https://gerrit.wikimedia.org/r/320804