Create the GrowthExperiments Add Link tables in Wikimedia production:
The tables were added to createExtensionTables.php in 95f6e699 and 40f2e369, and to the private filter in b4fbbb75.
For now, only to testwiki: T277173: Deploy Add Link on testwiki
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | MMiller_WMF | T252822 [EPIC] Growth: "add a link" structured task 1.0 | |||
| Resolved | kostajh | T266437 Add a link engineering: backend product specifications | |||
| Resolved | kostajh | T261396 Add a link: engineering tasks for initial release | |||
| Resolved | Tgr | T278710 Add a link: production deployment | |||
| Resolved | Urbanecm_WMF | T275300 Update GrowthExperiments setup to tell users that table creation script is now necessary | |||
| Resolved | Tgr | T266913 Add a link engineering: create tables in Wikimedia production |
@Tgr we're probably getting close enough to do this, I'll let you mark as unstalled and put into Ready for Development if you agree we're ready.
I'd delay this until we are ready to deploy, it saves the effort of having to do schema changes.
@Tgr do you think we're close enough to do this now, and then deploy the puppet change to run refreshLinkRecommendations.php? AIUI the search functionality for hasrecommendation:link is in place, and the DB schema seems unlikely to change at this moment.
I said this in Slack, but I want to note this here as well. Before creation, we need to decide whether the table should be replicated to Toolforge wiki replicas, and tell DBA to make sure correct filtering is set in place. See T272957 as an example of past task asking for a table to be marked as private.
For the record, adding to $private_tables in puppet should be enough. @Marostegui is the expert on that through :).
That's correct, they need to be added there, before they get added to production.
Once the patch is ready, add me as a reviewer so I can merge and restart mysql on sanitarium hosts, so they pick the new replicating filters excluding those tables.
Thank you!
Change 666216 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[operations/puppet@production] Add GrowthExperiments tables to private_tables
Change 666216 merged by Kormat:
[operations/puppet@production] Add GrowthExperiments tables to private_tables
Mentioned in SAL (#wikimedia-operations) [2021-02-23T09:40:11Z] <kormat@cumin1001> START - Cookbook sre.hosts.downtime for 1:30:00 on 6 hosts with reason: Restart mariadb to pick up config changes T266913
Mentioned in SAL (#wikimedia-operations) [2021-02-23T09:40:18Z] <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:30:00 on 6 hosts with reason: Restart mariadb to pick up config changes T266913
https://gerrit.wikimedia.org/r/666216 is deployed to all sanitarium hosts, and mariadb has been restarted to pick it up. Let me know when the tables are created, so i can double-check that they don't get replicated.
I thought about this and I think we should leave table creation to the last moment. It's fast, schema changes in production are a hassle, and the recommendation caching logic is complicated, so let's wait until it passed product review on beta.
Done for testwiki in T277173: Deploy Add Link on testwiki. Will get back to it once we deploy to production wikis.
@Tgr Let's plan to do this on Tuesday? By that point our statsd monitoring code will be in place so we'll be visualize how long it takes to fill the task pool sizes.
If we let the maintenance script roll out with the train (and I don't think those two days would be worth the effort), then we can do the puppet change and the databases on Tuesday, and enable the link recommendations backend on group0/1/2 on Tuesday/Wednesday/Thursday in the post-train backport window.
Oh, sorry, you meant next Tuesday. We can do it tomorrow though, I already scheduled it and this task is trivial anyway.
Actually enabling the Add Link backend will be more complicated; I made T278710: Add a link: production deployment to track that.
The confusion is my fault, the Add Link board did have a date, I just failed to notice it.
For the record: the tables are being created on new wikis with GrowthExpeirments deployed (as running createExtensionTables.php is a standard part of installing an extension that has database tables). The tables are only needed on the older target wikis.
@Urbanecm_WMF: do you have an example of a wiki with the tables created? I haven't been able to find any yet.
Ohh, i see where i went wrong. These tables are in x1 section, not in the normal mw sections. x1 doesn't get replicated to wikireplicas, so this shouldn't be an issue.
I think this has happened everywhere because the tables are part of the createExtensionTables.php script which was run for other reasons, even on existing wikis (T279853: Migrate mentor/mentee relationship to a separate database table on Wikimedia wikis, probably).
tgr@mwmaint1002:~$ for wiki in `cat /srv/mediawiki/dblists/growthexperiments.dblist | tail -n+2`; do sql $wiki --cluster=extension1 -- -e "show tables like 'growthexperiments%';"; done +---------------------------------------+ | Tables_in_arwiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------+ | Tables_in_bnwiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +----------------------------------------+ | Tables_in_cawiki (growthexperiments%) | +----------------------------------------+ | growthexperiments_link_recommendations | | growthexperiments_link_submissions | | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +----------------------------------------+ +----------------------------------------+ | Tables_in_cswiki (growthexperiments%) | +----------------------------------------+ | growthexperiments_link_recommendations | | growthexperiments_link_submissions | | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +----------------------------------------+ +---------------------------------------+ | Tables_in_dawiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +----------------------------------------+ | Tables_in_elwiki (growthexperiments%) | +----------------------------------------+ | growthexperiments_link_recommendations | | growthexperiments_link_submissions | | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +----------------------------------------+ +---------------------------------------+ | Tables_in_eowiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------+ | Tables_in_eswiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------+ | Tables_in_euwiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------+ | Tables_in_fawiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------+ | Tables_in_frwiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------------+ | Tables_in_frwiktionary (growthexperiments%) | +---------------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------------+ +---------------------------------------+ | Tables_in_hewiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------+ | Tables_in_hiwiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------+ | Tables_in_hrwiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------+ | Tables_in_huwiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------+ | Tables_in_hywiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------+ | Tables_in_idwiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------+ | Tables_in_jawiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------+ | Tables_in_kowiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------+ | Tables_in_mswiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------+ | Tables_in_nowiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------+ | Tables_in_plwiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------+ | Tables_in_ptwiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------+ | Tables_in_rowiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------+ | Tables_in_ruwiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------+ | Tables_in_shwiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +-------------------------------------------+ | Tables_in_simplewiki (growthexperiments%) | +-------------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +-------------------------------------------+ +---------------------------------------+ | Tables_in_sqwiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------+ | Tables_in_srwiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------+ | Tables_in_svwiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------+ | Tables_in_tawiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +-----------------------------------------+ | Tables_in_testwiki (growthexperiments%) | +-----------------------------------------+ | growthexperiments_link_recommendations | | growthexperiments_link_submissions | | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +-----------------------------------------+ +---------------------------------------+ | Tables_in_tewiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------+ | Tables_in_thwiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------+ | Tables_in_trwiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +---------------------------------------+ | Tables_in_ukwiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+ +----------------------------------------+ | Tables_in_urwiki (growthexperiments%) | +----------------------------------------+ | growthexperiments_link_recommendations | | growthexperiments_link_submissions | | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +----------------------------------------+ +---------------------------------------+ | Tables_in_viwiki (growthexperiments%) | +---------------------------------------+ | growthexperiments_mentee_data | | growthexperiments_mentor_mentee | +---------------------------------------+
So apparently not quite true.
Change 684416 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):
[mediawiki/extensions/WikimediaMaintenance@master] createExtensionTables.php: check whether tables exist
I personally run it only for new wikis, because I'm not sure what happens when I run it for wikis with some of the tables existing. For those wikis, I ran mwscript sql.php --wiki=xxwiki --cluster=extension1 /path/to/table/definition.sql.
Mentioned in SAL (#wikimedia-operations) [2021-05-04T11:10:29Z] <Urbanecm> Create growthexperiments_link_recommendations and growthexperiments_link_submissions on arwiki,bnwiki,viwiki x1 (T266913)
Change 684416 merged by jenkins-bot:
[mediawiki/extensions/WikimediaMaintenance@master] createExtensionTables.php: check whether tables exist
Not yet done everywhere:
[urbanecm@mwmaint1002 ~]$ sql frwiki --cluster=extension1 wikiadmin@10.64.48.111(frwiki)> select * from growthexperiments_link_submissions limit 1; ERROR 1146 (42S02): Table 'frwiki.growthexperiments_link_submissions' doesn't exist wikiadmin@10.64.48.111(frwiki)> use kowiki; Database changed wikiadmin@10.64.48.111(kowiki)> select * from growthexperiments_link_submissions limit 1; ERROR 1146 (42S02): Table 'kowiki.growthexperiments_link_submissions' doesn't exist wikiadmin@10.64.48.111(kowiki)> use ptwiki Database changed wikiadmin@10.64.48.111(ptwiki)> select * from growthexperiments_link_submissions limit 1; ERROR 1146 (42S02): Table 'ptwiki.growthexperiments_link_submissions' doesn't exist wikiadmin@10.64.48.111(ptwiki)>
This is only done on wikis we deployed to after 95f6e699 and 40f2e369 and on wikis with add a link enabled.
@kostajh @Tgr Any reason to _not_ create the tables on all wikis, regardless of add a link status there? I find having the tables only on some wikis confusing, and since we'd already have to go through prod schema change process if we want to change anything, it's IMO time to run the creation script on all GE wikis and resolve this.
Yes, it is very confusing to have tables on just a few wikis. Let's create them everywhere (it is fine if they stay empty)
Note the extension isn't currently deployed to most wikis -- I was proposing to create them on wikis in growthexperiments.dblist (which is all wikis the extension is currently enabled at). Would creating them only on wikis the extension is available at be ok?
I don't think it's unusual to only have a certain extension table on some wikis. That said, I don't have anything against creating it on all GE wikis or even all wikis. As you say, it won't make much difference for schema changes.
Mentioned in SAL (#wikimedia-operations) [2021-06-22T13:29:14Z] <urbanecm> urbanecm@mwmaint1002:~$ foreachwikiindblist growthexperiments extensions/WikimediaMaintenance/createExtensionTables.php growthexperiments # T266913
I took the liberty to finish this, and create all GrowthExperiments tables at all GrowthExperiments wikis per DBA recommendation. Script output is at P16692. Running for wiki in $(cat /srv/mediawiki/dblists/growthexperiments.dblist | tail -n+2); do sql $wiki --cluster=extension1 -- -e "show tables like 'growthexperiments%';"; done shows the tables are indeed live everywhere.
Closing.