Hi @Nikerabbit, as noted in the task description, we're planning on migrating a number of messages (and some entire message files) from the WikibaseMediaInfo extension into a new extension, MediaSearch. We want to avoid disrupting things on translatewiki.net and I was hoping to get your advice on making this transition as smooth as possible. My general plan is to merge a patch in WBMI removing the relevant messages and message files in tandem with the MediaSearch extension deployment, which will include the messages pulled from WBMI with different keys (swapping out the extension prefixes, mostly).

Is there anything we need to do differently to prevent duplicate work on translatewiki.net, or any other concerns? For example, if we merge a patch in MediaSearch with the messages from WBMI before MediaSearch is deployed, will those messages show up on translatewiki.net, or will that only happen post-deployment? I want to make sure we're getting the timing right.

My recommendation would be the following:

1. (we) Synchronize with translatewiki.net to ensure everything is up to date
2. Copy or move all message files to MediaSearch to preserve authorship and translations in languages under threshold (if copying, WBMI syncing should be disabled temporarily).
3. Remove messages that you do not need in MediaSearch en.json(s)
4. (we) Synchronize with translatewiki.net. This will drop unused keys from translation files.
5. Rename message keys
6. (we) Synchronize with translatewiki.net (we will perform mass rename on our own, which can take hours)
7. Remove unused messages from WBMI (if not done earlier) and restore syncing
8. (we) Synchronize with translatewiki.net
9. You can deploy now

@Raymond Can you handle this or do you need help?

In T266345#6587909, @Nikerabbit wrote:

@Raymond Can you handle this or do you need help?

Should be doable by me.

@AnneT Please add me as reviewer for all these patches so that I can do the necessary actions on translatewiki.net

Thanks very much for the information, @Nikerabbit and @Raymond.

FYI, it seems unlikely that we will be pursuing this separate extension for the time being. If we ever do, or if we move messages like this in the future, we will certainly follow the processes outlined here.

Hey @Raymond, I wanted to let you know that the Structured Data team has decided to go ahead and move forward with this plan to remove the Special:MediaSearch code from the WikibaseMediaInfo extension and move it to a new extension, MediaSearch.

Right now my team is reviewing an initial patch to migrate the code. Once that patch is ready to merge, can I ping you to ask you to synch the WBMI messages with translatewiki, so I can include an up-to-date batch of messages in the MediaSearch patch? I'm hoping we'll be ready to do this within a week.

I'll add you as a reviewer on that patch—right now I've copied over and renamed the English messages just for the sake of being able to test the extension locally, but when we're ready to merge I'll copy the i18n files directly, remove unneeded messages, and wait before renaming the messages, as outlined in Nikerabbit's comment.

Actually @Raymond, scratch that: I'm realizing that since we can't remove the messages from WBMI immediately and don't want to turn off synching to WBMI for a long time, we're going to need to do all of this in tandem with deploying the MediaSearch extension, which won't happen for a little while.

Here's a question: could you turn off synching to MediaSearch instead? This would allow us to at least include the en.json and qqq.json files in the master branch so we could test locally without errors until we can actually deploy the new extension.

Hey @Raymond, I just wanted to check in on my previous comment—does it make sense to turn off synching for the new MediaSearch extension until we're closer to deploying it? I want to make sure all our ducks are in a row before we merge the initial patch that contains just the en.json and qqq.json files for testing purposes.

In T266345#6812841, @AnneT wrote:

Hey @Raymond, I just wanted to check in on my previous comment—does it make sense to turn off synching for the new MediaSearch extension until we're closer to deploying it? I want to make sure all our ducks are in a row before we merge the initial patch that contains just the en.json and qqq.json files for testing purposes.

@AnneT Sorry, I missed your comment. But good news: The new MediaSearch extension was not and is not registered on translatewiki. Therefore nothing to do for now. Please give me a ping once I may register MediaSearch on translatewiki.net

Aha, perfect! Thanks, will do :)

Change 674609 had a related patch set uploaded (by Matthias Mullie; author: Matthias Mullie):

[mediawiki/extensions/MediaSearch@master] Build MediaSearch i18n messages from 2 sources

https://gerrit.wikimedia.org/r/674609

In T266345#6587909, @Nikerabbit wrote:

7. Remove unused messages from WBMI (if not done earlier) and restore syncing
8. ...
9. You can deploy now

Above is a bit of a problem. We already have the exact same code live, in production (in Extension:WikibaseMediaInfo).
We can't remove the i18n files from Extension:WikibaseMediaInfo until after we've fully deployed Extension:MediaInfo (which will replace what is currently served from Extension:WikibaseMediaInfo)

AIUI, we'll end up in one of these scenarios:

1. we first deploy Extension:MediaSearch, then sync the messages afterwards, but this means we'll have a brief moment with stale & very incomplete messages.
2. we first sync the messages, then deploy Extension:MediaSearch, but this means Extension:WikibaseMediaInfo may no longer receive message updates in the meantime (this could be days, if we encounter issues with deployment)

I guess we could bite the bullet and live with stale messages for a few days if we have to, but I'd rather not.

I just submitted a patch that would leverage the old PHP-style message format to serve as a stopgap solution by reading message files from both extensions & merge them for a short time.
The idea is basically that:

• Extension:MediaSearch registers ExtensionMessagesFiles instead of MessagesDirs
• The messagesfile in Extension:MediaSearch reads both its own (if any) and Extension:WikibaseMediaInfo's json files (but changes wikibasemediainfo-special-mediasearch- prefixes to mediasearch-)
• As long as Extension:WikibaseMediaInfo json continue to receive updates to the relevant messages, they'll continue to be used in Extension:MediaSearch
• We can copy json files to Extension:MediaSearch whenever we want
• Once json files have been copied, we remove them from Extension:MediaSearch - now the script starts showing the Extension:MediaSearch versions
• And we can now remove the ExtensionMessagesFiles script and simply register MessagesDirs instead, which will continue to show thee same messages

I think we can still follow the process outlined by @Nikerabbit, but it decouples the i18n message copying from the rest of the deployment: we could deploy any time before, during or after the messages copy, since it'll fall over to the messages from the other extension.

However, does the above hacky workaround work with the translatewiki systems & processes? Other things I've overlooked?

@Raymond do you have thoughts on Matthias's comment above?

In T266345#7000054, @AnneT wrote:

@Raymond do you have thoughts on Matthias's comment above?

Sounds good for me. I cannot test @matthiasmullie 's patch at the moment but this way it should works.

One suggestion: Commit a patch to the translatewiki.net repo whith enabling Extension:MediaSearch and disabling Extension:WBMI (by # comments, not complete removing at the moment) in the same patch. This way the switch can be done a bt faster

Change 682095 had a related patch set uploaded (by Matthias Mullie; author: Matthias Mullie):

[mediawiki/extensions/WikibaseMediaInfo@master] Use consistent prefix for mediasearch i18n messages

https://gerrit.wikimedia.org/r/682095

Change 682096 had a related patch set uploaded (by Matthias Mullie; author: Matthias Mullie):

[mediawiki/extensions/MediaSearch@master] Remove messages

https://gerrit.wikimedia.org/r/682096

We will soon start deploying Extension:MediaSearch without i18n messages, using the stopgap solution described above (T266345#6967371).
We expect it'll take ~3 weeks before everything is up & running and has overtaken the original extension (which will continue to be the place that the i18n messages live)
Once all of that has settled and the risk of having to revert anything has gone away, we'll continue the i18n messages migration as outlined by @Nikerabbit (T266345#6587909) & @Raymond (T266345#7001995)
Does that make sense?

Change 682095 merged by jenkins-bot:

[mediawiki/extensions/WikibaseMediaInfo@master] Use consistent prefix for mediasearch i18n messages

https://gerrit.wikimedia.org/r/682095

Change 674609 merged by jenkins-bot:

[mediawiki/extensions/MediaSearch@master] Build MediaSearch i18n messages from 2 sources

https://gerrit.wikimedia.org/r/674609

Change 682096 merged by jenkins-bot:

[mediawiki/extensions/MediaSearch@master] Remove messages

https://gerrit.wikimedia.org/r/682096

Change 702459 had a related patch set uploaded (by Anne Tomasevich; author: Anne Tomasevich):

[mediawiki/extensions/MediaSearch@master] Add i18n files from WikibaseMediaInfo

https://gerrit.wikimedia.org/r/702459

Change 702460 had a related patch set uploaded (by Anne Tomasevich; author: Anne Tomasevich):

[mediawiki/extensions/MediaSearch@master] Remove messages not needed by MediaSearch

https://gerrit.wikimedia.org/r/702460

Change 702461 had a related patch set uploaded (by Anne Tomasevich; author: Anne Tomasevich):

[mediawiki/extensions/MediaSearch@master] Rename message keys

https://gerrit.wikimedia.org/r/702461

Change 702465 had a related patch set uploaded (by Anne Tomasevich; author: Anne Tomasevich):

[mediawiki/extensions/WikibaseMediaInfo@master] Remove all messages related to Special:MediaSearch

https://gerrit.wikimedia.org/r/702465

Hi @Raymond, I've opened up patches to cover the process outlined in this comment so we can get these messages migrated from WikibaseMediaInfo to MediaSearch. Some thoughts and questions:

• I think it would be best to first move the messages to MediaSearch, then delete them from WBMI after that's all done, so I think we will need for you to disabling syncing with WBMI temporarily.
• Please let me know if these patches sufficiently cover what you'll need to sync everything with translatewiki.net, or if there's something I could do differently to facilitate this process
• In terms of actually syncing everything and merging these patches: it would probably be best if we could do this all between when one branch is cut and the next, so that all of the work can roll out at once via the train one week. FYI, those of us on the Structured Data team will be unavailable all of next week. Please let me know how you'd like to coordinate this. I'm not sure if you need to merge a patch before syncing to translatewiki.net, and if you'd like to merge the patches yourself or have one of us do it. Either way, I am happy to coordinate that here or on IRC.

Thanks very much for your help!

In T266345#7188586, @AnneT wrote:

Hi @Raymond, I've opened up patches to cover the process outlined in this comment so we can get these messages migrated from WikibaseMediaInfo to MediaSearch. Some thoughts and questions:

• I think it would be best to first move the messages to MediaSearch, then delete them from WBMI after that's all done, so I think we will need for you to disabling syncing with WBMI temporarily.
• Please let me know if these patches sufficiently cover what you'll need to sync everything with translatewiki.net, or if there's something I could do differently to facilitate this process
• In terms of actually syncing everything and merging these patches: it would probably be best if we could do this all between when one branch is cut and the next, so that all of the work can roll out at once via the train one week. FYI, those of us on the Structured Data team will be unavailable all of next week. Please let me know how you'd like to coordinate this. I'm not sure if you need to merge a patch before syncing to translatewiki.net, and if you'd like to merge the patches yourself or have one of us do it. Either way, I am happy to coordinate that here or on IRC.

Thanks very much for your help!

Patches look OK, thank you.

Could the messages be removed from WBMI in the same train? This way I could move and rename the translations in translatewiki.net so that the translator's history could be kept. This is not a hard condition but more smooth.

I can merge all patches by myself.

Pls ping me on IRC (Raymond or Raymond_) for further coordination if needed.

In T266345#7189130, @Raymond wrote:

Could the messages be removed from WBMI in the same train? This way I could move and rename the translations in translatewiki.net so that the translator's history could be kept. This is not a hard condition but more smooth.

I *think* that should be possible if we adjust the current <load-from-dual-sources> hack that we have in place.
I've added feedback in Gerrit, but haven't actually tested it out for real yet.

Change 702752 had a related patch set uploaded (by Anne Tomasevich; author: Anne Tomasevich):

[mediawiki/extensions/WikibaseMediaInfo@master] Remove all messages related to Special:MediaSearch

https://gerrit.wikimedia.org/r/702752

Change 702752 abandoned by Anne Tomasevich:

[mediawiki/extensions/WikibaseMediaInfo@master] Remove all messages related to Special:MediaSearch

Reason:

Mixed up depends-on and change-id, this shouldn't exist

https://gerrit.wikimedia.org/r/702752

AFAICT, all patches and their depends-on relationships look good and can be merged at will, according to whatever schedule best suits translatewiki to do the rest of the operations on their side.

Change 702459 merged by jenkins-bot:

[mediawiki/extensions/MediaSearch@master] Add i18n files from WikibaseMediaInfo

https://gerrit.wikimedia.org/r/702459

Change 702460 merged by jenkins-bot:

[mediawiki/extensions/MediaSearch@master] Remove messages not needed by MediaSearch

https://gerrit.wikimedia.org/r/702460

Change 702461 merged by jenkins-bot:

[mediawiki/extensions/MediaSearch@master] Rename message keys and remove messages hack

https://gerrit.wikimedia.org/r/702461

Change 702465 merged by Raimond Spekking:

[mediawiki/extensions/WikibaseMediaInfo@master] Remove all messages related to Special:MediaSearch

https://gerrit.wikimedia.org/r/702465

Change 703069 had a related patch set uploaded (by Raimond Spekking; author: Raimond Spekking):

[translatewiki@master] [MediaSearch] Register extension

https://gerrit.wikimedia.org/r/703069

Change 703069 merged by jenkins-bot:

[translatewiki@master] [MediaSearch] Register extension

https://gerrit.wikimedia.org/r/703069