Page MenuHomePhabricator

Show Now Commons info for files that were moved
Closed, ResolvedPublic1 Story Points

Description

Motivation
When a file is moved to Commons, most of the time it should be deleted from the source wiki. Reasons are:

  • We don’t want to have two copies of the same file.
  • Changes should be made to the version on Commons, and the files should not develop differently
  • As long as the file on the source wiki is not deleted, it is shown, and not the one on Commons.

We should support this as well as possible.

e.g. see: {{Now Commons|1=Crow-Hassan Park Reserve 030527.jpg|2=no|date=2010-07-01|reviewer=SomeUser}}

Acceptance Criteria

  • Change the text below the success message to: Please return to the original file and add a template to declare that the file is now on <target wiki name>.
  • For files coming from wikis where we can guess the correct template, use this text:: Please return to the original file and add the following template to declare that the file is now on <target wiki name>: {{Now Commons| ... }} ( with fitting parameters pre filled )
  • Use the wikidata item to find the appropriate template for the source wiki
  • When there is no usable Wikidata source configured to retrieve a usable template, show the current generic message as a fallback

Note

  • Part of the job is to take a look at the Now Commons template and figure out what's useful and needed for this story

Findings
I've thumbed through a few wikis with the NowCommons template, linked by this wikidata item:

*wiki**in use**translated**filename**other useful params*
afwikiyesyesyesno
alswikinoyesyesno
amwikinoyesyesdate
arwikiyesyesyesno
aswikiyesnoyesdate, bot
ptwikisourcenoyesyesno
plwikisourcenoyesyesno
frwikisourcenoyesyesno
enwikisourcenoyesyesno
ruwikibooksnoyesyesno
plwikibooksnoyesyesno
nlwikibooksnoyesyesno
jawikibooksnoyesyesno
itwikibooksyesyesyesno
metawikinoyesyesno

My conclusion is that the templates are not widely used, but users have in fact gone through the trouble of providing translations. In almost every case, only the file name parameter is supported so there's not much point in us trying to integrate with the bot parameter. In the next task, where we make the edits automatically, we should decide on a change tag, username, or edit summary method for marking the edits as ours, since we can't count on the template to support inclusion of the bot name in text.

Demo

Run a successful import either on the beta cluster or on test.wikipedia.org.

Details

Related Gerrit Patches:
operations/mediawiki-config : masterNew configuration to pull sitelinks from Wikidata
mediawiki/extensions/FileImporter : masterSimplify WikidataTemplateLookup code utilizing array_reduce()
mediawiki/extensions/FileImporter : masterSuggest NowCommons template for source wiki

Event Timeline

Restricted Application added a subscriber: Liuxinyu970226. · View Herald TranscriptMay 21 2019, 1:19 PM
awight renamed this task from Show Now Commons info for files that where moved from en.wikipedia to Show Now Commons info for files that were moved from en.wikipedia.May 21 2019, 1:19 PM
WMDE-Fisch set the point value for this task to 5.May 21 2019, 1:22 PM
awight updated the task description. (Show Details)May 27 2019, 11:13 AM
JStrodt_WMDE updated the task description. (Show Details)May 27 2019, 6:30 PM

Here's a screenshot of the feature as I'm currently implementing it:

When the template cannot be automatically discovered for the source wiki, we'll fall back to the original text which doesn't include the "for example, <EXAMPLE>".

A quick note about how we discover the equivalent NowCommons template, and the alternatives. There is already a correspondence built on Wikidata, the site links on https://www.wikidata.org/wiki/Q5611625 . This solution has the following properties:

  • doesn't require duplicate data entry or a custom data format (i.e. extending CommonsHelper2 config)
  • doesn't require developer intervention (e.g. config files)
  • encourages the wiki-way of linking our templates across wikis
  • editable by normal users

This last point might be a liability though, as with everything on-wiki. If the sitelinks are vandalized, we'll be showing the garbage on the import success screen. Once we automate the process, this might become a bigger consideration, since there won't be an explicit human review of the template name.

The simplest alternative would be to introduce a configuration variable for each wiki, where the template name can be specified.

@awight sorry for the late change, but could you adapt the text to how it is currently phrased in the description? Johanna went over it and polished it.

And just to be sure: Your comment (T224007#5216020) is just general observation / food for discussion, right?

@awight sorry for the late change, but could you adapt the text to how it is currently phrased in the description? Johanna went over it and polished it.

Thanks, I'll update the text.

And just to be sure: Your comment (T224007#5216020) is just general observation / food for discussion, right?

It's a description of the implementation I used--open to more discussion, of course!

Updated screenshot:

Change 512675 had a related patch set uploaded (by Awight; owner: Awight):
[mediawiki/extensions/FileImporter@master] Suggest NowCommons template for source wiki

https://gerrit.wikimedia.org/r/512675

And just to be sure: Your comment (T224007#5216020) is just general observation / food for discussion, right?

It's a description of the implementation I used--open to more discussion, of course!

Apologies, re-reading the task I see that we had restricted the scope to just enwiki. I definitely went beyond that by generalizing to all wikis already... It would be possible to split the patch up to separate the Wikidata retrieval from an enwiki-only prototype, but we should discuss whether it's worth the trouble at this point. There's actually a very small amount of code involved and I've written tests, so if we decide this is good approach then I think we might want to just keep the code as-is.

Jan_Dittrich added a subscriber: Jan_Dittrich.EditedMay 29 2019, 8:55 AM

We kindly ask you …

I suggest "Please"

and add this template to declare

I suggest "add the following temp…"
Because "this" could point to a previously mentioned thing (which it does not)

@JStrodt_WMDE – is this fine?

Lea_WMDE renamed this task from Show Now Commons info for files that were moved from en.wikipedia to Show Now Commons info for files that were moved.May 29 2019, 8:57 AM
Lea_WMDE triaged this task as Normal priority.
Lea_WMDE updated the task description. (Show Details)
awight updated the task description. (Show Details)May 29 2019, 8:58 AM
WMDE-Fisch updated the task description. (Show Details)May 29 2019, 9:11 AM
WMDE-Fisch changed the point value for this task from 5 to 3.

We kindly ask you …

I suggest "Please"

and add this template to declare

I suggest "add the following temp…"

Thanks, the patch includes this improved text now.

Change 514502 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)):
[mediawiki/extensions/FileImporter@master] Simplify WikidataTemplateLookup code utilizing array_reduce()

https://gerrit.wikimedia.org/r/514502

Change 512675 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] Suggest NowCommons template for source wiki

https://gerrit.wikimedia.org/r/512675

TODO: We still need to add&deploy a config change for production that sets the Wikidata address so this can be used.

awight added a comment.Jun 6 2019, 1:31 PM

TODO: We still need to add&deploy a config change for production that sets the Wikidata address so this can be used.

Much appreciated! Patching that now...

Change 514715 had a related patch set uploaded (by Awight; owner: Awight):
[operations/mediawiki-config@master] New configuration to pull from Wikidata

https://gerrit.wikimedia.org/r/514715

Change 514502 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] Simplify WikidataTemplateLookup code utilizing array_reduce()

https://gerrit.wikimedia.org/r/514502

I used the test beta wiki, moving this file from de.wikipedia.org there: https://test.wikimedia.beta.wmflabs.org/wiki/File:Rajamangala_University_of_Technology_Rattanakosin_Salaya_Campus_Stadium.jpg

What I received was the message if we do not know what the template is, but I was expecting to be suggested using Vorlage:NowCommons, because this is what the Wikidata item has in store for de.wikipedia.org

I used the test beta wiki, moving this file from de.wikipedia.org there: https://test.wikimedia.beta.wmflabs.org/wiki/File:Rajamangala_University_of_Technology_Rattanakosin_Salaya_Campus_Stadium.jpg
What I received was the message if we do not know what the template is, but I was expecting to be suggested using Vorlage:NowCommons, because this is what the Wikidata item has in store for de.wikipedia.org

The system not work across wikifarm's ( in that example production vs beta ). We need a valid site id for the source wiki to get the fitting source template from Wikidata. The site id is retrieved from the local sites table that is wikifarm specific. - Since the source is not part of the local sites table in your case, the id can't be retrieved and the template lookup fails. Therefor you get the default, fallback message. :-)

I used the test beta wiki, moving this file from de.wikipedia.org there: https://test.wikimedia.beta.wmflabs.org/wiki/File:Rajamangala_University_of_Technology_Rattanakosin_Salaya_Campus_Stadium.jpg
What I received was the message if we do not know what the template is, but I was expecting to be suggested using Vorlage:NowCommons, because this is what the Wikidata item has in store for de.wikipedia.org

:D This verifies that the fallback is working.

I created T225609: Supplement the FileImporter environment on beta to allow full testing as follow up, to allow the testing of the Wikidata template retrieval on the beta clusters.

Change 514715 merged by jenkins-bot:
[operations/mediawiki-config@master] New configuration to pull sitelinks from Wikidata

https://gerrit.wikimedia.org/r/514715

Mentioned in SAL (#wikimedia-operations) [2019-06-17T11:11:59Z] <awight@deploy1001> Synchronized wmf-config/CommonSettings.php: wmf-config/CommonSettings-labs.php SWAT: [[gerrit:514715|FileImporter configuration to fetch sitelinks from Wikidata (T225609 T224007)]] (duration: 00m 47s)

Mentioned in SAL (#wikimedia-operations) [2019-06-17T12:13:11Z] <awight@deploy1001> Synchronized wmf-config/CommonSettings-labs.php: SWAT: [[gerrit:514715|FileImporter configuration to fetch sitelinks from Wikidata (T225609 T224007)]] - finishing partial deployment (duration: 00m 47s)

Confirmed not working as intended on testwiki. The workflow isn't broken, but we see the generic text rather than Vorlage:NowCommons.

The feature is working on the beta cluster, FWIW

Our code wasn't deployed yet. This will deploy to testwiki with the train on Tuesday, 18 June, and to commonswiki on Wednesday, 19 June.

Working on testwiki.

awight updated the task description. (Show Details)Jun 19 2019, 2:02 PM

I think this is working great. There are just a few things I cannot test, so I would highly appreciate if you could just tell me my assumptions are right :)
On beta commons I see this:

  • The target wiki is called "Commons", which is not what I expected (I expected Beta Commons or so). But this is just the way the wiki is set up, and if the target wiki was called differently, we would see another name, right?
  • Same goes for "NowCommons", doesn't it? The template name for en beta is just coincidentally the same as the one we have e.g. on en wiki?
  • The target wiki is called "Commons", which is not what I expected (I expected Beta Commons or so). But this is just the way the wiki is set up, and if the target wiki was called differently, we would see another name, right?

Interesting--but yeah it's just an artifact of configuration. We should probably file a task to fix that in the future, but it comes from {{SITENAME}}.

  • Same goes for "NowCommons", doesn't it? The template name for en beta is just coincidentally the same as the one we have e.g. on en wiki?

In this case, the coincidence is my fault. It's just because I copied existing templates over and kept the name. Note that you can also test on test.wikipedia.org, which will let you import from a wiki where the template name is more exciting.

Lea_WMDE closed this task as Resolved.Jun 25 2019, 2:26 PM

thanks @awight!