Page MenuHomePhabricator
Create Task
Maniphest T111618

Make use of the project field in categorisation jobs
Closed, ResolvedPublic
Actions

Description

In monuments_all table, we do not keep the origin wiki project of the monument.

The ErfgoedBot categorization bot tries to match an image to Commons categories, using several methods.
One follows the monument_article to look for {{CommonsCat}}-like template on it. But this assumes that the monument_article lives on a Wikipedia project. As we have lists from other projects (like WikiVoyage), that the harvesting job supports, then we cannot categorise these files.

Update: There is now a project field in monuments_all. What is needed is for the categorisation jobs to make use of this new data.

Details

SubjectRepoBranchLines +/-
Make categorisation project awarelabs/tools/heritagemaster+141 -72
Customize query in gerrit

Related Objects

Event Timeline

JeanFred created this task.Sep 5 2015, 11:29 PM
JeanFred raised the priority of this task from to High.
JeanFred updated the task description. (Show Details)
JeanFred added a project: Wiki-Loves-Monuments-Database.
JeanFred subscribed.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 5 2015, 11:29 PM
Lokal_Profil merged a task: T131681: Add project field to database tables.Apr 3 2016, 10:14 PM
Lokal_Profil subscribed.

Plan of attack:

Add project field to country databases and monuments_all.

Once done this should also be used when converting wikitext links to urls (api/CommonFunctions.php.processWikitext())

Lokal_Profil added a comment.May 23 2016, 9:37 AM

The project field has now been added to monuments_all.

Was there a separate task for implementing this in the categorization? If not more details should be added to this task to identify the next step.

Lokal_Profil added a comment.May 23 2016, 9:43 AM

categorize_images.py seem to include some unused bits (specifically getArticle() ) but from a quick look I would say:

  • commonscatTemplates (and associated calling functions) needs to be made project aware.
  • getMonData() needs to get project from the SQL query.
  • get_new_categories() needs to make use of project delivered through monData .
Lokal_Profil renamed this task from The monuments_all table does not keep track of the origin wiki, hindering the categorisation to Make use of the project field in categorisation jobs.Jul 15 2016, 8:18 AM
Lokal_Profil updated the task description. (Show Details)
gerritbot subscribed.Jul 16 2016, 12:05 AM

Change 299273 had a related patch set uploaded (by Lokal Profil):
Make categorisation project aware

https://gerrit.wikimedia.org/r/299273

gerritbot added a project: Patch-For-Review.Jul 16 2016, 12:05 AM
gerritbot added a comment.Jul 27 2016, 10:14 AM

Change 299273 merged by jenkins-bot:
Make categorisation project aware

https://gerrit.wikimedia.org/r/299273

Lokal_Profil claimed this task.Jul 27 2016, 10:16 AM
Stashbot subscribed.Aug 1 2016, 7:54 AM

Mentioned in SAL [2016-08-01T07:54:26Z] <Lokal_Profil> (correction to last line) Deployed latest from Git, 5fe42fe (T111618), 1ec3530, 9a630b5 (T139258)

Stashbot mentioned this in T139258: Figure out improved matching of monuments for Iran.Aug 1 2016, 7:54 AM
Lokal_Profil added a comment.Aug 1 2016, 7:56 AM

@JeanFred Do we have a good test case to see if this is working? I would expect something like number of uncategorised images on ru_(ru)?

Lokal_Profil closed this task as Resolved.Aug 8 2016, 12:17 PM

I'm going to assume that this works as expected until I hear something suggesting otherwise.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL