Page MenuHomePhabricator

Expand Commons:Monuments database/Statistics to also include empty sources
Closed, ResolvedPublic

Description

On https://commons.wikimedia.org/wiki/Commons:Monuments_database/Statistics we have a report of all the sources we index. Only sources for which we find records show up here. Things like T203348 are very hard to detect right now. Shouldn't be too hard to also go over https://github.com/wikimedia/labs-tools-heritage/tree/master/erfgoedbot/monuments_config and add these as empty sources and maybe overwrite it with the real data based on the database query. That way we always include all configured sources in the report.

Current code is at https://github.com/wikimedia/labs-tools-heritage/blob/master/erfgoedbot/database_statistics.py

Event Timeline

Change 459783 had a related patch set uploaded (by Jean-Frédéric; owner: Jean-Frédéric):
[labs/tools/heritage@master] Include empty sources in Monuments database statistics

https://gerrit.wikimedia.org/r/459783

Change 459783 merged by jenkins-bot:
[labs/tools/heritage@master] Include empty sources in Monuments database statistics

https://gerrit.wikimedia.org/r/459783

Mentioned in SAL (#wikimedia-cloud) [2018-09-20T19:22:20Z] <JeanFred> Deploy latest from Git master: a14a46d (T203349)

Lokal_Profil added a comment.EditedSep 21 2018, 7:11 PM

We forgot to check the -skip flag. Should remove e.g. nl-wd_nl.

Lokal_Profil added a comment.EditedSep 22 2018, 2:08 PM

We forgot to check the -skip flag. Should remove e.g. nl-wd_nl.

And to implement the -skip_wd flag which we have enabled for the harvest (explains au_en).

And skip the wlpa ones (is it time to drop support for these?)

Change 462809 had a related patch set uploaded (by Lokal Profil; owner: Lokal Profil):
[labs/tools/heritage@master] Apply same filters to database statistics as to update monuments

https://gerrit.wikimedia.org/r/462809

Change 462809 merged by jenkins-bot:
[labs/tools/heritage@master] Apply same filters to database statistics as to update monuments

https://gerrit.wikimedia.org/r/462809

Change 462861 had a related patch set uploaded (by Lokal Profil; owner: Lokal Profil):
[labs/tools/heritage@master] Exclude wlpa from database statistics

https://gerrit.wikimedia.org/r/462861

Change 462861 merged by jenkins-bot:
[labs/tools/heritage@master] Exclude wlpa from database statistics

https://gerrit.wikimedia.org/r/462861

Mentioned in SAL (#wikimedia-cloud) [2018-10-02T21:02:19Z] <Lokal_Profil> Deploy latest from Git master: 33ab2cc, 7e7533f, 07bf55b, 63530f4, ae38b67, 5747927 (T203349), 72822ee (T204581), 35e9b82 (T204580), 0b2466a (T176724)

Effeietsanders closed this task as Resolved.Nov 18 2018, 1:48 AM
Effeietsanders claimed this task.