Page MenuHomePhabricator
Create Task
Maniphest T140488

Add Albanian Wikipedia lists to database
Closed, ResolvedPublic
Actions

Description

We got a request to add the structured list on Albanian Wikipedia ( Albania and Kosovo ) to the database

We need to confirm those are the lists and get the other details listed at https://commons.wikimedia.org/wiki/Commons:Monuments_database/Harvesting

Mappings:

Albania in Albanian

project : wikipedia
lang : sq
headerTemplate : Kreu Monumentet në Shqipëri
rowTemplate : Rreshti Monumentet në Shqipëri
commonsTemplate : Cultural Heritage Albania
commonsTrackerCategory : Cultural heritage monuments in Albania with known IDs
commonsCategoryBase : Cultural heritage monuments in Albania
autoGeocode : False
unusedImagesPage : Wikipedia:WikiProjekti Vende Historike/Shqipëri/Foto të papërdorura
imagesWithoutIdPage : Wikipedia:WikiProjekti Vende Historike/Shqipëri/Foto pa nr. unik
registrantUrlBase:
namespaces : [0]
table : monuments_al_(sq)
truncate : False
primkey : idno <i.e. Nr>
fields (source : dest):

  • Emërtimi : name
  • Nr : idno
  • Lokacioni : place
  • Bashkia : municipality
  • lat : lat
  • lon : lon
  • Imazh : image

Kosovo in Albanian

project : wikipedia
lang : sq
headerTemplate : Kreu Monumentet në Kosovë
rowTemplate : Rreshti Monumentet në Kosovë
commonsTemplate : Cultural Heritage Kosovo
commonsTrackerCategory : Cultural heritage monuments in Kosovo with known IDs
commonsCategoryBase : Cultural heritage monuments in Kosovo
autoGeocode : False
unusedImagesPage : Wikipedia:WikiProjekti Vende Historike/Kosovë/Foto të papërdorura
imagesWithoutIdPage : Wikipedia:WikiProjekti Vende Historike/Kosovë/Foto pa nr. unik
registrantUrlBase:
namespaces : [0]
table : monuments_xk_(sq)
truncate : False
primkey : idno <i.e. Nr_Unik>
fields (source : dest):

  • Emërtimi : name
  • Periudhat_Datimi : date_period
  • Vendi: place
  • Komuna: municipality
  • Statusi_Nr_Vendimit : status
  • Kategoria : category
  • Nenkategoria : subcategory
  • lat : lat
  • lon : lon
  • Nr_Unik : idno
  • Imazh : image

Details

SubjectRepoBranchLines +/-
Add Albania and Kosovo to the databaselabs/tools/heritagemaster+188 -0
Customize query in gerrit

Related Objects

Event Timeline

Lokal_Profil created this task.Jul 15 2016, 6:07 PM
Restricted Application added subscribers: Zppix, JeanFred, Aklapper. · View Herald TranscriptJul 15 2016, 6:07 PM
Lokal_Profil renamed this task from Add Albanian.Wikipedia lists to database to Add Albanian Wikipedia lists to database.Jul 15 2016, 6:07 PM
Lokal_Profil updated the task description. (Show Details)Jul 18 2016, 8:56 AM
Lokal_Profil added a subscriber: Liridon.Jul 18 2016, 8:59 AM

Confirming that those are the lists and adding basic config layout.

@Liridon: Could you fill in the blanks? The definitions can be found at Commons:Monuments_database/Harvesting#Configure a new source.

• Mattias_Ostmar-WMSE subscribed.Jul 18 2016, 9:02 AM
JeanFred awarded a token.Jul 18 2016, 9:07 AM
Arianit subscribed.Jul 18 2016, 9:28 AM
Liridon updated the task description. (Show Details)Jul 18 2016, 9:47 AM
Liridon added subscribers: Margott, Olsi.
Liridon added a comment.Jul 18 2016, 9:58 AM

@Lokal_Profil Thanks a lot.

I updated some blank fields, not all yet. Also, with @Arianit & @Margott, we will improve and update information on monuments lists in Albanian Wikipedia, before you launch the bot to harvest data.

JeanFred added a comment.Jul 18 2016, 10:00 AM
In T140488#2470041, @Liridon wrote:

I updated some blank fields, not all yet.

Thanks!

Also, with @Arianit & @Margott, we will improve and update information on monuments lists in Albanian Wikipedia, before you launch the bot to harvest data.

Don’t worry too much about it − the bot re-harvests all the data from scratch every day − all your improvements will make it in no time :)

Lokal_Profil mentioned this in T140343: Setup empty mapping table page on Wikidata for Albania and F.Y.R. Macedonia .Jul 18 2016, 1:36 PM
Lokal_Profil updated the task description. (Show Details)Jul 19 2016, 1:27 PM
Lokal_Profil added a comment.Jul 19 2016, 1:30 PM
In T140488#2470041, @Liridon wrote:

@Lokal_Profil Thanks a lot.

I updated some blank fields, not all yet. Also, with @Arianit & @Margott, we will improve and update information on monuments lists in Albanian Wikipedia, before you launch the bot to harvest data.

Looks good.

I added the template parameters and mapped the obvious ones. Please check that I didn't do that wrong. I also highlighted the missing ones to make them easier to spot.

Lokal_Profil updated the task description. (Show Details)Jul 19 2016, 1:32 PM

I also added commonsCategoryBase based on the campaign pages

Liridon updated the task description. (Show Details)Jul 19 2016, 7:12 PM
Liridon added a comment.Jul 19 2016, 7:22 PM

@Lokal_Profil, just completed highlighted fields. For some fields in Template of Kosovo I added the description, but don't know if the database has appropriate fields for those, either way, those fields are less important.

Lokal_Profil updated the task description. (Show Details)Jul 21 2016, 8:21 AM
Lokal_Profil added a comment.Jul 21 2016, 8:23 AM
In T140488#2476786, @Liridon wrote:

@Lokal_Profil, just completed highlighted fields. For some fields in Template of Kosovo I added the description, but don't know if the database has appropriate fields for those, either way, those fields are less important.

Thanks!
I spotted one more field (for each) which I had missed. registrantUrlBase is a url which (together with the unique id) could be used to get information about the monument from the government. If there isn't any then that is ok, we just need to know.

Lokal_Profil added a comment.Jul 21 2016, 8:54 AM

@Liridon:

  1. For the Kosovo list is there a way of determining when Vendi_Komuna is a municipality and when it isn't?
  2. I've made the assumption that any wiki links in Emërtimi will go to an article about that monument. Let me know if this assumption is wrong.

@JeanFred: Are we happy even if we have no adm structure below adm0?

gerritbot subscribed.Jul 21 2016, 8:59 AM

Change 300233 had a related patch set uploaded (by Lokal Profil):
[Don't merge] Add Albania and Kosovo to the database

https://gerrit.wikimedia.org/r/300233

gerritbot added a project: Patch-For-Review.Jul 21 2016, 8:59 AM
Lokal_Profil claimed this task.Jul 21 2016, 8:59 AM
Liridon added a comment.Jul 21 2016, 10:40 AM
  1. For the Kosovo list is there a way of determining when Vendi_Komuna is a municipality and when it isn't?

'Vendi' is Location (often a village) and 'Komuna' is Municipality (also name of the city), some monuments have the value for Location some don't, but all of them have Municipality. If necessary, I will split field 'Vendi_Komuna' in two fields, "Vendi" and "Komuna".

  1. I've made the assumption that any wiki links in Emërtimi will go to an article about that monument. Let me know if this assumption is wrong.

Yes, if there is an article about that monument, it will have the name of 'Emërtimi' or a redirection to another name about that momument.

Liridon added a comment.Jul 21 2016, 10:44 AM

I spotted one more field (for each) which I had missed. registrantUrlBase is a url which (together with the unique id) could be used to get information about the monument from the government. If there isn't any then that is ok, we just need to know.

No.

Lokal_Profil added a comment.Jul 21 2016, 2:55 PM
In T140488#2483152, @Liridon wrote:
  1. For the Kosovo list is there a way of determining when Vendi_Komuna is a municipality and when it isn't?

'Vendi' is Location (often a village) and 'Komuna' is Municipality (also name of the city), some monuments have the value for Location some don't, but all of them have Municipality. If necessary, I will split field 'Vendi_Komuna' in two fields, "Vendi" and "Komuna".

I think that could be useful.

Liridon updated the task description. (Show Details)Jul 21 2016, 3:33 PM
Liridon added a comment.Jul 21 2016, 7:06 PM
In T140488#2483495, @Lokal_Profil wrote:
In T140488#2483152, @Liridon wrote:
  1. For the Kosovo list is there a way of determining when Vendi_Komuna is a municipality and when it isn't?

'Vendi' is Location (often a village) and 'Komuna' is Municipality (also name of the city), some monuments have the value for Location some don't, but all of them have Municipality. If necessary, I will split field 'Vendi_Komuna' in two fields, "Vendi" and "Komuna".

I think that could be useful.

It's done.

Lokal_Profil added a comment.Jul 26 2016, 8:24 AM
In T140488#2484825, @Liridon wrote:
In T140488#2483495, @Lokal_Profil wrote:
In T140488#2483152, @Liridon wrote:
  1. For the Kosovo list is there a way of determining when Vendi_Komuna is a municipality and when it isn't?

'Vendi' is Location (often a village) and 'Komuna' is Municipality (also name of the city), some monuments have the value for Location some don't, but all of them have Municipality. If necessary, I will split field 'Vendi_Komuna' in two fields, "Vendi" and "Komuna".

I think that could be useful.

It's done.

Thanks!

I've prepared a patch which should start harvesting these to the monuments database.

gerritbot added a comment.Jul 26 2016, 8:42 AM

Change 300233 merged by jenkins-bot:
Add Albania and Kosovo to the database

https://gerrit.wikimedia.org/r/300233

Stashbot subscribed.Jul 26 2016, 1:42 PM

Mentioned in SAL [2016-07-26T13:42:48Z] <JeanFred> Deployed latest from Git: 0c848e6 (T140488)

Liridon added a comment.Jul 26 2016, 6:14 PM

Thanks!

I've prepared a patch which should start harvesting these to the monuments database.

Thank you @Lokal_Profil :)

JeanFred added a comment.Jul 27 2016, 7:27 AM

Harvesting is now done!

  • Kosovo: 1411 monuments − list
  • Albania: 1376 monuments − list
Lokal_Profil closed this task as Resolved.Jul 27 2016, 7:51 AM

Thanks @Liridon and @JeanFred !

JeanFred mentioned this in T141505: ErfogedBot categorisation of Kosovo pictures is wrong.Jul 28 2016, 9:38 AM
JeanFred added a comment.Jul 28 2016, 12:30 PM

Links for Statistics:

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits