Page MenuHomePhabricator

Fully terminate use of Wikidata item descriptions as short article descriptions for English Wikipedia articles
Open, Needs TriagePublic

Description

NOTE: Please avoid adding "+1"/"me too" comments. If you plan to comment, please first read the previous conversation to make sure not to repeat input - thanks!

Per T187285 the magic word {{SHORTDESC:}} was created to replace the (mis)use of wikidata item-descriptions as short article-descriptions for English Wikipedia. The transition was to be completed once EnWiki created local descriptions for two million articles. That milestone has been reached.

Wikidata item-descriptions should no longer be pulled or displayed as article-descriptions for English Wikipedia articles, anywhere. This includes mobile article views, anywhere in any APP, on-wiki search result descriptions, wikipedia.org search result descriptions, VisualEditor add-link tool search results, Visual Editor edit-tool article-description, any other usage inside Visual Editor, Related Articles descriptions, and any other location I may have missed.

Edit: I found a list of locations, which I copy-paste below. I do not know if the list is complete.

  • Apps under the title
  • Visual editor (desktop and mobile)
  • Portal search (www.wikipedia.org)
  • Mobile web and mobile app search
  • Related pages (bottom of mobile web and app pages)
  • iOS widgets
  • App maps feature
  • Apps feed feature

A filter was also created to block descriptions consisting of spaces/punctuation/non-breaking-spaces/?other? That filter should be removed as obsolete code complexity / code cruft.

Event Timeline

Alsee created this task.Mar 25 2020, 10:58 AM
Restricted Application added subscribers: Liuxinyu970226, Aklapper. · View Herald TranscriptMar 25 2020, 10:58 AM
Alsee updated the task description. (Show Details)Mar 25 2020, 11:05 AM
Aklapper renamed this task from Fully terminate use of Wikidata ITEM-DESCRIPTIONS as short ARTICLE-DESCRIPTIONS for English Wikipedia articles to Fully terminate use of Wikidata item descriptions as short article descriptions for English Wikipedia articles.Mar 25 2020, 3:01 PM
Aklapper removed a project: Community-Tech.
JJMC89 added a subscriber: JJMC89.Mar 25 2020, 4:28 PM

I really hope this never happens, and that the locally-defined descriptions are merged back to Wikidata. There are 2.8+ million infoboxes on Commons that are using the English descriptions from Wikidata, and there is currently no way that they can access the enwp descriptions.

Alsee added a comment.Mar 27 2020, 7:38 AM

I really hope this never happens

There was repeated consensus on this, and if you recall you participated in one. The Foundation handled things badly, but committed to terminating wikidata descriptions when we reached 2 million local descriptions. I expect the result would be Bad all around if the Foundation were to renege.

There are 2.8+ million infoboxes on Commons that are using the English descriptions from Wikidata, and there is currently no way that they can access the enwp descriptions.

Should we open a different task for English article-descriptions to replace the English field on wikidata?

There are 2.8+ million infoboxes on Commons that are using the English descriptions from Wikidata, and there is currently no way that they can access the enwp descriptions.

Should we open a different task for English article-descriptions to replace the English field on wikidata?

I'm sorry but this is not gonna happen.

I really hope this never happens

There was repeated consensus on this, and if you recall you participated in one. The Foundation handled things badly, but committed to terminating wikidata descriptions when we reached 2 million local descriptions. I expect the result would be Bad all around if the Foundation were to renege.

Yes, my comment there still seems to hold true. "I personally think it is a bad idea and a waste of developer time." Add to that the massive waste of editor time to add those local descriptions, which can't even be copied back to Wikidata due to copyright restrictions. This whole situation sucks.

Pppery added a subscriber: Pppery.EditedMar 29 2020, 3:45 AM

I note that Category:Articles with short description only has 1.8 million entries. Why the difference?

Alsee updated the task description. (Show Details)Apr 16 2020, 10:14 AM
QEDK added a subscriber: QEDK.May 15 2020, 1:43 PM

I note that Category:Articles with short description only has 1.8 million entries. Why the difference?

Category numbers are terrible for determining anything (cache voodoo?). Elasticsearch indexing is more accurate and can be used for this purpose.

Alsee added a comment.May 15 2020, 2:18 PM

The current mainspace search count of about 2,250,000 matches up well with the category count when you include non-article mainspace pages, primarily disambiguation.

Category:Pages_with_short_description currently reports approximately 1,939,000 articles, approximately 312,000 disambiguation pages, plus some thousands of pages of other types in and out of mainspace.

RexxS added a subscriber: RexxS.May 15 2020, 11:48 PM

I really hope this never happens

There was repeated consensus on this, and if you recall you participated in one. The Foundation handled things badly, but committed to terminating wikidata descriptions when we reached 2 million local descriptions. I expect the result would be Bad all around if the Foundation were to renege.

Yes, my comment there still seems to hold true. "I personally think it is a bad idea and a waste of developer time." Add to that the massive waste of editor time to add those local descriptions, which can't even be copied back to Wikidata due to copyright restrictions. This whole situation sucks.

The disadvantages of using a database field that isn't tracked and is not capable of being sourced to verify its contents far outweigh the value of being able to pull an English-only value into other projects. Those arguments are well-rehearsed and the consensus is clear.

I seriously doubt that there are any genuine copyright barriers to importing enwiki short descriptions into the Wikidata description fields for two reasons:
(1) short descriptions are almost certainly too short to be copyrightable (see https://www.bitlaw.com/copyright/unprotected.html for example);
(2) we already import the English short description into Wikidata when we use short description helper to create a short description where Wikidata has no English description.

The task T191531 asks for local short descriptions to be exposed via a Scribunto library call, and is still open. The simplest work-around is to use the Wikidata description field to hold each wiki's local description where it exists. There's no mechanism for that to happen automatically as far as I can see, but having a bot crawling for changes to the local short description and updating Wikidata would be nothing unusual and timely enough for our applications on Commons, etc.

WMF staff, please hold up your end of our agreement by disabling the Wikidata fallback for short descriptions at en.WP.

See this discussion on en.WP, which summarizes the current request.

Aklapper updated the task description. (Show Details)May 16 2020, 6:08 PM
Tgr added a subscriber: Tgr.May 18 2020, 11:42 PM
Certes added a subscriber: Certes.Jun 5 2020, 8:56 PM
kaldari added a subscriber: kaldari.Jun 5 2020, 9:12 PM

Since the only wikis using $wgWikibaseAllowLocalShortDesc are English Wikipedia and Test Wikipedia, I think we should just repurpose that config variable rather than creating a new one. To implement this change, we'll probably need to do some refactoring in client/includes/Store/DescriptionLookup.php and client/includes/Api/Description.php and update all the associated tests (within the Wikibase extension).

DannyH added a subscriber: DannyH.Mon, Jun 22, 9:54 PM

Hi everyone, I posted a response on the English WP Village Pump (WMF) page, saying that we are going to be working on this ticket. There are still a few things to work out, so there's no firm date. You can follow along by watching this ticket.

My post is here: https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(WMF)#Update%3A_Scheduled_shutdown_of_Wikidata_descriptions_on_EnWiki

If you've got any thoughts or questions, please post them on that VP page, and do not post them on this ticket, even if it's a positive "yay, glad to see this" comment. This ticket is a workspace for the developers who are going to be working on making this happen, so please don't distract them from doing the thing that you want them to do. :) Thanks, talk to you on the VP.

Excuse me, I understand correctly that the devs, instead of including a changes in the Wikidata element's description to the watchlist and recent changes, agreed to such a strange decision?

@Iniquity - There are other issues as well. For example, the Wikidata community's insistence that Wikidata descriptions are primarily for disambiguation on Wikidata and not intended for other use cases. This is why we have Wikidata descriptions like:

one of two or more individuals having at least one parent in common. Avoid using with "relative" (P1038), use "sibling" (P3373) instead

... which are unusable anywhere besides Wikidata. Hopefully, this will one day be alleviated via T97566, however.

There are other issues as well. For example, the Wikidata community's insistence that Wikidata descriptions are primarily for disambiguation on Wikidata and not intended for other use cases. This is why we have Wikidata descriptions like

@kaldari thanks for your answer! In this case, I don’t really imagine why these descriptions were included in all projects on mobile devices. But this, again, is not a reason to make crutches only for the English Wikipedia.