Page MenuHomePhabricator

Sunset /page/definitions?
Closed, DeclinedPublic

Description

The MCS /page/definitions endpoint supports Wiktionary definition lookups from an ActionMode button within wiki pages in the Android app. It is only available for English because of the substantial amount of non-technical / coordination work that would be required to expand it to more languages; the work necessary to make this happen is not on the agenda for the foreseeable future. The current state of affairs is justified by the notion that the endpoint is "experimental," but it hasn't been meaningfully updated since ~early 2016.

I am currently in the process of extracting MCS bits from PCS, and I believe we should consider deleting this endpoint. Before doing that, we need to evaluate whether the current in-app usage justifies the computing and engineering resources needed to maintain the endpoint in production.

@Dbrant Can you weigh in here with usage numbers and other thoughts?

Event Timeline

So it looks like we have an average of 3800 clicks per day on the "Define" option, over the last month:

image.png (286×709 px, 42 KB)

I would say that's slightly more than negligible. Even though we only offer it in English, we should contemplate carefully how to proceed (cc @Charlotte).
If the intention is to deprecate this endpoint, then our options would be as follows:

  • Remove the function from the app, if that is the product decision.
  • Move the parsing logic into the app, while continuing to read the content from Wiktionary. This would make it rather brittle -- it won't be resilient against structural changes of Wiktionary pages. (although I don't think we've needed to do much maintenance of the existing endpoint for these reasons?)
  • Bounce the user to the mobile web version of the Wiktionary item being defined. This would make it jarring for the user.
  • Integrate with some other dictionary API

Thanks for checking those numbers, @Dbrant. That's more clicks than I would have guessed.

I guess there is not an urgent need to sunset this endpoint; it has been standing up on its own all this time with essentially no maintenance effort. Of course that could change tomorrow, but it probably won't.

At a quick glance I don't see any promising alternative definition APIs. Perhaps it's best to decline this, then, and just keep in mind that we would be eager to sunset this in the event that a suitable alternative appears. But I'll leave this open for @Charlotte to chime in before closing.

Even if we removed the functionality in the app we would have to keep the endpoint running for a bit for older versions of the Android app.
This task might also be a reminder to look into definition alternatives, i.e. if the Android OS provides one.

The Lexeme work would eventually be able to replace this (and in all languages), but it's not there yet.

action=wbsearchentities &type=lexeme &language=en &search=first; the search response is:

"search": [
    {
        "repository": "",
        "id": "L2",
        "concepturi": "http://www.wikidata.org/entity/L2",
        "title": "Lexeme:L2",
        "pageid": 54386964,
        "url": "//www.wikidata.org/wiki/Lexeme:L2",
        "label": "first",
        "description": "English, noun",
        "match": {
            "type": "label",
            "language": "en",
            "text": "first"
        }
    },
    {
        "repository": "",
        "id": "L14410",
        "concepturi": "http://www.wikidata.org/entity/L14410",
        "title": "Lexeme:L14410",
        "pageid": 56141960,
        "url": "//www.wikidata.org/wiki/Lexeme:L14410",
        "label": "firstly",
        "description": "English, adverb",
        "match": {
            "type": "label",
            "language": "en",
            "text": "firstly"
        }
    },
    {
        "repository": "",
        "id": "L34200",
        "concepturi": "http://www.wikidata.org/entity/L34200",
        "title": "Lexeme:L34200",
        "pageid": 57898177,
        "url": "//www.wikidata.org/wiki/Lexeme:L34200",
        "label": "firsthand",
        "description": "English, adjective",
        "match": {
            "type": "label",
            "language": "en",
            "text": "firsthand"
        }
    }
],

Then sling the resultant id into action=wbgetentities &ids=L2; the senses response is:

"senses": [
	{
		"id": "L2-S1",
		"glosses": {
			"en": {
				"language": "en",
				"value": "Element in an ordered list which comes before all others according to the ordering"
			},
			"de": {
				"language": "de",
				"value": "einer Ordnung folgend das Element vor allen anderen"
			},
			"es": {
				"language": "es",
				"value": "Elemento que se ubica antes que todos los dem\u00e1s en una lista ordenada."
			},
			"eu": {
				"language": "eu",
				"value": "Ordenatutako zerrenda batean besteen aurretik dagoen elementua"
			},
			"te": {
				"language": "te",
				"value": "\u0c2e\u0c4a\u0c26\u0c1f\u0c3f "
			},
			"pt-br": {
				"language": "pt-br",
				"value": "Elemento em uma lista ordenada que precede todos os demais conforme o ordenamento"
			}
		},

But though "Element in an ordered list which comes before all others according to the ordering" is helpful and correct, (a) two different API calls for this is heavy, (b) there's little filtering available to slim the request down anyway, and most importantly (c) there's little actual content for senses yet, even in English.

The lexeme work sounds promising. (That would be a big enough change to warrant at least a major version bump if not a new endpoint.)

(a) two different API calls for this is heavy,

The definition endpoint could do that for external clients.

(b) there's little filtering available to slim the request down anyway,

The definition endpoint could do that for external clients. We could make the response language specific, triggered by the Accept-Language header.

(c) there's little actual content for senses yet, even in English.

That's probably a matter of time, I hope.

Thanks for your comments on this, gents. I will decline for now, with the understanding that when the lexeme work is suitably advanced, we will revisit.

Change 530259 had a related patch set uploaded (by Mholloway; owner: Michael Holloway):
[mediawiki/services/mobileapps@master] Definitions: Update to use WikibaseLexeme

https://gerrit.wikimedia.org/r/530259