Page MenuHomePhabricator

wbeditentity response does not contain lemma data
Closed, ResolvedPublic8 Estimated Story Points

Description

Problem:
Given a request to wbeditentity to change or add a lemma of a Lexeme the response JSON does not contain the lemmas.

Example:
Sending a wbeditentity request for L42 with the data
{"lemmas":{"en":{"language":"en","value":"answer"}}} (Example sandbox link on Beta Wikidata) results in the following response:

{
    "entity": {
        "claims": {},
        "id": "L31",
        "type": "lexeme",
        "lastrevid": 275,
        "nochange": ""
    },
    "success": 1
}

Expected response:

{
    "entity": {
        "id": "L31",
        "type": "lexeme",
        "lemmas": {
            "en": {
                "language": "en",
                "value": "answer"
            },
            "de": {
                "language": "de",
                "value": "antwort"
            },
            "es": {
                "language": "es",
                "value": "respuesto"
            }
        },
        "claims": {},
        "lastrevid": 275,
        "nochange": ""
    },
    "success": 1
}

This prevents the lemma editing UI from correctly updating its fields upon saving e.g. to reflect trimmed whitespace (T250550). This is possibly similar to T200255.
Acceptance criteria:

  • When using wbeditentity to change/add a lemma then the response JSON contains the lemma.

Event Timeline

Lydia_Pintscher added a subscriber: Lydia_Pintscher.

@Jakob_WMDE Can you help with an example of the expected response in the description?

darthmon_wmde set the point value for this task to 8.Jan 20 2021, 1:09 PM

Task inspection note: it might be possible to get the Lemmas by build the EditEntity result the way GetEntities does.
See https://github.com/wikimedia/Wikibase/blob/fa50e8941ca62e3c798fd6b4a949c2fa43b9f3cd/repo/includes/Api/GetEntities.php#L346 for more info.

This will probably be a breaking change for wbeditentity requests to MediaInfo entities: currently, the response has statements under "claims", this will change that to "statements" (like in wbgetentities and Special:EntityData) – compare T149410. What’s our strategy for WikibaseMediaInfo breaking changes? Is Structured Data on Commons covered by our Stable Interface Policy?

Change 658601 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[mediawiki/core@master] Allow replacing ApiResult value with same value

https://gerrit.wikimedia.org/r/658601

Change 658602 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/Wikibase@master] Make wbeditentity return full entity data

https://gerrit.wikimedia.org/r/658602

Change 658603 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/WikibaseLexeme@master] Test that wbeditentity returns full entity data

https://gerrit.wikimedia.org/r/658603

Change 658645 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/WikibaseLexeme@master] Don’t try to read lemmas from wbeditentity response

https://gerrit.wikimedia.org/r/658645

Change 658601 merged by jenkins-bot:
[mediawiki/core@master] Allow replacing ApiResult value with same value

https://gerrit.wikimedia.org/r/658601

This will probably be a breaking change for wbeditentity requests to MediaInfo entities: currently, the response has statements under "claims", this will change that to "statements" (like in wbgetentities and Special:EntityData) – compare T149410. What’s our strategy for WikibaseMediaInfo breaking changes? Is Structured Data on Commons covered by our Stable Interface Policy?

We discussed this topic but IIRC concluded that MediaInfo isn't really covered.
@CBogen @MarkTraceur What are your thoughts?

@Keegan and @matthiasmullie, do you have a sense of what this would break on our end and/or for the community? and @Keegan do you know of any policies that this change would fall under?

We could, for a limited time, make wbeditentity return both "statements" and "claims", so that people could migrate their tools more easily.

@Keegan and @matthiasmullie, do you have a sense of what this would break on our end and/or for the community? and @Keegan do you know of any policies that this change would fall under?

The MediaInfo UI doesn't use wbeditentity (except for some test case), but it could break community tools/bots. I suspect it wouldn't be too hard to reach out to the relevant community members to get their tools updated, if per @Lucas_Werkmeister_WMDE's suggestion, both variants are included in the response for some time.

We could, for a limited time, make wbeditentity return both "statements" and "claims", so that people could migrate their tools more easily.

It turns out that the "statements" and "claims" will not contain exactly the same data (unless we add more code to make them match) – "statements" will be missing the "datatype" in each snak (compare T246809).

Alright, the latest patch set adds "claims" back. (And does not include any changes to add a "datatype" to the "statements" part.)

@Keegan and @matthiasmullie, do you have a sense of what this would break on our end and/or for the community? and @Keegan do you know of any policies that this change would fall under?

I'm not aware of a specific policy relating to this change, but my familiarity with Wikidata itself is pretty limited. I'll check around. In the meantime we can at least post a notice about this change to a mailing list or wikipage to give people notice. @Lydia_Pintscher @Lucas_Werkmeister_WMDE would Wikitech or Wikidata be the better mailing list for this? Or both?

The announcement is ready (OKed by WMF and WMDE people), I’ll send it on Monday (Feb 8th) stating that the significant change will be deployed on Feb 10th and the breaking change on March 3rd. To make sure those dates are met even when there are train problems, I suggest we backport the Wikibase change on those days, so it’s available on both the old and the new train. (The backport would be before the new train reaches group1 anyways, so that any potential problems it causes are noticed early.)

Change 658645 merged by jenkins-bot:
[mediawiki/extensions/WikibaseLexeme@master] Don’t try to read lemmas from wbeditentity response

https://gerrit.wikimedia.org/r/658645

Change 658602 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Make wbeditentity return full entity data

https://gerrit.wikimedia.org/r/658602

Change 658603 merged by jenkins-bot:
[mediawiki/extensions/WikibaseLexeme@master] Test that wbeditentity returns full entity data

https://gerrit.wikimedia.org/r/658603

The announcement is ready (OKed by WMF and WMDE people), I’ll send it on Monday (Feb 8th) stating that the significant change will be deployed on Feb 10th and the breaking change on March 3rd. To make sure those dates are met even when there are train problems, I suggest we backport the Wikibase change on those days, so it’s available on both the old and the new train. (The backport would be before the new train reaches group1 anyways, so that any potential problems it causes are noticed early.)

Announcement sent.

@Lucas_Werkmeister_WMDE do we have tickets for the various future steps that will be needed (per a reading of the announcement?)
Sounds like this includes the removal of the back compat parts of the api response on media info for example?

Not yet, if you’d prefer to close this task after the Wednesday rollout then we can create a second one for removing the backwards compatibility code. That would be the only follow-up I can think of right now.

This change is subject to the https://www.wikidata.org/wiki/Wikidata:Stable_Interface_Policy . Please complete the steps listed there.

Can you clarify which part of the policy you think hasn’t been followed so far?

This change is subject to the https://www.wikidata.org/wiki/Wikidata:Stable_Interface_Policy . Please complete the steps listed there.

Can you clarify which part of the policy you think hasn’t been followed so far?

"Breaking changes to stable interfaces will be duly announced in advance on the relevant mailing lists (wikidata-tech, wikidata and pywikibot) and on the Project Chat. The announcement will generally be made four weeks before, but no less than two weeks before the change is deployed to https://www.wikidata.org/. "

I'm more than slightly annoyed that the Wikidata development team yet again didn't inform the pywikibot list. It's very clearly stated you should inform the pywikibot list. Why did you yet again not do that? This isn't the first time this happened.

Hello Multichill, and apologies for forgetting the pywikibot mailing-list in the process - it's fixed now.

Is the wbentities API versioned in some way?

The wbeditentity API is not directly versioned, no.

Not yet, if you’d prefer to close this task after the Wednesday rollout then we can create a second one for removing the backwards compatibility code. That would be the only follow-up I can think of right now.

Do you still want to do this or do we close it? The beta test in the descriptions looks good to me for verification.

checked with the sandbox link for beta and it works!