[Bug] Merging broken, doesn't move content or creates redirects
Closed, ResolvedPublic

Description

Sjoerddebruin updated the task description. (Show Details)
Sjoerddebruin raised the priority of this task from to High.
Sjoerddebruin added a subscriber: Sjoerddebruin.
Restricted Application added a project: Wikidata. · View Herald TranscriptOct 19 2015, 5:24 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Nikki added a subscriber: Nikki.Oct 19 2015, 6:49 PM

Just tested it here and it did everything except to create the redirect and it it to its target.

Using Special:MergeItem I get the following error:

The link enwiki:Protoplanetary disk is already used by item Protoplanetaire schijf (Q20940243). You may remove it from Protoplanetaire schijf (Q20940243) if it does not belong there or merge the items if they are about the exact same topic.
The link dewiki:Protoplanetare Scheibe is already used by item Protoplanetaire schijf (Q20940243). You may remove it from Protoplanetaire schijf (Q20940243) if it does not belong there or merge the items if they are about the exact same topic.
The link hrwiki:Protoplanetarni disk is already used by item Protoplanetaire schijf (Q20940243). You may remove it from Protoplanetaire schijf (Q20940243) if it does not belong there or merge the items if they are about the exact same topic.
The link hewiki:דיסקה קדם-פלנטרית is already used by item Protoplanetaire schijf (Q20940243). You may remove it from Protoplanetaire schijf (Q20940243) if it does not belong there or merge the items if they are about the exact same topic.
The link srwiki:Protoplanetarni disk is already used by item Protoplanetaire schijf (Q20940243). You may remove it from Protoplanetaire schijf (Q20940243) if it does not belong there or merge the items if they are about the exact same topic.
The link fawiki:قرص پیش–سیاره‌ای is already used by item Protoplanetaire schijf (Q20940243). You may remove it from Protoplanetaire schijf (Q20940243) if it does not belong there or merge the items if they are about the exact same topic.

I filled in:
Merge from: Q20940243
Merge to: Q505781

So might not be a problem with the merge gadget, but the merge function itself.

Can confirm, it's the merge feature itself.

Sjoerddebruin renamed this task from Merge.js is broken: failed-save to Merging broken, doesn't move content or creates redirects.Oct 19 2015, 7:54 PM
Sjoerddebruin raised the priority of this task from High to Unbreak Now!.
Sjoerddebruin set Security to None.

If it takes a while, the merging rights should be revoked to avoid a big mess.

aude added a subscriber: aude.Oct 19 2015, 9:13 PM

I see some entries in the error logs like:

/w/api.php   Wikibase\Lib\Store\RevisionedUnresolvedRedirectException from line 94 of /srv/mediawiki/php-1.27.0-wmf.3/extensions/Wikidata/extensions/Wikibase/lib/includes/store/sql/WikiPageEntityRevisionLookup.php: Unresolved redirect to Q2366921 {"exception_id":"0c1eba05"} 
[Exception Wikibase\Lib\Store\RevisionedUnresolvedRedirectException] (/srv/mediawiki/php-1.27.0-wmf.3/extensions/Wikidata/extensions/Wikibase/lib/includes/store/sql/WikiPageEntityRevisionLookup.php:94) Unresolved redirect to Q2366921
  #0 /srv/mediawiki/php-1.27.0-wmf.3/extensions/Wikidata/extensions/Wikibase/repo/includes/api/ModifyEntity.php(209): Wikibase\Lib\Store\WikiPageEntityRevisionLookup->getEntityRevision(Wikibase\DataModel\Entity\ItemId, string)
  #1 /srv/mediawiki/php-1.27.0-wmf.3/extensions/Wikidata/extensions/Wikibase/repo/includes/api/ModifyEntity.php(415): Wikibase\Repo\Api\ModifyEntity->getEntityRevisionFromApiParams(array)
  #2 /srv/mediawiki/php-1.27.0-wmf.3/includes/api/ApiMain.php(1270): Wikibase\Repo\Api\ModifyEntity->execute()
  #3 /srv/mediawiki/php-1.27.0-wmf.3/includes/api/ApiMain.php(466): ApiMain->executeAction()
  #4 /srv/mediawiki/php-1.27.0-wmf.3/includes/api/ApiMain.php(438): ApiMain->executeActionWithErrorHandling()
  #5 /srv/mediawiki/php-1.27.0-wmf.3/api.php(88): ApiMain->execute()
  #6 /srv/mediawiki/w/api.php(3): include(string)

this one had a redirect created today and is likely related

aude added a comment.Oct 19 2015, 9:29 PM

also could be an issue with slave lag

Change 247464 had a related patch set (by Aude) published:
Check for EntityLookupException in ModifyEntity

https://gerrit.wikimedia.org/r/247464

aude added a comment.Oct 19 2015, 9:36 PM

I can't reproduce the issue locally (even after installing the merge gadget for myself), so my patches are only educated guess and based on / fixing what I see in the logs.

555 added a subscriber: 555.Oct 20 2015, 4:05 AM

I could reproduce the problem with the gadget by merging Q15866980 to Q6702577. The first POST

action	wbmergeitems
format	json
fromid	Q15866980
ignoreconflicts	description|label
summary	[[MediaWiki:Gadget-Merge.js|merge.js]] 
toid	Q6702577
token	xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx+\

generates the response

{"servedby":"mw1225","warnings":{"wbmergeitems":{"*":"Unrecognized value for parameter 'ignoreconflicts': label"}},"error":{"code":"failed-save","info":"The save has failed (Der Link [https://de.wikipedia.org/wiki/Lukovi%C4%87 dewiki:Lukovi\u0107] wird bereits vom Objekt [[Q15866980|Q15866980]] verwendet. Du kannst ihn von [[Q15866980|Q15866980]] entfernen, falls er nicht dorthin geh\u00f6rt oder die Objekte zusammenf\u00fchren, falls sie das gleiche Thema behandeln.)","messages":[{"name":"wikibase-api-failed-save","parameters":[],"html":{"*":"The save has failed"}}],"*":"See https://www.wikidata.org/w/api.php for API usage"}}

The problem seems to involve any interwiki additions using the sidebar of wikipedia articles.
I have earlier linked the en:wp article on the Gangte language https://en.wikipedia.org/wiki/Gangte_language with the pms:wp article https://pms.wikipedia.org/wiki/Lenga_Gangte, resulting in a merge of https://www.wikidata.org/wiki/Q1775221 with https://www.wikidata.org/wiki/Q12952442 without receiving the "Don't try to merge items ..." warning. Does that mean the Edit language links option in all wikipedias should be disabled?

aude added a comment.Oct 20 2015, 8:38 AM

can reproduce if my items that I try to merge have site links

aude added a comment.Oct 20 2015, 10:28 AM

git bisect finds https://gerrit.wikimedia.org/r/#/c/244407/ (Effectively remove EDIT_DEFER_UPDATES flag) is causing the issue.

We use SiteLinkConflictLookup (SiteLinkTable) to check for site link conflicts when saving an edit.

For merging, it's two consecutive saves (save the old item, removing stuff then save the new item with everything). Before, we must have executed the SiteLinkTable update immediately on save and now it is deferred until later in the request (e.g. until after the second save).

deferring is generally the right thing to do, but we need to figure out a way to work around this issue that doesn't cause problems with merging or other things.

Change 247464 merged by jenkins-bot:
Check for EntityLookupException in ModifyEntity

https://gerrit.wikimedia.org/r/247464

aude added a comment.Oct 20 2015, 11:44 AM

one solution might be to make the SiteLinkUniquenessValidator aware of items/sitelinks conflicts to ignore (the merge-from item) .

the difficultly is with how all these things are hooked together, but still think this would be the most simple solution.

He7d3r added a subscriber: He7d3r.Oct 20 2015, 12:54 PM

All items merged after 19th October 2015, 17:00 UTC: http://quarry.wmflabs.org/query/5692

Lydia_Pintscher renamed this task from Merging broken, doesn't move content or creates redirects to [Bug] Merging broken, doesn't move content or creates redirects.
Bene added a comment.Oct 20 2015, 2:54 PM

Why wasn't this catched by any of our integration tests? Don't we run them before we deploy a new MW version to wikidata.org?

I'm guessing deferred updates don't work in the say way in test conditions?
I was also surprised merging broke until looking at what caused the break...

Change 247588 had a related patch set (by Aude) published:
Split SiteLinkConflictLookup from SiteLinkTable

https://gerrit.wikimedia.org/r/247588

aude added a comment.Oct 20 2015, 3:49 PM

@Bene @Addshore probably this specific case is not covered (adequately) by phpunit tests nor browser tests.

and then we don't run the browser tests for every core version, although they do run daily on beta (with core master and wikibase master, which could help). problem also is that some browser tests constantly fail (e.g. due to time outs or other issues).

We are aware and want to work on fixing this, asap and try to find other ways to improve CI for these sorts of issues.

Adding User-notice for the next Tech News problems section

Change 247626 had a related patch set uploaded (by Aude):
Ignore site link uniqueness conflicts of from-item during merge [WIP]

https://gerrit.wikimedia.org/r/247626

Change 247633 had a related patch set uploaded (by Hoo man):
Ignore constraints on save in ItemMergeInteractor

https://gerrit.wikimedia.org/r/247633

Change 247626 abandoned by Aude:
Ignore site link uniqueness conflicts of from-item during merge [WIP]

https://gerrit.wikimedia.org/r/247626

Change 247634 had a related patch set uploaded (by Aude):
Ignore constraints on save in ItemMergeInteractor

https://gerrit.wikimedia.org/r/247634

Change 247635 had a related patch set uploaded (by Aude):
Ignore constraints on save in ItemMergeInteractor

https://gerrit.wikimedia.org/r/247635

Change 247633 merged by jenkins-bot:
Ignore constraints on save in ItemMergeInteractor

https://gerrit.wikimedia.org/r/247633

SzMithrandir added a subscriber: SzMithrandir.EditedOct 20 2015, 7:12 PM

(a nooby thought passing by) Oh gees Christ I've been always disliking wikidata since it got this much buggy features: label, description, alias, statements, you name it. Miss the old days when it just link the things... The craziest part is that most edits cannot be read off from the diff code easily; how could we follow it?!

Nevertheless, maybe I'm too reactionary; but I really don't believe WikiData should continue like this in future.

  • did I comment right at the time of fixing complete? XD
hoo closed this task as Resolved.Oct 20 2015, 7:12 PM
hoo claimed this task.
hoo removed a project: Patch-For-Review.

Will (hopefully) backport the fix later today.

hoo moved this task from Doing to Done on the Wikidata-Sprint-2015-10-13 board.Oct 20 2015, 7:12 PM

Change 247588 merged by jenkins-bot:
Split SiteLinkConflictLookup from SiteLinkTable

https://gerrit.wikimedia.org/r/247588

Change 247635 abandoned by Hoo man:
Ignore constraints on save in ItemMergeInteractor

Reason:
Not needed: wmf1 is only on clients as of now.

https://gerrit.wikimedia.org/r/247635

Change 247634 merged by jenkins-bot:
Ignore constraints on save in ItemMergeInteractor

https://gerrit.wikimedia.org/r/247634

<3
Thanks, folks!

Agabi10 removed a subscriber: Agabi10.Oct 22 2015, 7:59 PM