Today, we discovered a major regression in Wikilabels. We've patched the issue and made an emergency deployment. We also deleted some labels that were saved while the system was compromised. In this post, we'll describe what happened.
In order to generate visual representations of edit diffs for labeling, we use the API behind Wikipedia. The old way of asking the API to generate a diff was to use ?action=query&prop=revisions&revids=...&rvdiffto=.... The revision for the rvdiffto parameter would appear on the left and the revision for the revids parameter would appear on the right. Recently, this method of gathering diffs from the API has been deprecated in favor of using ?action=compare&torev=...&fromrev=... The revision for the fromrev parameter would appear on the left and the revision for the torev parameter would appear on the right. But we got that backwards! So the UI would show that an edit removing vandalism was performing it! Or worse, an edit vandalizing Wikipedia would look like it was cleaning up vandalism. So needless to say, we can't trust the labels saved while Wikilabels was in a compromised state.
We've deleted the 36 labels that were submitted to editquality campaigns during this period. All labels should be clean from this point forward. Labelers will notice that worksets they though were completed are now incomplete. We're very sorry for this inconvenience.
Special thanks to Paupass for making us aware of this issue on the ORES flow board. Sorry to Papuass, Ivi104, Lsanabria, and 4shadoww for deleting their hard work. I hope it won't slow them down much.
These are the deleted labels for reference (CCO):
task_id | user_id | timestamp | data
---------+----------+----------------------------+----------------------------------------------------------
445742 | 38109688 | 2017-08-30 15:31:52.354537 | {"goodfaith": false, "damaging": true, "unsure": false}
445765 | 38109688 | 2017-08-30 15:32:30.420068 | {"goodfaith": false, "damaging": true, "unsure": true}
445812 | 38109688 | 2017-08-30 15:32:35.83588 | {"goodfaith": true, "damaging": false, "unsure": false}
445877 | 38109688 | 2017-08-30 15:32:38.619456 | {"goodfaith": true, "damaging": false, "unsure": false}
446388 | 38109688 | 2017-08-30 15:33:48.12102 | {"goodfaith": true, "damaging": false, "unsure": false}
446433 | 38109688 | 2017-08-30 15:34:12.903998 | {"goodfaith": true, "damaging": false, "unsure": false}
446562 | 38109688 | 2017-08-30 15:34:16.460096 | {"goodfaith": true, "damaging": false, "unsure": false}
446752 | 38109688 | 2017-08-30 15:34:21.357112 | {"goodfaith": true, "damaging": false, "unsure": false}
447128 | 38109688 | 2017-08-30 15:34:24.683543 | {"goodfaith": true, "damaging": false, "unsure": false}
447219 | 38109688 | 2017-08-30 15:34:52.929963 | {"goodfaith": true, "damaging": false, "unsure": false}
447228 | 38109688 | 2017-08-30 15:35:41.194955 | {"goodfaith": true, "damaging": false, "unsure": false}
370394 | 35332911 | 2017-08-30 22:25:45.732859 | {"goodfaith": true, "damaging": false, "unsure": false}
370724 | 35332911 | 2017-08-30 22:25:52.676656 | {"goodfaith": true, "damaging": true, "unsure": false}
370788 | 35332911 | 2017-08-30 22:26:36.25715 | {"goodfaith": true, "damaging": false, "unsure": false}
371101 | 35332911 | 2017-08-30 22:27:38.908425 | {"goodfaith": true, "damaging": false, "unsure": false}
370368 | 35332911 | 2017-08-31 05:41:25.739392 | {"goodfaith": true, "damaging": false, "unsure": false}
433975 | 4075 | 2017-08-31 09:14:10.80414 | {"goodfaith": true, "damaging": false, "unsure": false}
434380 | 4075 | 2017-08-31 09:15:04.198489 | {"goodfaith": true, "damaging": false, "unsure": false}
434389 | 4075 | 2017-08-31 09:15:16.009233 | {"goodfaith": false, "damaging": true, "unsure": false}
451268 | 14073293 | 2017-08-31 11:46:47.204268 | {"goodfaith": true, "damaging": false, "unsure": false}
451324 | 14073293 | 2017-08-31 11:47:11.079806 | {"goodfaith": true, "damaging": true, "unsure": false}
451688 | 14073293 | 2017-08-31 11:48:25.680257 | {"goodfaith": false, "damaging": true, "unsure": true}
452159 | 14073293 | 2017-08-31 11:48:50.324313 | {"goodfaith": true, "damaging": true, "unsure": true}
452203 | 14073293 | 2017-08-31 11:49:09.644101 | {"goodfaith": false, "damaging": true, "unsure": false}
452210 | 14073293 | 2017-08-31 11:49:34.381716 | {"goodfaith": false, "damaging": false, "unsure": false}
452315 | 14073293 | 2017-08-31 11:49:45.622214 | {"goodfaith": true, "damaging": false, "unsure": false}
452376 | 14073293 | 2017-08-31 11:51:33.128251 | {"goodfaith": true, "damaging": false, "unsure": false}
452431 | 14073293 | 2017-08-31 11:52:11.38513 | {"goodfaith": true, "damaging": true, "unsure": true}
452581 | 14073293 | 2017-08-31 11:52:42.420328 | {"goodfaith": false, "damaging": true, "unsure": false}
452734 | 14073293 | 2017-08-31 11:53:02.748251 | {"goodfaith": true, "damaging": false, "unsure": false}
452967 | 14073293 | 2017-08-31 11:54:23.412685 | {"goodfaith": true, "damaging": false, "unsure": false}
453027 | 14073293 | 2017-08-31 11:55:02.218287 | {"goodfaith": true, "damaging": true, "unsure": false}
453162 | 14073293 | 2017-08-31 11:56:35.027953 | {"goodfaith": true, "damaging": false, "unsure": false}
453192 | 14073293 | 2017-08-31 11:56:46.138465 | {"goodfaith": false, "damaging": true, "unsure": false}
453237 | 14073293 | 2017-08-31 11:58:35.359926 | {"goodfaith": false, "damaging": true, "unsure": false}
453463 | 14073293 | 2017-08-31 11:58:46.129857 | {"goodfaith": true, "damaging": false, "unsure": false}