Page MenuHomePhabricator

Ensure ORES data violating constraints do not affect production
Closed, ResolvedPublic

Description

We merged the schema change, it got reverted because it broke beta, then I checked and realized it was bad data in the ores_classification table (We need to be sure they don't exist in production tables too before applying the schema change).

  • Tables changed: ores_classification
  • Databases needing this update: fawiki, wikidatawiki, ruwiki, ptwiki, trwiki, plwiki, nlwiki, enwiki
  • Develop maintenance script described in T145503
  • Run mainenance script on enwiki, ptwiki, ruwiki, fawiki, trwiki, nlwiki, wikidatawiki, plwiki

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 12 2016, 5:02 AM
Ladsgroup updated the task description. (Show Details)Sep 12 2016, 5:30 AM
jcrespo added a subscriber: jcrespo.

Amir, I am not sure what you want #DBAs to do here, as I do not have a patch to apply or review. In any case, this doesn't seem like a Blocked-on-schema-change ticket, but you are asking for help with a review (that only requires the DBA tag). If that is the case, please say so explicitly, and add me to such review, I cannot understand well what do you want from us yet (I am not involved with ORES on a day-to-day basis except on the database side).

I am happy to help, just tell me how.

I'm trying to follow https://wikitech.wikimedia.org/wiki/Schema_changes#Workflow_of_a_schema_change. This is the article 3: "Once the solution has been agreed among all developers (usually that means merged to HEAD including the schema change, but disabled on configuration), and you need to apply that to the live databases, create a separate specific task (subtask of the main issue) for the Blocked-on-schema-change, owned by DBA, providing the following specific information, one per batch (group of alters to be done at the same time)"
I copied the title from the example given there T119752: Review schema changes for T119495
What I need is applying these index changes in the live database and that's it :)

Ladsgroup, this is reverted https://gerrit.wikimedia.org/r/#/c/309818/ . Even if you did that, it is not "merged to HEAD", as far as I can see. So this is not ready for deploy yet.

We need to be sure they don't exist in production tables too before applying the schema change

Which means we have to check for that BEFORE we apply any further change. So this is not a schema change (yet), it is a regular DBA ticket, then if we are successful, we can do the schema change with no issue.

However, you did not include any details on how to check for data corruption. Please tell me so I can check. What was the issue on beta? You did not add those details.

jcrespo renamed this task from Review schema changes for T143962 to Check ORES data corruption on Beta do not affect production.Sep 12 2016, 8:29 AM

And that revert got reverted later on:
https://gerrit.wikimedia.org/r/#/c/309837/

So It' in HEAD now. We should wait for a few days until wmf.19 gets deployed in production and deploy these changes afterwards.

Regarding the corrupt data. We are adding two unique indexes. By corrupt, I mean data that violate these unique constraints. Just thinking out loud: Something like this SELECT oresc_rev, oresc_model, oresc_class, count(*) FROM ores_classification GROUP BY oresc_rev, oresc_model, oresc_class HAVING count(*) > 1; I want to check that but I was thinking that it might be better to be done before the schema update because we might get some corrupt data between now and the next week (or any time that we want to apply this schema change). If you tell me to check it now, I do it now.

jcrespo claimed this task.Sep 12 2016, 8:47 AM
jcrespo moved this task from Triage to Next on the DBA board.

@Ladsgroup, allow me to do that for you properly, so we do not affect the performance of the other queries. I will get back to you.

jcrespo renamed this task from Check ORES data corruption on Beta do not affect production to Check ORES data violating constraints on Beta do not affect production.Sep 13 2016, 8:12 AM

@Ladsgroup the title I added would have been more descriptive for me to understand the issue. I will be checking this now.

jcrespo moved this task from Next to In progress on the DBA board.Sep 13 2016, 10:33 AM

There are multiple rows with issue on production, here is the list I got some minutes ago:

1$ for wiki in fawiki wikidatawiki ruwiki ptwiki trwiki plwiki nlwiki enwiki; do echo "$wiki"; mysql -h dbstore1002 $wiki -e "SELECT oresc_rev, oresc_model, oresc_class, count(*) FROM ores_classification GROUP BY oresc_rev, oresc_model, oresc_class HAVING count(*) > 1;SELECT oresc_rev, oresc_model, oresc_class, count(*) FROM ores_classification GROUP BY oresc_rev, oresc_is_predicted, oresc_model HAVING count(*) > 1"; done
2fawiki
3+-----------+-------------+-------------+----------+
4| oresc_rev | oresc_model | oresc_class | count(*) |
5+-----------+-------------+-------------+----------+
6| 18004937 | 7 | 1 | 2 |
7| 18005230 | 7 | 1 | 2 |
8+-----------+-------------+-------------+----------+
9+-----------+-------------+-------------+----------+
10| oresc_rev | oresc_model | oresc_class | count(*) |
11+-----------+-------------+-------------+----------+
12| 18004937 | 7 | 1 | 2 |
13| 18005230 | 7 | 1 | 2 |
14+-----------+-------------+-------------+----------+
15wikidatawiki
16+-----------+-------------+-------------+----------+
17| oresc_rev | oresc_model | oresc_class | count(*) |
18+-----------+-------------+-------------+----------+
19| 371063479 | 7 | 1 | 2 |
20| 371063486 | 7 | 1 | 2 |
21| 371063490 | 7 | 1 | 2 |
22| 371063497 | 7 | 1 | 2 |
23| 371063502 | 7 | 1 | 2 |
24| 371063503 | 7 | 1 | 2 |
25| 371063507 | 7 | 1 | 2 |
26| 371063508 | 7 | 1 | 2 |
27| 371063512 | 7 | 1 | 2 |
28| 371063520 | 7 | 1 | 2 |
29| 371066039 | 7 | 1 | 2 |
30| 371066042 | 7 | 1 | 2 |
31| 371066207 | 7 | 1 | 2 |
32| 371067262 | 7 | 1 | 2 |
33| 371067267 | 7 | 1 | 2 |
34| 371067691 | 7 | 1 | 2 |
35| 371073123 | 7 | 1 | 2 |
36| 371074817 | 7 | 1 | 2 |
37| 371075293 | 7 | 1 | 2 |
38| 371075452 | 7 | 1 | 2 |
39| 371075775 | 7 | 1 | 2 |
40| 371079004 | 7 | 1 | 2 |
41| 371372839 | 7 | 1 | 2 |
42+-----------+-------------+-------------+----------+
43+-----------+-------------+-------------+----------+
44| oresc_rev | oresc_model | oresc_class | count(*) |
45+-----------+-------------+-------------+----------+
46| 371063479 | 7 | 1 | 2 |
47| 371063486 | 7 | 1 | 2 |
48| 371063490 | 7 | 1 | 2 |
49| 371063497 | 7 | 1 | 2 |
50| 371063502 | 7 | 1 | 2 |
51| 371063503 | 7 | 1 | 2 |
52| 371063507 | 7 | 1 | 2 |
53| 371063508 | 7 | 1 | 2 |
54| 371063512 | 7 | 1 | 2 |
55| 371063520 | 7 | 1 | 2 |
56| 371066039 | 7 | 1 | 2 |
57| 371066042 | 7 | 1 | 2 |
58| 371066207 | 7 | 1 | 2 |
59| 371067262 | 7 | 1 | 2 |
60| 371067267 | 7 | 1 | 2 |
61| 371067691 | 7 | 1 | 2 |
62| 371073123 | 7 | 1 | 2 |
63| 371074817 | 7 | 1 | 2 |
64| 371075293 | 7 | 1 | 2 |
65| 371075452 | 7 | 1 | 2 |
66| 371075775 | 7 | 1 | 2 |
67| 371079004 | 7 | 1 | 2 |
68| 371372839 | 7 | 1 | 2 |
69+-----------+-------------+-------------+----------+
70ruwiki
71+-----------+-------------+-------------+----------+
72| oresc_rev | oresc_model | oresc_class | count(*) |
73+-----------+-------------+-------------+----------+
74| 80522981 | 9 | 1 | 2 |
75| 80524242 | 9 | 1 | 2 |
76| 80525414 | 9 | 1 | 2 |
77| 80525587 | 9 | 1 | 2 |
78+-----------+-------------+-------------+----------+
79+-----------+-------------+-------------+----------+
80| oresc_rev | oresc_model | oresc_class | count(*) |
81+-----------+-------------+-------------+----------+
82| 80522981 | 9 | 1 | 2 |
83| 80524242 | 9 | 1 | 2 |
84| 80525414 | 9 | 1 | 2 |
85| 80525587 | 9 | 1 | 2 |
86+-----------+-------------+-------------+----------+
87ptwiki
88+-----------+-------------+-------------+----------+
89| oresc_rev | oresc_model | oresc_class | count(*) |
90+-----------+-------------+-------------+----------+
91| 46572008 | 7 | 1 | 2 |
92+-----------+-------------+-------------+----------+
93+-----------+-------------+-------------+----------+
94| oresc_rev | oresc_model | oresc_class | count(*) |
95+-----------+-------------+-------------+----------+
96| 46572008 | 7 | 1 | 2 |
97+-----------+-------------+-------------+----------+
98trwiki
99+-----------+-------------+-------------+----------+
100| oresc_rev | oresc_model | oresc_class | count(*) |
101+-----------+-------------+-------------+----------+
102| 17541889 | 7 | 1 | 2 |
103+-----------+-------------+-------------+----------+
104+-----------+-------------+-------------+----------+
105| oresc_rev | oresc_model | oresc_class | count(*) |
106+-----------+-------------+-------------+----------+
107| 17541889 | 7 | 1 | 2 |
108+-----------+-------------+-------------+----------+
109plwiki
110nlwiki
111+-----------+-------------+-------------+----------+
112| oresc_rev | oresc_model | oresc_class | count(*) |
113+-----------+-------------+-------------+----------+
114| 47365689 | 7 | 1 | 2 |
115+-----------+-------------+-------------+----------+
116+-----------+-------------+-------------+----------+
117| oresc_rev | oresc_model | oresc_class | count(*) |
118+-----------+-------------+-------------+----------+
119| 47365689 | 7 | 1 | 2 |
120+-----------+-------------+-------------+----------+
121enwiki
122+-----------+-------------+-------------+----------+
123| oresc_rev | oresc_model | oresc_class | count(*) |
124+-----------+-------------+-------------+----------+
125| 736737621 | 21 | 1 | 2 |
126| 736737622 | 21 | 1 | 2 |
127| 736737657 | 21 | 1 | 2 |
128| 736737980 | 21 | 1 | 2 |
129| 736737981 | 21 | 1 | 2 |
130| 736738145 | 21 | 1 | 2 |
131| 736738146 | 21 | 1 | 2 |
132| 736738147 | 21 | 1 | 2 |
133| 736738148 | 21 | 1 | 2 |
134| 736738149 | 21 | 1 | 2 |
135| 736738150 | 21 | 1 | 2 |
136| 736738151 | 21 | 1 | 2 |
137| 736738152 | 21 | 1 | 2 |
138| 736738153 | 21 | 1 | 2 |
139| 736738154 | 21 | 1 | 2 |
140| 736738155 | 21 | 1 | 2 |
141| 736738174 | 21 | 1 | 2 |
142| 736738175 | 21 | 1 | 2 |
143| 736738196 | 21 | 1 | 2 |
144| 736738252 | 21 | 1 | 2 |
145| 736738255 | 21 | 1 | 2 |
146| 736738300 | 21 | 1 | 2 |
147| 736738322 | 21 | 1 | 2 |
148| 736738442 | 21 | 1 | 2 |
149| 736738446 | 21 | 1 | 2 |
150| 736738496 | 21 | 1 | 2 |
151| 736738509 | 21 | 1 | 2 |
152| 736738511 | 21 | 1 | 2 |
153| 736738674 | 21 | 1 | 2 |
154| 736738810 | 21 | 1 | 2 |
155| 736738828 | 21 | 1 | 2 |
156| 736738933 | 21 | 1 | 2 |
157| 736738998 | 21 | 1 | 2 |
158| 736739004 | 21 | 1 | 2 |
159| 736739011 | 21 | 1 | 2 |
160| 736739012 | 21 | 1 | 2 |
161| 736739013 | 21 | 1 | 2 |
162| 736739076 | 21 | 1 | 2 |
163| 736739105 | 21 | 1 | 2 |
164| 736739255 | 21 | 1 | 2 |
165| 736739757 | 21 | 1 | 2 |
166| 736740178 | 21 | 1 | 2 |
167| 736740179 | 21 | 1 | 2 |
168| 736740182 | 21 | 1 | 2 |
169| 736740183 | 21 | 1 | 2 |
170| 736740642 | 21 | 1 | 2 |
171| 736740743 | 21 | 1 | 2 |
172| 736740847 | 21 | 1 | 2 |
173| 736740849 | 21 | 1 | 2 |
174| 736740851 | 21 | 1 | 2 |
175| 736740887 | 21 | 1 | 2 |
176| 736741070 | 21 | 1 | 2 |
177| 736741086 | 21 | 1 | 2 |
178| 736741088 | 21 | 1 | 2 |
179| 736741089 | 21 | 1 | 2 |
180| 736741125 | 21 | 1 | 2 |
181| 736741405 | 21 | 1 | 2 |
182| 736741406 | 21 | 1 | 2 |
183| 736741407 | 21 | 1 | 2 |
184| 736741409 | 21 | 1 | 2 |
185| 736741482 | 21 | 1 | 2 |
186| 736741554 | 21 | 1 | 2 |
187| 736744533 | 21 | 1 | 2 |
188| 736744538 | 21 | 1 | 2 |
189| 736744655 | 21 | 1 | 2 |
190| 736744674 | 21 | 1 | 2 |
191| 736744902 | 21 | 1 | 2 |
192| 736744903 | 21 | 1 | 2 |
193| 736744904 | 21 | 1 | 2 |
194| 736744905 | 21 | 1 | 2 |
195| 736744906 | 21 | 1 | 2 |
196| 736744907 | 21 | 1 | 2 |
197| 736744908 | 21 | 1 | 2 |
198| 736744909 | 21 | 1 | 2 |
199| 736744910 | 21 | 1 | 2 |
200| 736744912 | 21 | 1 | 2 |
201| 736744913 | 21 | 1 | 2 |
202| 736744914 | 21 | 1 | 2 |
203| 736744915 | 21 | 1 | 2 |
204| 736744916 | 21 | 1 | 2 |
205| 736744925 | 21 | 1 | 2 |
206| 736744929 | 21 | 1 | 2 |
207| 736744930 | 21 | 1 | 2 |
208| 736744931 | 21 | 1 | 2 |
209| 736744932 | 21 | 1 | 2 |
210| 736744933 | 21 | 1 | 2 |
211| 736744934 | 21 | 1 | 2 |
212| 736744935 | 21 | 1 | 2 |
213| 736744936 | 21 | 1 | 2 |
214| 736744937 | 21 | 1 | 2 |
215| 736744938 | 21 | 1 | 2 |
216| 736745006 | 21 | 1 | 2 |
217| 736745011 | 21 | 1 | 2 |
218| 736745022 | 21 | 1 | 2 |
219| 736745029 | 21 | 1 | 2 |
220| 736745038 | 21 | 1 | 2 |
221| 736745045 | 21 | 1 | 2 |
222| 736745058 | 21 | 1 | 2 |
223| 736745059 | 21 | 1 | 2 |
224| 736745095 | 21 | 1 | 2 |
225| 736745149 | 21 | 1 | 2 |
226| 736745153 | 21 | 1 | 2 |
227| 736745158 | 21 | 1 | 2 |
228| 736745251 | 21 | 1 | 2 |
229| 736745263 | 21 | 1 | 2 |
230| 736745264 | 21 | 1 | 2 |
231| 736745268 | 21 | 1 | 2 |
232| 736745270 | 21 | 1 | 2 |
233| 736745275 | 21 | 1 | 2 |
234| 736745281 | 21 | 1 | 2 |
235| 736745284 | 21 | 1 | 2 |
236| 736745287 | 21 | 1 | 2 |
237| 736745288 | 21 | 1 | 2 |
238| 736745290 | 21 | 1 | 2 |
239| 736745292 | 21 | 1 | 2 |
240| 736745294 | 21 | 1 | 2 |
241| 736745295 | 21 | 1 | 2 |
242| 736745296 | 21 | 1 | 2 |
243| 736745297 | 21 | 1 | 2 |
244| 736745298 | 21 | 1 | 2 |
245| 736745299 | 21 | 1 | 2 |
246| 736745300 | 21 | 1 | 2 |
247| 736745301 | 21 | 1 | 2 |
248| 736745302 | 21 | 1 | 2 |
249| 736745308 | 21 | 1 | 2 |
250| 736745312 | 21 | 1 | 2 |
251| 736745313 | 21 | 1 | 2 |
252| 736745314 | 21 | 1 | 2 |
253| 736745315 | 21 | 1 | 2 |
254| 736745317 | 21 | 1 | 2 |
255| 736745318 | 21 | 1 | 2 |
256| 736745319 | 21 | 1 | 2 |
257| 736745320 | 21 | 1 | 2 |
258| 736745321 | 21 | 1 | 2 |
259| 736745322 | 21 | 1 | 2 |
260| 736745323 | 21 | 1 | 2 |
261| 736745324 | 21 | 1 | 2 |
262| 736745325 | 21 | 1 | 2 |
263| 736745326 | 21 | 1 | 2 |
264| 736745327 | 21 | 1 | 2 |
265| 736745328 | 21 | 1 | 2 |
266| 736745333 | 21 | 1 | 2 |
267| 736745339 | 21 | 1 | 2 |
268| 736745340 | 21 | 1 | 2 |
269| 736745341 | 21 | 1 | 2 |
270| 736745342 | 21 | 1 | 2 |
271| 736745343 | 21 | 1 | 2 |
272| 736745344 | 21 | 1 | 2 |
273| 736745345 | 21 | 1 | 2 |
274| 736745346 | 21 | 1 | 2 |
275| 736745347 | 21 | 1 | 2 |
276| 736745348 | 21 | 1 | 2 |
277| 736745349 | 21 | 1 | 2 |
278| 736745350 | 21 | 1 | 2 |
279| 736745351 | 21 | 1 | 2 |
280| 736745352 | 21 | 1 | 2 |
281| 736745353 | 21 | 1 | 2 |
282| 736745354 | 21 | 1 | 2 |
283| 736745355 | 21 | 1 | 2 |
284| 736745356 | 21 | 1 | 2 |
285| 736745357 | 21 | 1 | 2 |
286| 736745358 | 21 | 1 | 2 |
287| 736745359 | 21 | 1 | 2 |
288| 736745360 | 21 | 1 | 2 |
289| 736745361 | 21 | 1 | 2 |
290| 736745362 | 21 | 1 | 2 |
291| 736745363 | 21 | 1 | 2 |
292| 736745388 | 21 | 1 | 2 |
293| 736745393 | 21 | 1 | 2 |
294| 736745401 | 21 | 1 | 2 |
295| 736745402 | 21 | 1 | 2 |
296| 736745404 | 21 | 1 | 2 |
297| 736745477 | 21 | 1 | 2 |
298| 736745593 | 21 | 1 | 2 |
299| 736745594 | 21 | 1 | 2 |
300| 736745595 | 21 | 1 | 2 |
301| 736745596 | 21 | 1 | 2 |
302| 736745597 | 21 | 1 | 2 |
303| 736745598 | 21 | 1 | 2 |
304| 736745599 | 21 | 1 | 2 |
305| 736745600 | 21 | 1 | 2 |
306| 736745601 | 21 | 1 | 2 |
307| 736745602 | 21 | 1 | 2 |
308| 736745603 | 21 | 1 | 2 |
309| 736745604 | 21 | 1 | 2 |
310| 736745605 | 21 | 1 | 2 |
311| 736745606 | 21 | 1 | 2 |
312| 736745607 | 21 | 1 | 2 |
313| 736745608 | 21 | 1 | 2 |
314| 736745609 | 21 | 1 | 2 |
315| 736745638 | 21 | 1 | 2 |
316| 736745695 | 21 | 1 | 2 |
317| 736745696 | 21 | 1 | 2 |
318| 736745697 | 21 | 1 | 2 |
319| 736745698 | 21 | 1 | 2 |
320| 736745701 | 21 | 1 | 2 |
321| 736745789 | 21 | 1 | 2 |
322| 736745790 | 21 | 1 | 2 |
323| 736745791 | 21 | 1 | 2 |
324| 736745792 | 21 | 1 | 2 |
325| 736745793 | 21 | 1 | 2 |
326| 736745794 | 21 | 1 | 2 |
327| 736745795 | 21 | 1 | 2 |
328| 736745796 | 21 | 1 | 2 |
329| 736745797 | 21 | 1 | 2 |
330| 736745800 | 21 | 1 | 2 |
331| 736745938 | 21 | 1 | 2 |
332| 736746061 | 21 | 1 | 2 |
333| 736746290 | 21 | 1 | 2 |
334| 736746291 | 21 | 1 | 2 |
335| 736746292 | 21 | 1 | 2 |
336| 736746294 | 21 | 1 | 2 |
337| 736746295 | 21 | 1 | 2 |
338| 736746296 | 21 | 1 | 2 |
339| 736746328 | 21 | 1 | 2 |
340| 736746343 | 21 | 1 | 2 |
341| 736746345 | 21 | 1 | 2 |
342| 736746472 | 21 | 1 | 2 |
343| 736746473 | 21 | 1 | 2 |
344| 736746474 | 21 | 1 | 2 |
345| 736746476 | 21 | 1 | 2 |
346| 736746477 | 21 | 1 | 2 |
347| 736746478 | 21 | 1 | 2 |
348| 736746479 | 21 | 1 | 2 |
349| 736746563 | 21 | 1 | 2 |
350| 736746617 | 21 | 1 | 2 |
351| 736746618 | 21 | 1 | 2 |
352| 736746619 | 21 | 1 | 2 |
353| 736746620 | 21 | 1 | 2 |
354| 736746621 | 21 | 1 | 2 |
355| 736746622 | 21 | 1 | 2 |
356| 736746623 | 21 | 1 | 2 |
357| 736746624 | 21 | 1 | 2 |
358| 736746883 | 21 | 1 | 2 |
359| 736746884 | 21 | 1 | 2 |
360| 736746885 | 21 | 1 | 2 |
361| 736746901 | 21 | 1 | 2 |
362| 736746902 | 21 | 1 | 2 |
363| 736746906 | 21 | 1 | 2 |
364| 736746935 | 21 | 1 | 2 |
365| 736747036 | 21 | 1 | 2 |
366| 736747190 | 21 | 1 | 2 |
367| 736747194 | 21 | 1 | 2 |
368| 736747226 | 21 | 1 | 2 |
369| 736747245 | 21 | 1 | 2 |
370| 736747291 | 21 | 1 | 2 |
371| 736747292 | 21 | 1 | 2 |
372| 736747359 | 21 | 1 | 2 |
373| 736747361 | 21 | 1 | 2 |
374| 736747438 | 21 | 1 | 2 |
375| 736747492 | 21 | 1 | 2 |
376| 736747493 | 21 | 1 | 2 |
377| 736747494 | 21 | 1 | 2 |
378| 736747495 | 21 | 1 | 2 |
379| 736747496 | 21 | 1 | 2 |
380| 736747497 | 21 | 1 | 2 |
381| 736747498 | 21 | 1 | 2 |
382| 736747499 | 21 | 1 | 2 |
383| 736747500 | 21 | 1 | 2 |
384| 736747506 | 21 | 1 | 2 |
385| 736747507 | 21 | 1 | 2 |
386| 736747508 | 21 | 1 | 2 |
387| 736747509 | 21 | 1 | 2 |
388| 736747515 | 21 | 1 | 2 |
389| 736747586 | 21 | 1 | 2 |
390| 736747590 | 21 | 1 | 2 |
391| 736747601 | 21 | 1 | 2 |
392| 736747602 | 21 | 1 | 2 |
393| 736747603 | 21 | 1 | 2 |
394| 736747604 | 21 | 1 | 2 |
395| 736747605 | 21 | 1 | 2 |
396| 736747608 | 21 | 1 | 2 |
397| 736747609 | 21 | 1 | 2 |
398| 736747610 | 21 | 1 | 2 |
399| 736747611 | 21 | 1 | 2 |
400| 736747612 | 21 | 1 | 2 |
401| 736747627 | 21 | 1 | 2 |
402| 736747630 | 21 | 1 | 2 |
403| 736747632 | 21 | 1 | 2 |
404| 736747634 | 21 | 1 | 2 |
405| 736747637 | 21 | 1 | 2 |
406| 736747691 | 21 | 1 | 2 |
407| 736747692 | 21 | 1 | 2 |
408| 736747693 | 21 | 1 | 2 |
409| 736747777 | 21 | 1 | 2 |
410| 736747778 | 21 | 1 | 2 |
411| 736747779 | 21 | 1 | 2 |
412| 736747866 | 21 | 1 | 2 |
413| 736747888 | 21 | 1 | 2 |
414| 736747913 | 21 | 1 | 2 |
415| 736748046 | 21 | 1 | 2 |
416| 736748047 | 21 | 1 | 2 |
417| 736748069 | 21 | 1 | 2 |
418| 736748072 | 21 | 1 | 2 |
419| 736748163 | 21 | 1 | 2 |
420| 736748178 | 21 | 1 | 2 |
421| 736748179 | 21 | 1 | 2 |
422| 736748180 | 21 | 1 | 2 |
423| 736748181 | 21 | 1 | 2 |
424| 736748266 | 21 | 1 | 2 |
425| 736748274 | 21 | 1 | 2 |
426| 736748491 | 21 | 1 | 2 |
427| 736748492 | 21 | 1 | 2 |
428| 736748493 | 21 | 1 | 2 |
429| 736748598 | 21 | 1 | 2 |
430| 736748599 | 21 | 1 | 2 |
431| 736748747 | 21 | 1 | 2 |
432| 736748762 | 21 | 1 | 2 |
433| 736748966 | 21 | 1 | 2 |
434| 736749037 | 21 | 1 | 2 |
435| 736749196 | 21 | 1 | 2 |
436| 736749197 | 21 | 1 | 2 |
437| 736749198 | 21 | 1 | 2 |
438| 736749199 | 21 | 1 | 2 |
439| 736749209 | 21 | 1 | 2 |
440| 736749210 | 21 | 1 | 2 |
441| 736749211 | 21 | 1 | 2 |
442| 736749212 | 21 | 1 | 2 |
443| 736749213 | 21 | 1 | 2 |
444| 736749214 | 21 | 1 | 2 |
445| 736749215 | 21 | 1 | 2 |
446| 736749216 | 21 | 1 | 2 |
447| 736749218 | 21 | 1 | 2 |
448| 736749219 | 21 | 1 | 2 |
449| 736749602 | 21 | 1 | 2 |
450| 736749611 | 21 | 1 | 2 |
451| 736749613 | 21 | 1 | 2 |
452| 736750373 | 21 | 1 | 2 |
453| 736750548 | 21 | 1 | 2 |
454| 736750778 | 21 | 1 | 2 |
455| 736750819 | 21 | 1 | 2 |
456| 736750825 | 21 | 1 | 2 |
457| 736750879 | 21 | 1 | 2 |
458| 736750903 | 21 | 1 | 2 |
459| 736750976 | 21 | 1 | 2 |
460| 736751021 | 21 | 1 | 2 |
461| 736751138 | 21 | 1 | 2 |
462| 736751139 | 21 | 1 | 2 |
463| 736751197 | 21 | 1 | 2 |
464| 736751198 | 21 | 1 | 2 |
465| 736751199 | 21 | 1 | 2 |
466| 736751200 | 21 | 1 | 2 |
467| 736751201 | 21 | 1 | 2 |
468| 736751202 | 21 | 1 | 2 |
469| 736751203 | 21 | 1 | 2 |
470| 736751206 | 21 | 1 | 2 |
471| 736751244 | 21 | 1 | 2 |
472| 736751495 | 21 | 1 | 2 |
473| 736751496 | 21 | 1 | 2 |
474| 736751498 | 21 | 1 | 2 |
475| 736751499 | 21 | 1 | 2 |
476| 736751502 | 21 | 1 | 2 |
477| 736751641 | 21 | 1 | 2 |
478| 736751667 | 21 | 1 | 2 |
479| 736751838 | 21 | 1 | 2 |
480| 736751991 | 21 | 1 | 2 |
481| 736751993 | 21 | 1 | 2 |
482| 736752171 | 21 | 1 | 2 |
483| 736752172 | 21 | 1 | 2 |
484+-----------+-------------+-------------+----------+
485+-----------+-------------+-------------+----------+
486| oresc_rev | oresc_model | oresc_class | count(*) |
487+-----------+-------------+-------------+----------+
488| 736737621 | 21 | 1 | 2 |
489| 736737622 | 21 | 1 | 2 |
490| 736737657 | 21 | 1 | 2 |
491| 736737980 | 21 | 1 | 2 |
492| 736737981 | 21 | 1 | 2 |
493| 736738145 | 21 | 1 | 2 |
494| 736738146 | 21 | 1 | 2 |
495| 736738147 | 21 | 1 | 2 |
496| 736738148 | 21 | 1 | 2 |
497| 736738149 | 21 | 1 | 2 |
498| 736738150 | 21 | 1 | 2 |
499| 736738151 | 21 | 1 | 2 |
500| 736738152 | 21 | 1 | 2 |
501| 736738153 | 21 | 1 | 2 |
502| 736738154 | 21 | 1 | 2 |
503| 736738155 | 21 | 1 | 2 |
504| 736738174 | 21 | 1 | 2 |
505| 736738175 | 21 | 1 | 2 |
506| 736738196 | 21 | 1 | 2 |
507| 736738252 | 21 | 1 | 2 |
508| 736738255 | 21 | 1 | 2 |
509| 736738300 | 21 | 1 | 2 |
510| 736738322 | 21 | 1 | 2 |
511| 736738442 | 21 | 1 | 2 |
512| 736738446 | 21 | 1 | 2 |
513| 736738496 | 21 | 1 | 2 |
514| 736738509 | 21 | 1 | 2 |
515| 736738511 | 21 | 1 | 2 |
516| 736738674 | 21 | 1 | 2 |
517| 736738810 | 21 | 1 | 2 |
518| 736738828 | 21 | 1 | 2 |
519| 736738933 | 21 | 1 | 2 |
520| 736738998 | 21 | 1 | 2 |
521| 736739004 | 21 | 1 | 2 |
522| 736739011 | 21 | 1 | 2 |
523| 736739012 | 21 | 1 | 2 |
524| 736739013 | 21 | 1 | 2 |
525| 736739076 | 21 | 1 | 2 |
526| 736739105 | 21 | 1 | 2 |
527| 736739255 | 21 | 1 | 2 |
528| 736739757 | 21 | 1 | 2 |
529| 736740178 | 21 | 1 | 2 |
530| 736740179 | 21 | 1 | 2 |
531| 736740182 | 21 | 1 | 2 |
532| 736740183 | 21 | 1 | 2 |
533| 736740642 | 21 | 1 | 2 |
534| 736740743 | 21 | 1 | 2 |
535| 736740847 | 21 | 1 | 2 |
536| 736740849 | 21 | 1 | 2 |
537| 736740851 | 21 | 1 | 2 |
538| 736740887 | 21 | 1 | 2 |
539| 736741070 | 21 | 1 | 2 |
540| 736741086 | 21 | 1 | 2 |
541| 736741088 | 21 | 1 | 2 |
542| 736741089 | 21 | 1 | 2 |
543| 736741125 | 21 | 1 | 2 |
544| 736741405 | 21 | 1 | 2 |
545| 736741406 | 21 | 1 | 2 |
546| 736741407 | 21 | 1 | 2 |
547| 736741409 | 21 | 1 | 2 |
548| 736741482 | 21 | 1 | 2 |
549| 736741554 | 21 | 1 | 2 |
550| 736744533 | 21 | 1 | 2 |
551| 736744538 | 21 | 1 | 2 |
552| 736744655 | 21 | 1 | 2 |
553| 736744674 | 21 | 1 | 2 |
554| 736744902 | 21 | 1 | 2 |
555| 736744903 | 21 | 1 | 2 |
556| 736744904 | 21 | 1 | 2 |
557| 736744905 | 21 | 1 | 2 |
558| 736744906 | 21 | 1 | 2 |
559| 736744907 | 21 | 1 | 2 |
560| 736744908 | 21 | 1 | 2 |
561| 736744909 | 21 | 1 | 2 |
562| 736744910 | 21 | 1 | 2 |
563| 736744912 | 21 | 1 | 2 |
564| 736744913 | 21 | 1 | 2 |
565| 736744914 | 21 | 1 | 2 |
566| 736744915 | 21 | 1 | 2 |
567| 736744916 | 21 | 1 | 2 |
568| 736744925 | 21 | 1 | 2 |
569| 736744929 | 21 | 1 | 2 |
570| 736744930 | 21 | 1 | 2 |
571| 736744931 | 21 | 1 | 2 |
572| 736744932 | 21 | 1 | 2 |
573| 736744933 | 21 | 1 | 2 |
574| 736744934 | 21 | 1 | 2 |
575| 736744935 | 21 | 1 | 2 |
576| 736744936 | 21 | 1 | 2 |
577| 736744937 | 21 | 1 | 2 |
578| 736744938 | 21 | 1 | 2 |
579| 736745006 | 21 | 1 | 2 |
580| 736745011 | 21 | 1 | 2 |
581| 736745022 | 21 | 1 | 2 |
582| 736745029 | 21 | 1 | 2 |
583| 736745038 | 21 | 1 | 2 |
584| 736745045 | 21 | 1 | 2 |
585| 736745058 | 21 | 1 | 2 |
586| 736745059 | 21 | 1 | 2 |
587| 736745095 | 21 | 1 | 2 |
588| 736745149 | 21 | 1 | 2 |
589| 736745153 | 21 | 1 | 2 |
590| 736745158 | 21 | 1 | 2 |
591| 736745251 | 21 | 1 | 2 |
592| 736745263 | 21 | 1 | 2 |
593| 736745264 | 21 | 1 | 2 |
594| 736745268 | 21 | 1 | 2 |
595| 736745270 | 21 | 1 | 2 |
596| 736745275 | 21 | 1 | 2 |
597| 736745281 | 21 | 1 | 2 |
598| 736745284 | 21 | 1 | 2 |
599| 736745287 | 21 | 1 | 2 |
600| 736745288 | 21 | 1 | 2 |
601| 736745290 | 21 | 1 | 2 |
602| 736745292 | 21 | 1 | 2 |
603| 736745294 | 21 | 1 | 2 |
604| 736745295 | 21 | 1 | 2 |
605| 736745296 | 21 | 1 | 2 |
606| 736745297 | 21 | 1 | 2 |
607| 736745298 | 21 | 1 | 2 |
608| 736745299 | 21 | 1 | 2 |
609| 736745300 | 21 | 1 | 2 |
610| 736745301 | 21 | 1 | 2 |
611| 736745302 | 21 | 1 | 2 |
612| 736745308 | 21 | 1 | 2 |
613| 736745312 | 21 | 1 | 2 |
614| 736745313 | 21 | 1 | 2 |
615| 736745314 | 21 | 1 | 2 |
616| 736745315 | 21 | 1 | 2 |
617| 736745317 | 21 | 1 | 2 |
618| 736745318 | 21 | 1 | 2 |
619| 736745319 | 21 | 1 | 2 |
620| 736745320 | 21 | 1 | 2 |
621| 736745321 | 21 | 1 | 2 |
622| 736745322 | 21 | 1 | 2 |
623| 736745323 | 21 | 1 | 2 |
624| 736745324 | 21 | 1 | 2 |
625| 736745325 | 21 | 1 | 2 |
626| 736745326 | 21 | 1 | 2 |
627| 736745327 | 21 | 1 | 2 |
628| 736745328 | 21 | 1 | 2 |
629| 736745333 | 21 | 1 | 2 |
630| 736745339 | 21 | 1 | 2 |
631| 736745340 | 21 | 1 | 2 |
632| 736745341 | 21 | 1 | 2 |
633| 736745342 | 21 | 1 | 2 |
634| 736745343 | 21 | 1 | 2 |
635| 736745344 | 21 | 1 | 2 |
636| 736745345 | 21 | 1 | 2 |
637| 736745346 | 21 | 1 | 2 |
638| 736745347 | 21 | 1 | 2 |
639| 736745348 | 21 | 1 | 2 |
640| 736745349 | 21 | 1 | 2 |
641| 736745350 | 21 | 1 | 2 |
642| 736745351 | 21 | 1 | 2 |
643| 736745352 | 21 | 1 | 2 |
644| 736745353 | 21 | 1 | 2 |
645| 736745354 | 21 | 1 | 2 |
646| 736745355 | 21 | 1 | 2 |
647| 736745356 | 21 | 1 | 2 |
648| 736745357 | 21 | 1 | 2 |
649| 736745358 | 21 | 1 | 2 |
650| 736745359 | 21 | 1 | 2 |
651| 736745360 | 21 | 1 | 2 |
652| 736745361 | 21 | 1 | 2 |
653| 736745362 | 21 | 1 | 2 |
654| 736745363 | 21 | 1 | 2 |
655| 736745388 | 21 | 1 | 2 |
656| 736745393 | 21 | 1 | 2 |
657| 736745401 | 21 | 1 | 2 |
658| 736745402 | 21 | 1 | 2 |
659| 736745404 | 21 | 1 | 2 |
660| 736745477 | 21 | 1 | 2 |
661| 736745593 | 21 | 1 | 2 |
662| 736745594 | 21 | 1 | 2 |
663| 736745595 | 21 | 1 | 2 |
664| 736745596 | 21 | 1 | 2 |
665| 736745597 | 21 | 1 | 2 |
666| 736745598 | 21 | 1 | 2 |
667| 736745599 | 21 | 1 | 2 |
668| 736745600 | 21 | 1 | 2 |
669| 736745601 | 21 | 1 | 2 |
670| 736745602 | 21 | 1 | 2 |
671| 736745603 | 21 | 1 | 2 |
672| 736745604 | 21 | 1 | 2 |
673| 736745605 | 21 | 1 | 2 |
674| 736745606 | 21 | 1 | 2 |
675| 736745607 | 21 | 1 | 2 |
676| 736745608 | 21 | 1 | 2 |
677| 736745609 | 21 | 1 | 2 |
678| 736745638 | 21 | 1 | 2 |
679| 736745695 | 21 | 1 | 2 |
680| 736745696 | 21 | 1 | 2 |
681| 736745697 | 21 | 1 | 2 |
682| 736745698 | 21 | 1 | 2 |
683| 736745701 | 21 | 1 | 2 |
684| 736745789 | 21 | 1 | 2 |
685| 736745790 | 21 | 1 | 2 |
686| 736745791 | 21 | 1 | 2 |
687| 736745792 | 21 | 1 | 2 |
688| 736745793 | 21 | 1 | 2 |
689| 736745794 | 21 | 1 | 2 |
690| 736745795 | 21 | 1 | 2 |
691| 736745796 | 21 | 1 | 2 |
692| 736745797 | 21 | 1 | 2 |
693| 736745800 | 21 | 1 | 2 |
694| 736745938 | 21 | 1 | 2 |
695| 736746061 | 21 | 1 | 2 |
696| 736746290 | 21 | 1 | 2 |
697| 736746291 | 21 | 1 | 2 |
698| 736746292 | 21 | 1 | 2 |
699| 736746294 | 21 | 1 | 2 |
700| 736746295 | 21 | 1 | 2 |
701| 736746296 | 21 | 1 | 2 |
702| 736746328 | 21 | 1 | 2 |
703| 736746343 | 21 | 1 | 2 |
704| 736746345 | 21 | 1 | 2 |
705| 736746472 | 21 | 1 | 2 |
706| 736746473 | 21 | 1 | 2 |
707| 736746474 | 21 | 1 | 2 |
708| 736746476 | 21 | 1 | 2 |
709| 736746477 | 21 | 1 | 2 |
710| 736746478 | 21 | 1 | 2 |
711| 736746479 | 21 | 1 | 2 |
712| 736746563 | 21 | 1 | 2 |
713| 736746617 | 21 | 1 | 2 |
714| 736746618 | 21 | 1 | 2 |
715| 736746619 | 21 | 1 | 2 |
716| 736746620 | 21 | 1 | 2 |
717| 736746621 | 21 | 1 | 2 |
718| 736746622 | 21 | 1 | 2 |
719| 736746623 | 21 | 1 | 2 |
720| 736746624 | 21 | 1 | 2 |
721| 736746883 | 21 | 1 | 2 |
722| 736746884 | 21 | 1 | 2 |
723| 736746885 | 21 | 1 | 2 |
724| 736746901 | 21 | 1 | 2 |
725| 736746902 | 21 | 1 | 2 |
726| 736746906 | 21 | 1 | 2 |
727| 736746935 | 21 | 1 | 2 |
728| 736747036 | 21 | 1 | 2 |
729| 736747190 | 21 | 1 | 2 |
730| 736747194 | 21 | 1 | 2 |
731| 736747226 | 21 | 1 | 2 |
732| 736747245 | 21 | 1 | 2 |
733| 736747291 | 21 | 1 | 2 |
734| 736747292 | 21 | 1 | 2 |
735| 736747359 | 21 | 1 | 2 |
736| 736747361 | 21 | 1 | 2 |
737| 736747438 | 21 | 1 | 2 |
738| 736747492 | 21 | 1 | 2 |
739| 736747493 | 21 | 1 | 2 |
740| 736747494 | 21 | 1 | 2 |
741| 736747495 | 21 | 1 | 2 |
742| 736747496 | 21 | 1 | 2 |
743| 736747497 | 21 | 1 | 2 |
744| 736747498 | 21 | 1 | 2 |
745| 736747499 | 21 | 1 | 2 |
746| 736747500 | 21 | 1 | 2 |
747| 736747506 | 21 | 1 | 2 |
748| 736747507 | 21 | 1 | 2 |
749| 736747508 | 21 | 1 | 2 |
750| 736747509 | 21 | 1 | 2 |
751| 736747515 | 21 | 1 | 2 |
752| 736747586 | 21 | 1 | 2 |
753| 736747590 | 21 | 1 | 2 |
754| 736747601 | 21 | 1 | 2 |
755| 736747602 | 21 | 1 | 2 |
756| 736747603 | 21 | 1 | 2 |
757| 736747604 | 21 | 1 | 2 |
758| 736747605 | 21 | 1 | 2 |
759| 736747608 | 21 | 1 | 2 |
760| 736747609 | 21 | 1 | 2 |
761| 736747610 | 21 | 1 | 2 |
762| 736747611 | 21 | 1 | 2 |
763| 736747612 | 21 | 1 | 2 |
764| 736747627 | 21 | 1 | 2 |
765| 736747630 | 21 | 1 | 2 |
766| 736747632 | 21 | 1 | 2 |
767| 736747634 | 21 | 1 | 2 |
768| 736747637 | 21 | 1 | 2 |
769| 736747691 | 21 | 1 | 2 |
770| 736747692 | 21 | 1 | 2 |
771| 736747693 | 21 | 1 | 2 |
772| 736747777 | 21 | 1 | 2 |
773| 736747778 | 21 | 1 | 2 |
774| 736747779 | 21 | 1 | 2 |
775| 736747866 | 21 | 1 | 2 |
776| 736747888 | 21 | 1 | 2 |
777| 736747913 | 21 | 1 | 2 |
778| 736748046 | 21 | 1 | 2 |
779| 736748047 | 21 | 1 | 2 |
780| 736748069 | 21 | 1 | 2 |
781| 736748072 | 21 | 1 | 2 |
782| 736748163 | 21 | 1 | 2 |
783| 736748178 | 21 | 1 | 2 |
784| 736748179 | 21 | 1 | 2 |
785| 736748180 | 21 | 1 | 2 |
786| 736748181 | 21 | 1 | 2 |
787| 736748266 | 21 | 1 | 2 |
788| 736748274 | 21 | 1 | 2 |
789| 736748491 | 21 | 1 | 2 |
790| 736748492 | 21 | 1 | 2 |
791| 736748493 | 21 | 1 | 2 |
792| 736748598 | 21 | 1 | 2 |
793| 736748599 | 21 | 1 | 2 |
794| 736748747 | 21 | 1 | 2 |
795| 736748762 | 21 | 1 | 2 |
796| 736748966 | 21 | 1 | 2 |
797| 736749037 | 21 | 1 | 2 |
798| 736749196 | 21 | 1 | 2 |
799| 736749197 | 21 | 1 | 2 |
800| 736749198 | 21 | 1 | 2 |
801| 736749199 | 21 | 1 | 2 |
802| 736749209 | 21 | 1 | 2 |
803| 736749210 | 21 | 1 | 2 |
804| 736749211 | 21 | 1 | 2 |
805| 736749212 | 21 | 1 | 2 |
806| 736749213 | 21 | 1 | 2 |
807| 736749214 | 21 | 1 | 2 |
808| 736749215 | 21 | 1 | 2 |
809| 736749216 | 21 | 1 | 2 |
810| 736749218 | 21 | 1 | 2 |
811| 736749219 | 21 | 1 | 2 |
812| 736749602 | 21 | 1 | 2 |
813| 736749611 | 21 | 1 | 2 |
814| 736749613 | 21 | 1 | 2 |
815| 736750373 | 21 | 1 | 2 |
816| 736750548 | 21 | 1 | 2 |
817| 736750778 | 21 | 1 | 2 |
818| 736750819 | 21 | 1 | 2 |
819| 736750825 | 21 | 1 | 2 |
820| 736750879 | 21 | 1 | 2 |
821| 736750903 | 21 | 1 | 2 |
822| 736750976 | 21 | 1 | 2 |
823| 736751021 | 21 | 1 | 2 |
824| 736751138 | 21 | 1 | 2 |
825| 736751139 | 21 | 1 | 2 |
826| 736751197 | 21 | 1 | 2 |
827| 736751198 | 21 | 1 | 2 |
828| 736751199 | 21 | 1 | 2 |
829| 736751200 | 21 | 1 | 2 |
830| 736751201 | 21 | 1 | 2 |
831| 736751202 | 21 | 1 | 2 |
832| 736751203 | 21 | 1 | 2 |
833| 736751206 | 21 | 1 | 2 |
834| 736751244 | 21 | 1 | 2 |
835| 736751495 | 21 | 1 | 2 |
836| 736751496 | 21 | 1 | 2 |
837| 736751498 | 21 | 1 | 2 |
838| 736751499 | 21 | 1 | 2 |
839| 736751502 | 21 | 1 | 2 |
840| 736751641 | 21 | 1 | 2 |
841| 736751667 | 21 | 1 | 2 |
842| 736751838 | 21 | 1 | 2 |
843| 736751991 | 21 | 1 | 2 |
844| 736751993 | 21 | 1 | 2 |
845| 736752171 | 21 | 1 | 2 |
846| 736752172 | 21 | 1 | 2 |
847+-----------+-------------+-------------+----------+
848

jcrespo removed jcrespo as the assignee of this task.Sep 13 2016, 10:34 AM
jcrespo moved this task from In progress to Blocked external/Not db team on the DBA board.

Okay, this is too big to be done manually. I think, I should write a maintenance script to do this job.

I blocked myself to wait for your input, I am only standing by, but let me know when you need me again.

I will ask you soon. Thanks.

Halfak renamed this task from Check ORES data violating constraints on Beta do not affect production to Ensure ORES data violating constraints do not affect production.Sep 15 2016, 2:42 PM
Halfak updated the task description. (Show Details)Sep 22 2016, 2:27 PM
Halfak added a subscriber: Halfak.Sep 22 2016, 2:29 PM

Assigning this to @Ladsgroup because it looks like the last thing to do is to run the maintenance script. We can do that once the change for T145503 is merged and deployed. So I'm moving this to the backlog for now.

Halfak assigned this task to Ladsgroup.Sep 22 2016, 2:30 PM
Halfak triaged this task as High priority.
Halfak moved this task from Untriaged to Maintenance/cleanup on the Scoring-platform-team board.
Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptSep 22 2016, 2:31 PM

Can I have a look at it before it is run? It is very easy to create lag by accident, and I could maybe give some tips if that was a possibility?

@jcrespo yes please. I'm confirming now what should be reviewed.

jcrespo added a comment.EditedSep 23 2016, 7:35 AM

Sorry, I checked and there is less than 1000 rows to delete per wiki; things should be done really badly to create lag- and you already did well by separating the reads on a separate replica and deleting using the primary key, so everything is ok. If at some point you do other maintenance ping me, as we should be otherwise more careful.

The only thing I would ask you is to do it during a scheduled deployment time (this should not take more than 5 minutes) in case something else goes wrong, notice it immediately.

Ladsgroup added a comment.EditedOct 11 2016, 5:24 PM
ladsgroup@terbium:~$ mwscript extensions/ORES/maintenance/CleanDuplicateScores.php --wiki=fawiki
Got 2 duplicates, cleaning them.Done
ladsgroup@terbium:~$ mwscript extensions/ORES/maintenance/CleanDuplicateScores.php --wiki=wikidatawiki
Got 4359 duplicates, cleaning them.Done
ladsgroup@terbium:~$ mwscript extensions/ORES/maintenance/CleanDuplicateScores.php --wiki=plwiki
Got 495 duplicates, cleaning them.Done
ladsgroup@terbium:~$ mwscript extensions/ORES/maintenance/CleanDuplicateScores.php --wiki=nlwiki
Got 384 duplicates, cleaning them.Done
ladsgroup@terbium:~$ mwscript extensions/ORES/maintenance/CleanDuplicateScores.php --wiki=ptwiki
Got 1 duplicates, cleaning them.Done
ladsgroup@terbium:~$ mwscript extensions/ORES/maintenance/CleanDuplicateScores.php --wiki=trwiki
Got 312 duplicates, cleaning them.Done
ladsgroup@terbium:~$ mwscript extensions/ORES/maintenance/CleanDuplicateScores.php --wiki=ruwiki
Got 11 duplicates, cleaning them.Done
ladsgroup@terbium:~$ mwscript extensions/ORES/maintenance/CleanDuplicateScores.php --wiki=enwiki
Got 5358 duplicates, cleaning them.Done

Mentioned in SAL (#wikimedia-operations) [2016-10-11T17:25:13Z] <Amir1> ladsgroup@terbium:~$ mwscript extensions/ORES/maintenance/CleanDuplicateScores.php on eight wikis (T145356)

@jcrespo: I ran the maintenance script and then this:

1ladsgroup@terbium:~$ for wiki in fawiki wikidatawiki ruwiki ptwiki trwiki plwiki nlwiki enwiki; do echo "$wiki"; sql $wiki -e "SELECT oresc_rev, oresc_model, oresc_class, count(*) FROM ores_classification GROUP BY oresc_rev, oresc_model, oresc_class HAVING count(*) > 1;SELECT oresc_rev, oresc_model, oresc_class, count(*) FROM ores_classification GROUP BY oresc_rev, oresc_is_predicted, oresc_model HAVING count(*) > 1"; done
2fawiki
3wikidatawiki
4ruwiki
5ptwiki
6trwiki
7plwiki
8nlwiki
9enwiki

Is it okay now? Thanks