Page MenuHomePhabricator

Dry-run, then actually run updateVarDumps
Open, Needs TriagePublic

Description

What: Dry-run, audit results, then run for real the updateVarDumps.php script for AbuseFilter
When: Whenever this patch reaches production.
Why: To kill many layers of back-compat and a lot of tech debt (see parent task).
Where: All wikis - beta cluster first, then production.
Who: Whoever wants to do that. (@Daimona can do the Beta Cluster part).

How:

  1. First of all, run updateVarDumps.php --dry-run and post the results.
  2. @Daimona will audit the results
  3. If necessary, we may have to re-run the script with --dry-run-verbose for selected wikis, then repeat (2.)
  4. Find a DateTime suitable for both of us
  5. Time to run it for real. The ideal steps are:
    1. Measure the size of the abuse_filter_log table (on enwiki?); measure the size of the ExternalStore (I don't know whether it's feasible)
    2. Run the script!
    3. Measure the sizes again and check how much space was saved.
  6. Done!

Details

Related Gerrit Patches:
mediawiki/extensions/AbuseFilter : masterupdateVarDumps: Print orphaned ES records, don't try updating ES records
mediawiki/extensions/AbuseFilter : masterupdateVarDumps: Use new ExternalStore method for updating records
mediawiki/extensions/AbuseFilter : wmf/1.35.0-wmf.22Minor fixes for the updateVarDumps script
mediawiki/extensions/AbuseFilter : masterMinor fixes for the updateVarDumps script

Event Timeline

Daimona created this task.Feb 29 2020, 4:52 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 29 2020, 4:52 PM

Dry-run on the beta cluster:

1daimona@deployment-deploy01:/srv/mediawiki/php-master$ foreachwiki extensions/AbuseFilter/maintenance/updateVarDumps.php --dry-run
2-----------------------------------------------------------------
3aawiki
4-----------------------------------------------------------------
5aawiki: ...the abuse_filter_log table is empty.
6-----------------------------------------------------------------
7arwiki
8-----------------------------------------------------------------
9arwiki: ...Checking for missing dumps (1/4)
10arwiki: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
11arwiki: ...Moving serialized data away from the abuse_filter_log table (2/4).
12arwiki: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
13arwiki: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
14arwiki: ...found 29 text rows to update.
15arwiki: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
16arwiki: ...would change afl_var_dump for 42 rows.
17-----------------------------------------------------------------
18cawiki
19-----------------------------------------------------------------
20cawiki: ...Checking for missing dumps (1/4)
21cawiki: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
22cawiki: ...Moving serialized data away from the abuse_filter_log table (2/4).
23cawiki: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
24cawiki: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
25cawiki: ...found 3 text rows to update.
26cawiki: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
27cawiki: ...would change afl_var_dump for 3 rows.
28-----------------------------------------------------------------
29commonswiki
30-----------------------------------------------------------------
31commonswiki: ...Checking for missing dumps (1/4)
32commonswiki: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
33commonswiki: ...Moving serialized data away from the abuse_filter_log table (2/4).
34commonswiki: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
35commonswiki: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
36commonswiki: ...found 3080 text rows to update.
37commonswiki: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
38commonswiki: ...would change afl_var_dump for 3640 rows.
39-----------------------------------------------------------------
40crhwiki
41-----------------------------------------------------------------
42crhwiki: ...the abuse_filter_log table is empty.
43-----------------------------------------------------------------
44cswiki
45-----------------------------------------------------------------
46cswiki: ...Checking for missing dumps (1/4)
47cswiki: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
48cswiki: ...Moving serialized data away from the abuse_filter_log table (2/4).
49cswiki: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
50cswiki: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
51cswiki: ...found 1 text rows to update.
52cswiki: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
53cswiki: ...would change afl_var_dump for 1 rows.
54-----------------------------------------------------------------
55deploymentwiki
56-----------------------------------------------------------------
57deploymentwiki: ...Checking for missing dumps (1/4)
58deploymentwiki: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
59deploymentwiki: ...Moving serialized data away from the abuse_filter_log table (2/4).
60deploymentwiki: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
61deploymentwiki: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
62deploymentwiki: ...found 1608 text rows to update.
63deploymentwiki: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
64deploymentwiki: ...would change afl_var_dump for 1820 rows.
65-----------------------------------------------------------------
66dewiki
67-----------------------------------------------------------------
68dewiki: ...Checking for missing dumps (1/4)
69dewiki: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
70dewiki: ...Moving serialized data away from the abuse_filter_log table (2/4).
71dewiki: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
72dewiki: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
73dewiki: ...found 72 text rows to update.
74dewiki: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
75dewiki: ...would change afl_var_dump for 74 rows.
76-----------------------------------------------------------------
77dewiktionary
78-----------------------------------------------------------------
79dewiktionary: ...the abuse_filter_log table is empty.
80-----------------------------------------------------------------
81en_rtlwiki
82-----------------------------------------------------------------
83en_rtlwiki: ...the abuse_filter_log table is empty.
84-----------------------------------------------------------------
85enwiki
86-----------------------------------------------------------------
87enwiki: ...Checking for missing dumps (1/4)
88enwiki: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
89enwiki: ...Moving serialized data away from the abuse_filter_log table (2/4).
90enwiki: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
91enwiki: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
92enwiki: ...found 75549 text rows to update.
93enwiki: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
94enwiki: ...would change afl_var_dump for 75970 rows.
95-----------------------------------------------------------------
96enwikibooks
97-----------------------------------------------------------------
98enwikibooks: ...the abuse_filter_log table is empty.
99-----------------------------------------------------------------
100enwikinews
101-----------------------------------------------------------------
102enwikinews: ...the abuse_filter_log table is empty.
103-----------------------------------------------------------------
104enwikiquote
105-----------------------------------------------------------------
106enwikiquote: ...the abuse_filter_log table is empty.
107-----------------------------------------------------------------
108enwikisource
109-----------------------------------------------------------------
110enwikisource: ...Checking for missing dumps (1/4)
111enwikisource: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
112enwikisource: ...Moving serialized data away from the abuse_filter_log table (2/4).
113enwikisource: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
114enwikisource: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
115enwikisource: ...found 2 text rows to update.
116enwikisource: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
117enwikisource: ...would change afl_var_dump for 2 rows.
118-----------------------------------------------------------------
119enwikiversity
120-----------------------------------------------------------------
121enwikiversity: ...Checking for missing dumps (1/4)
122enwikiversity: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
123enwikiversity: ...Moving serialized data away from the abuse_filter_log table (2/4).
124enwikiversity: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
125enwikiversity: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
126enwikiversity: ...found 3 text rows to update.
127enwikiversity: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
128enwikiversity: ...would change afl_var_dump for 4 rows.
129-----------------------------------------------------------------
130enwikivoyage
131-----------------------------------------------------------------
132enwikivoyage: ...Checking for missing dumps (1/4)
133enwikivoyage: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
134enwikivoyage: ...Moving serialized data away from the abuse_filter_log table (2/4).
135enwikivoyage: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
136enwikivoyage: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
137enwikivoyage: ...found 4 text rows to update.
138enwikivoyage: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
139enwikivoyage: ...would change afl_var_dump for 6 rows.
140-----------------------------------------------------------------
141enwiktionary
142-----------------------------------------------------------------
143enwiktionary: ...Checking for missing dumps (1/4)
144enwiktionary: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
145enwiktionary: ...Moving serialized data away from the abuse_filter_log table (2/4).
146enwiktionary: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
147enwiktionary: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
148enwiktionary: ...found 6 text rows to update.
149enwiktionary: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
150enwiktionary: ...would change afl_var_dump for 6 rows.
151-----------------------------------------------------------------
152eowiki
153-----------------------------------------------------------------
154eowiki: ...Checking for missing dumps (1/4)
155eowiki: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
156eowiki: ...Moving serialized data away from the abuse_filter_log table (2/4).
157eowiki: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
158eowiki: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
159eowiki: ...found 1 text rows to update.
160eowiki: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
161eowiki: ...would change afl_var_dump for 1 rows.
162-----------------------------------------------------------------
163eswiki
164-----------------------------------------------------------------
165eswiki: ...the abuse_filter_log table is empty.
166-----------------------------------------------------------------
167eswikibooks
168-----------------------------------------------------------------
169eswikibooks: ...Checking for missing dumps (1/4)
170eswikibooks: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
171eswikibooks: ...Moving serialized data away from the abuse_filter_log table (2/4).
172eswikibooks: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
173eswikibooks: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
174eswikibooks: ...found 6 text rows to update.
175eswikibooks: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
176eswikibooks: ...would change afl_var_dump for 6 rows.
177-----------------------------------------------------------------
178fawiki
179-----------------------------------------------------------------
180fawiki: ...the abuse_filter_log table is empty.
181-----------------------------------------------------------------
182hewiki
183-----------------------------------------------------------------
184hewiki: ...Checking for missing dumps (1/4)
185hewiki: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
186hewiki: ...Moving serialized data away from the abuse_filter_log table (2/4).
187hewiki: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
188hewiki: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
189hewiki: ...found 4 text rows to update.
190hewiki: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
191hewiki: ...would change afl_var_dump for 6 rows.
192-----------------------------------------------------------------
193hewiktionary
194-----------------------------------------------------------------
195hewiktionary: ...the abuse_filter_log table is empty.
196-----------------------------------------------------------------
197hiwiki
198-----------------------------------------------------------------
199hiwiki: ...the abuse_filter_log table is empty.
200-----------------------------------------------------------------
201jawiki
202-----------------------------------------------------------------
203jawiki: ...Checking for missing dumps (1/4)
204jawiki: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
205jawiki: ...Moving serialized data away from the abuse_filter_log table (2/4).
206jawiki: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
207jawiki: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
208jawiki: ...found 1 text rows to update.
209jawiki: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
210jawiki: ...would change afl_var_dump for 1 rows.
211-----------------------------------------------------------------
212kowiki
213-----------------------------------------------------------------
214kowiki: ...Checking for missing dumps (1/4)
215kowiki: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
216kowiki: ...Moving serialized data away from the abuse_filter_log table (2/4).
217kowiki: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
218kowiki: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
219kowiki: ...found 5 text rows to update.
220kowiki: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
221kowiki: ...would change afl_var_dump for 5 rows.
222-----------------------------------------------------------------
223loginwiki
224-----------------------------------------------------------------
225loginwiki: ...the abuse_filter_log table is empty.
226-----------------------------------------------------------------
227metawiki
228-----------------------------------------------------------------
229metawiki: ...Checking for missing dumps (1/4)
230metawiki: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
231metawiki: ...Moving serialized data away from the abuse_filter_log table (2/4).
232metawiki: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
233metawiki: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
234metawiki: ...found 46 text rows to update.
235metawiki: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
236metawiki: ...would change afl_var_dump for 61 rows.
237-----------------------------------------------------------------
238nlwiki
239-----------------------------------------------------------------
240nlwiki: ...Checking for missing dumps (1/4)
241nlwiki: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
242nlwiki: ...Moving serialized data away from the abuse_filter_log table (2/4).
243nlwiki: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
244nlwiki: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
245nlwiki: ...found 2 text rows to update.
246nlwiki: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
247nlwiki: ...would change afl_var_dump for 2 rows.
248-----------------------------------------------------------------
249ruwiki
250-----------------------------------------------------------------
251ruwiki: ...Checking for missing dumps (1/4)
252ruwiki: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
253ruwiki: ...Moving serialized data away from the abuse_filter_log table (2/4).
254ruwiki: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
255ruwiki: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
256ruwiki: ...found 2 text rows to update.
257ruwiki: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
258ruwiki: ...would change afl_var_dump for 2 rows.
259-----------------------------------------------------------------
260simplewiki
261-----------------------------------------------------------------
262simplewiki: ...Checking for missing dumps (1/4)
263simplewiki: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
264simplewiki: ...Moving serialized data away from the abuse_filter_log table (2/4).
265simplewiki: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
266simplewiki: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
267simplewiki: ...found 17 text rows to update.
268simplewiki: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
269simplewiki: ...would change afl_var_dump for 20 rows.
270-----------------------------------------------------------------
271sqwiki
272-----------------------------------------------------------------
273sqwiki: ...the abuse_filter_log table is empty.
274-----------------------------------------------------------------
275srwiki
276-----------------------------------------------------------------
277srwiki: ...Checking for missing dumps (1/4)
278srwiki: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
279srwiki: ...Moving serialized data away from the abuse_filter_log table (2/4).
280srwiki: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
281srwiki: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
282srwiki: ...found 8 text rows to update.
283srwiki: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
284srwiki: ...would change afl_var_dump for 8 rows.
285-----------------------------------------------------------------
286svwiki
287-----------------------------------------------------------------
288svwiki: ...Checking for missing dumps (1/4)
289svwiki: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
290svwiki: ...Moving serialized data away from the abuse_filter_log table (2/4).
291svwiki: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
292svwiki: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
293svwiki: ...found 0 text rows to update.
294svwiki: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
295svwiki: ...would change afl_var_dump for 0 rows.
296-----------------------------------------------------------------
297testwiki
298-----------------------------------------------------------------
299testwiki: ...Checking for missing dumps (1/4)
300testwiki: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
301testwiki: ...Moving serialized data away from the abuse_filter_log table (2/4).
302testwiki: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
303testwiki: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
304testwiki: ...found 148 text rows to update.
305testwiki: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
306testwiki: ...would change afl_var_dump for 150 rows.
307-----------------------------------------------------------------
308ukwiki
309-----------------------------------------------------------------
310ukwiki: ...the abuse_filter_log table is empty.
311-----------------------------------------------------------------
312viwiki
313-----------------------------------------------------------------
314viwiki: ...Checking for missing dumps (1/4)
315viwiki: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
316viwiki: ...Moving serialized data away from the abuse_filter_log table (2/4).
317viwiki: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
318viwiki: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
319viwiki: ...found 0 text rows to update.
320viwiki: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
321viwiki: ...would change afl_var_dump for 0 rows.
322-----------------------------------------------------------------
323wikidatawiki
324-----------------------------------------------------------------
325wikidatawiki: ...Checking for missing dumps (1/4)
326wikidatawiki: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
327wikidatawiki: ...Moving serialized data away from the abuse_filter_log table (2/4).
328wikidatawiki: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
329wikidatawiki: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
330wikidatawiki: ...found 320 text rows to update.
331wikidatawiki: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
332wikidatawiki: ...would change afl_var_dump for 320 rows.
333-----------------------------------------------------------------
334zhwiki
335-----------------------------------------------------------------
336zhwiki: ...Checking for missing dumps (1/4)
337zhwiki: ...found 0 rows with blank afl_var_dump to delete, and 0 rows to rebuild.
338zhwiki: ...Moving serialized data away from the abuse_filter_log table (2/4).
339zhwiki: ...found 0 abuse_filter_log rows with serialized data and 0 truncated dumps to rebuild.
340zhwiki: ...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
341zhwiki: ...found 119 text rows to update.
342zhwiki: ...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
343zhwiki: ...would change afl_var_dump for 127 rows.
344-----------------------------------------------------------------
345zhwikivoyage
346-----------------------------------------------------------------
347zhwikivoyage: ...the abuse_filter_log table is empty.

Comments:

  • The results look good!
  • No fatals and no warnings/notices on logstash-beta!
  • The number of rows to update in the last two phases can be quite high (it's roughly the whole abuse_filter_log table), but the script uses batching.
  • The script looked pretty fast, at least in dry-run.
  • Phases 1 and 2 returned 0 affected rows for all wikis.

Since I have no obvious way to tell if things are working as expected, I'm going to run the script on a mildly affected (i.e. <10 rows) wiki and fiddle around to ensure that nothing broke catastrophically. Then I'll run it foreachwiki, and then it should be ready for prod.

Daimona updated the task description. (Show Details)Tue, Mar 3, 3:11 PM

Mentioned in SAL (#wikimedia-releng) [2020-03-03T15:23:14Z] <Daimona> mwscript extensions/AbuseFilter/maintenance/updateVarDumps.php --wiki=eswikibooks | T246539

I chose to run on eswikibooks first, because it only had 6 affected rows, and an existing local filter with a few hits. The script executed in a split second.

daimona@deployment-deploy01:/srv/mediawiki/php-master$ mwscript extensions/AbuseFilter/maintenance/updateVarDumps.php --wiki=eswikibooks
...Checking for missing dumps (1/4)
...deleted 0 rows with blank afl_var_dump, and rebuilt 0 rows.
...Moving serialized data away from the abuse_filter_log table (2/4).
...moved 0 abuse_filter_log rows and rebuilt 0 truncated dumps.
...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
...updated 6 text rows.
...Replacing the 'stored-text:' prefix with 'tt:' (4/4).
...updated afl_var_dump prefix for 6 rows.

Unfortunately, something went wrong with old entries, like https://es.wikibooks.beta.wmflabs.org/wiki/Especial:RegistroAbusos/4:

[Xl525awQBHcAACftOQsAAACK] /wiki/Especial:RegistroAbusos/4 Error from line 659 of /srv/mediawiki/php-master/extensions/AbuseFilter/includes/special/SpecialAbuseLog.php: Call to a member function dumpAllVars() on boolean

Backtrace:

#0 /srv/mediawiki/php-master/extensions/AbuseFilter/includes/special/SpecialAbuseLog.php(131): SpecialAbuseLog->showDetails(string)
#1 /srv/mediawiki/php-master/includes/specialpage/SpecialPage.php(575): SpecialAbuseLog->execute(string)
#2 /srv/mediawiki/php-master/includes/specialpage/SpecialPageFactory.php(621): SpecialPage->run(string)
#3 /srv/mediawiki/php-master/includes/MediaWiki.php(299): MediaWiki\SpecialPage\SpecialPageFactory->executePath(Title, RequestContext)
#4 /srv/mediawiki/php-master/includes/MediaWiki.php(972): MediaWiki->performRequest()
#5 /srv/mediawiki/php-master/includes/MediaWiki.php(535): MediaWiki->main()
#6 /srv/mediawiki/php-master/index.php(47): MediaWiki->run()
#7 /srv/mediawiki/w/index.php(3): require(string)
#8 {main}

I'm going to investigate what happened

Change 576405 had a related patch set uploaded (by Daimona Eaytoy; owner: Daimona Eaytoy):
[mediawiki/extensions/AbuseFilter@master] Minor fixes for the updateVarDumps script

https://gerrit.wikimedia.org/r/576405

Mentioned in SAL (#wikimedia-releng) [2020-03-03T18:22:11Z] <Daimona> Manually fixing eswikibooks DB rows with P10600 refs T246539

Change 576405 merged by jenkins-bot:
[mediawiki/extensions/AbuseFilter@master] Minor fixes for the updateVarDumps script

https://gerrit.wikimedia.org/r/576405

Change 576421 had a related patch set uploaded (by Jforrester; owner: Daimona Eaytoy):
[mediawiki/extensions/AbuseFilter@wmf/1.35.0-wmf.22] Minor fixes for the updateVarDumps script

https://gerrit.wikimedia.org/r/576421

Change 576421 merged by jenkins-bot:
[mediawiki/extensions/AbuseFilter@wmf/1.35.0-wmf.22] Minor fixes for the updateVarDumps script

https://gerrit.wikimedia.org/r/576421

Mentioned in SAL (#wikimedia-operations) [2020-03-03T19:38:47Z] <jforrester@deploy1001> Synchronized php-1.35.0-wmf.22/extensions/AbuseFilter: T213006 T246539: Minor fixes for the updateVarDumps script (duration: 01m 05s)

I tried again on beta-hewiki, which is a great testing ground because it has:

  • A hit from a global filter
  • A few old-style log entries from local filters
  • Some new-style entries (added manually)

So I tried running the script there, however:

daimona@deployment-deploy01:/srv/mediawiki/php-master$ mwscript extensions/AbuseFilter/maintenance/updateVarDumps.php --wiki=hewiki
...Checking for missing dumps (1/4)
...deleted 0 rows with blank afl_var_dump, and rebuilt 0 rows.
...Moving serialized data away from the abuse_filter_log table (2/4).
...moved 0 abuse_filter_log rows and rebuilt 0 truncated dumps.
...Re-storing serialized dumps as JSON-encoded arrays for all rows (3/4).
InvalidArgumentException from line 190 of /srv/mediawiki-staging/php-master/includes/libs/rdbms/lbfactory/LBFactoryMulti.php: Unknown cluster 'cluster1/256'
#0 /srv/mediawiki-staging/php-master/includes/libs/rdbms/lbfactory/LBFactoryMulti.php(207): Wikimedia\Rdbms\LBFactoryMulti->newExternalLB('cluster1/256', 240829879)
#1 /srv/mediawiki-staging/php-master/includes/externalstore/ExternalStoreDB.php(151): Wikimedia\Rdbms\LBFactoryMulti->getExternalLB('cluster1/256')
#2 /srv/mediawiki-staging/php-master/includes/externalstore/ExternalStoreDB.php(190): ExternalStoreDB->getLoadBalancer('cluster1/256')
#3 /srv/mediawiki-staging/php-master/includes/externalstore/ExternalStoreDB.php(116): ExternalStoreDB->getMaster('cluster1/256')
#4 /srv/mediawiki-staging/php-master/includes/externalstore/ExternalStore.php(104): ExternalStoreDB->store('cluster1/256', '\xEDX[\x8F\xAB6\x10\xFE+\xC8\xD2y)9\t...')
#5 /srv/mediawiki-staging/php-master/extensions/AbuseFilter/maintenance/updateVarDumps.php(509): ExternalStore::insert('DB://cluster1/2...', '\xEDX[\x8F\xAB6\x10\xFE+\xC8\xD2y)9\t...')
#6 /srv/mediawiki-staging/php-master/extensions/AbuseFilter/maintenance/updateVarDumps.php(457): UpdateVarDumps->doUpdateText(Object(Wikimedia\Rdbms\ResultWrapper))
#7 /srv/mediawiki-staging/php-master/extensions/AbuseFilter/maintenance/updateVarDumps.php(88): UpdateVarDumps->updateText()
#8 /srv/mediawiki-staging/php-master/maintenance/Maintenance.php(1759): UpdateVarDumps->doDBUpdates()
#9 /srv/mediawiki-staging/php-master/maintenance/doMaintenance.php(99): LoggedUpdateMaintenance->execute()
#10 /srv/mediawiki-staging/php-master/extensions/AbuseFilter/maintenance/updateVarDumps.php(650): require_once('/srv/mediawiki-...')
#11 /srv/mediawiki-staging/multiversion/MWScript.php(101): require_once('/srv/mediawiki-...')
#12 {main}

That line of code is trying to replace the content in the same ExternalStore entry. However, it doesn't seem to be possible to do that this way. So:

  • If it's possible to replace the content, I'll re-implement it the proper way
  • If it's not possible, we can just re-store the data; however, that would live an orphaned record for the old entry.

Change 577244 had a related patch set uploaded (by Daimona Eaytoy; owner: Daimona Eaytoy):
[mediawiki/extensions/AbuseFilter@master] updateVarDumps: Use new ExternalStore method for updating records

https://gerrit.wikimedia.org/r/577244

Change 577244 abandoned by Daimona Eaytoy:
updateVarDumps: Use new ExternalStore method for updating records

Reason:
Per core patch

https://gerrit.wikimedia.org/r/577244

Change 579274 had a related patch set uploaded (by Daimona Eaytoy; owner: Daimona Eaytoy):
[mediawiki/extensions/AbuseFilter@master] updateVarDumps: Print orphaned ES records, don't try updating ES records

https://gerrit.wikimedia.org/r/579274

Tagging for 1.35, ideally we should get the script working and added to update.php before the release. Running the script on WMF wikis is not included per se, but de facto that's how we're gonna see if the script is bug-free.

Daimona changed the task status from Stalled to Open.Thu, Mar 26, 12:33 PM