Maniphest T193894

Run listed queries for all wikis in order to unblock existing changes
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Daimona
	May 4 2018, 5:26 PM

Description

On AbuseFilter workboard we have some tasks blocked by the need of a database queries. In fact, in those tasks we're changing rules syntax and this requires that at the moment of the change some things shouldn't be used anywhere.
List of queries

For T191715:

SELECT af_id FROM abuse_filter WHERE af_pattern RLIKE '(context|(global|local)_log_ids) *:='

For T190639:

SELECT af_id FROM abuse_filter WHERE af_pattern RLIKE 'string\\(|\\\\n[\'"]'

For T187973:

SELECT af_id FROM abuse_filter WHERE af_pattern RLIKE 'article_(text|prefixedtext|articleid|restrictions_(edit|move|create|upload)|(recent|first)_contributor)|moved_(to|from)_(text|prefixedtext|articleid)'

And for T47301, only for thwiki we'd need the data in text table with old_id = 4626692, but that could be done with fetchText.php.

For all of these queries (apart from the last one), we'd need the wikiID where the filter is, together with filter IDs where a specific problem is found. Many thanks.

Related Objects

Mentioned In: T230262: Run DB query in production for abuse filters using variable variables
T211042: Volunteer NDA for Daimona
T202095: Require that CentralAuth's global groups all use lowercase internal identifiers
T74329: "added_lines" is not in "new_wikitext" anymore and causes false positives
T156096: Deprecate and then reject empty operators
T191978: Devise a process for finding and fixing filters that will be affected by changes in AbuseFilter
Mentioned Here: T192389: Make external variables true booleans
T47301: non-reproducible False positive AbuseFilter bug
T187973: Find all uses of deprecated AbuseFilter variable names on WMF wikis
T190639: Fix string cast behaviour for lists
T191715: Clearly reserve system variables in VariableHolder

Event Timeline

Daimona created this task.May 4 2018, 5:26 PM

Restricted Application added subscribers: • OldUser02, Aklapper. · View Herald TranscriptMay 4 2018, 5:26 PM

Daimona added a project: Wikimedia-Site-requests.May 4 2018, 5:28 PM

Daimona mentioned this in T191978: Devise a process for finding and fixing filters that will be affected by changes in AbuseFilter.

For T191715: there are no records for any wiki
For T190639: https://phabricator.wikimedia.org/P7088
For T187973: https://phabricator.wikimedia.org/P7089

Reopen the task if you need something else!

@Marostegui Many thanks, exactly what we needed! Just one more thing: could you please provide the last one for thwiki? It should be much quicker than the others, too :-) Thanks again!

root@db2035.codfw.wmnet[thwiki]> select * from text where old_id=4626692;
+---------+---------------+-----------+-----------------------+-------------+----------+---------------+---------------+----------------+---------------+-------------------+
| old_id  | old_namespace | old_title | old_text              | old_comment | old_user | old_user_text | old_timestamp | old_minor_edit | old_flags     | inverse_timestamp |
+---------+---------------+-----------+-----------------------+-------------+----------+---------------+---------------+----------------+---------------+-------------------+
| 4626692 |             0 |           | DB://cluster24/189938 |             |        0 |               |               |              0 | gzip,external |                   |
+---------+---------------+-----------+-----------------------+-------------+----------+---------------+---------------+----------------+---------------+-------------------+
1 row in set (0.04 sec)

@Marostegui sounds like we really need fetchText.php, since this way the data is unreadable. Could you please try with it? Thanks, and sorry for bothering :-)

@Daimona that should be run with a developer with rights for that. We do not normally do those ;-)
Sorry!

Oh, sorry, I wasn't aware of it. Thanks anyway for the nice job :-)

Huji awarded a token.May 7 2018, 1:42 PM

@Marostegui, sorry again, could you please run the following?

SELECT af_id FROM abuse_filter WHERE af_pattern RLIKE '\\\\n[\'"]'

It's the same as the second one, where however I had included too much stuff. Thanks again!

Apart from the query above, we'd kindly need the query below for T192389 as well.

SELECT af_id FROM abuse_filter WHERE af_pattern LIKE '%tor_exit_node%'

In T193894#4203093, @Daimona wrote:
Apart from the query above, we'd kindly need the query below for T192389 as well.
SELECT af_id FROM abuse_filter WHERE af_pattern LIKE '%tor_exit_node%'

And please don't post this one publicly.

In T193894#4195147, @Daimona wrote:
@Marostegui, sorry again, could you please run the following?
SELECT af_id FROM abuse_filter WHERE af_pattern RLIKE '\\\\n[\'"]'
It's the same as the second one, where however I had included too much stuff. Thanks again!

https://phabricator.wikimedia.org/P7122

In T193894#4203236, @Huji wrote:
In T193894#4203093, @Daimona wrote:
Apart from the query above, we'd kindly need the query below for T192389 as well.
SELECT af_id FROM abuse_filter WHERE af_pattern LIKE '%tor_exit_node%'
And please don't post this one publicly.

This one has a NDA paste: https://phabricator.wikimedia.org/P7123

Marostegui closed this task as Resolved.May 14 2018, 5:42 AM

Marostegui claimed this task.

@Marostegui Ehm, unfortunately I can't see it. May it have a more relaxed policy? After all, being global abusefilter helper, this is some info that I can already manually access. Thanks anyway :)

In T193894#4203586, @Daimona wrote:

@Marostegui Ehm, unfortunately I can't see it. May it have a more relaxed policy? After all, being global abusefilter helper, this is some info that I can already manually access. Thanks anyway :)

I am not sure how to handle that, not sure what you can and can't see so by default I prefer to make it NDA access only. Who can advise on this?
@MoritzMuehlenhoff maybe?

@Daimona I have talked to @MoritzMuehlenhoff and @Legoktm to clarify a few things - can you access the paste now? I have made you a subscriber.

Yes, I confirm I can see it. Many thanks

• Vvjjkkii renamed this task from Run listed queries for all wikis in order to unblock existing changes to dldaaaaaaa.Jul 1 2018, 1:11 AM

• Vvjjkkii reopened this task as Open.

• Vvjjkkii removed Marostegui as the assignee of this task.

• Vvjjkkii triaged this task as High priority.

• Vvjjkkii edited projects, added CheckUser, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), Tamil-Sites, Gamepress, Hashtags, Jade, KartoEditor, Language-2018-Apr-June, New-Editor-Experiences, Mail, TCB-Team (now WMDE-TechWish); removed AbuseFilter.

• Vvjjkkii updated the task description. (Show Details)

• Vvjjkkii removed subscribers: Huji, Aklapper.

Restricted Application added a subscriber: Dereckson. · View Herald TranscriptJul 1 2018, 1:11 AM

Daimona renamed this task from dldaaaaaaa to Run listed queries for all wikis in order to unblock existing changes.Jul 1 2018, 9:28 AM

Daimona closed this task as Resolved.

Daimona assigned this task to Marostegui.

Daimona raised the priority of this task from High to Needs Triage.

Daimona edited projects, added AbuseFilter; removed TCB-Team (now WMDE-TechWish), Mail, New-Editor-Experiences, Language-2018-Apr-June, KartoEditor, Jade, Hashtags, Gamepress, Tamil-Sites, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), CheckUser.

Daimona updated the task description. (Show Details)

Daimona added subscribers: Huji, Aklapper.

Daimona mentioned this in T156096: Deprecate and then reject empty operators.Jul 9 2018, 11:32 PM

Daimona mentioned this in T74329: "added_lines" is not in "new_wikitext" anymore and causes false positives.Jul 27 2018, 1:25 PM

Daimona mentioned this in T202095: Require that CentralAuth's global groups all use lowercase internal identifiers.Aug 17 2018, 10:17 AM

Daimona mentioned this in T211042: Volunteer NDA for Daimona.Dec 3 2018, 6:37 PM

Daimona mentioned this in T230262: Run DB query in production for abuse filters using variable variables.Aug 10 2019, 12:03 PM

Run listed queries for all wikis in order to unblock existing changesClosed, ResolvedPublicActions

Description

Related Objects

Event Timeline

Run listed queries for all wikis in order to unblock existing changes
Closed, ResolvedPublic
Actions