Page MenuHomePhabricator

Rules search may display wrong extracts when using regexes containing zero-length assertions
Open, Needs TriagePublic

Description

In https://en.wikipedia.org/wiki/Special:AbuseFilter (more exactly, in a wiki where you have the right to search in filters' code).

Let's assume a filter containing this code exists: fooxxx foobar.

Using the "Search within rules:" input, search for foo(?=bar) regular expression.

Actual:
"foo" from "fooxxx" is displayed and highlighted.

Expected:
"foo" from "foobar" should be displayed and highlighted.

That's because the code currently works as so:

  • there is a first pass using preg_match() for the actual search.
  • then, if there is a match, for highlighting purpose: the captured text (here, "foo") is used for another search… using a plaintext search.

Event Timeline

Change #1071059 had a related patch set uploaded (by Gerrit Patch Uploader; author: Anne Haunime):

[mediawiki/extensions/AbuseFilter@master] Fix possibly wrong extracts when search contains zero-length assertions

https://gerrit.wikimedia.org/r/1071059

Uploaded patch, that fixes this by making use of the PREG_OFFSET_CAPTURE option of preg_match().

On a related note:

The getHighlightedPattern() method relies on the fact that the regex will match for sure (except the "catastrophic backtracking" special case), the exact same search having been run previously, in the matchesPattern() method.

If ever, for any unexpected reason (such as an oversight during code refactoring), the regex were to not match, the following would happen,
in the strlen( $matches[0] ) === 0 part:

  • PHP warning "undefined array index" on the $matches[0], and it gives null.
  • strlen( null ) gives 0, so the equality passes and the function gracefully early returns, except for the PHP warning. (By the way, strlen( null ) (i.e. passing null) is deprecated since PHP 8, so it additionally produces a deprecation warning.)

Replacing that part with $matches[0] === '' would make us lose this "fortuitous graceful fallback".

My patchset 1 contains strlen( $matches[0][0] ) === 0 which keeps the same behavior in that unexcepted case, the only difference being yet one additional PHP warning.

The code might be made robuster. It works fine currently, but it seems to be prone to breakage.

  • We could early return if $check !== 1, i.e. false (that "catastrophic backtracking" special case) or 0 (unexpected no match).
  • We could throw an exception if ever $check === 0, as it is really not expected to happen.

Don't merge yet. Edit: should be fixed in patchset 2.

By complete chance, I just noticed this comment:

Attention! PREG_OFFSET_CAPTURE not UTF-8 aware when using u modifier
and it's not a but, it's a feature:
https://bugs.php.net/bug.php?id=37391

Possible workaround: Use mb_strpos to get the correct offset, instead of the flag.

UTF-8 support would be nice.

See:

That's terrible…

Uploaded patchset 3:

  • Check if preg_match() returned 0 (no match), which is unexcepted as I exposed above, and if ever that happens, throw an exception.
  • Some refactoring, by separating the "preg_match() returned false" and the "matched empty string" cases, as I think it's clearer as so.

About the "preg_match() may return false in case of catastrophic backtracking", I don't really know about this.
More specifically, I don't know if in such case of catastrophic backtracking, preg_match() may inconsistently return different results on repeated calls.
So, in doubt I simply left it as it is.

Note we are coming from an age where the rule search was done in SQL, and the subsequent search for highlighting in PHP. Nowadays, both searches are made in PHP, and with the same code (same preg_match() function, same pattern).

Refs:

  • T87455 - initial implementation of the rule search feature (note the "catastrophic backtracking" concern in the PHP part has been added here)
  • T193068 - converted the rule search from SQL to PHP

Uploaded patchset 4 (… and patchset 5, fuuuuu)

  • Moved the code comment about catastrophic backtracking to the place where it may happen.
  • Tweaked the security check: the second preg_match() (in getHighlightedPattern() method) should always return 1, as the first one (in matchesPattern() method) matched previously. If ever that's not the case, it means a bug has been introduced in the code.

Change #1325940 had a related patch set uploaded (by Dragoniez; author: Dragoniez):

[mediawiki/extensions/AbuseFilter@master] Improve regex matching in AbuseFilterPager

https://gerrit.wikimedia.org/r/1325940