Page MenuHomePhabricator

[SW] Update format constraint regex checks to stop errors from shellbox-constraints in the logs
Closed, ResolvedPublic

Description

As SRE
I don’t want to see errors from shellbox-constraints in the logs
and I don’t want retried requests to increase load.
So that we do not spend unnecessary resources

A/C

  • No "preg_match" errors are included in the logs

Notes

Planned changes:

  • When using shellbox, don’t directly call preg_match(), but instead send along a small snippet of PHP code that wraps the call and detects errors and returns them over the RPC interface (T362084#9700099, last part)
  • Instead of replacing / with \/ in the pattern, try using a different delimiter that makes additional escaping inside the pattern unnecessary (T362084#10113028)
  • Add a configuration setting for known-good patterns (default empty) and check these patterns directly using preg_match() without shellbox, and configure common patterns for that setting in production (T214378#9892636)

Investigation Task: T362084
Related Task: T214378

Event Timeline

Steps to get started with this task:

  • set up WikibaseQualityConstraints per its README, including ImportConstraintStatements
  • create or select a string-typed property
  • add a statement “property constraint: format constraint” with qualifier “format as a regular expression” set to, e.g., a\/b
  • add a statement with the property itself and a value like a/b or xy
  • see if a constraint check for the property itself (e.g. API sandbox, wbcheckconstraints) works as expected

Probably needs some more polishing though.

Change #1105782 had a related patch set uploaded (by Audrey Penven; author: Audrey Penven):

[mediawiki/libs/Shellbox@master] Check that error level should be handled before throwing

https://gerrit.wikimedia.org/r/1105782

Change #1105786 had a related patch set uploaded (by Audrey Penven; author: Audrey Penven):

[mediawiki/extensions/WikibaseQualityConstraints@master] [WIP] stop shellbox errors when checking regex

https://gerrit.wikimedia.org/r/1105786

Change #1105935 had a related patch set uploaded (by Audrey Penven; author: Audrey Penven):

[mediawiki/extensions/WikibaseQualityConstraints@master] [WIP] check known-good regexes directly

https://gerrit.wikimedia.org/r/1105935

Change #1105782 merged by jenkins-bot:

[mediawiki/libs/Shellbox@master] Check that error level should be handled before throwing

https://gerrit.wikimedia.org/r/1105782

Change #1108762 had a related patch set uploaded (by Audrey Penven; author: Audrey Penven):

[operations/deployment-charts@master] shellbox: release image 2025-01-07-141744

https://gerrit.wikimedia.org/r/1108762

Change #1108762 merged by jenkins-bot:

[operations/deployment-charts@master] shellbox: release image 2025-01-07-141744

https://gerrit.wikimedia.org/r/1108762

Change #1105786 merged by jenkins-bot:

[mediawiki/extensions/WikibaseQualityConstraints@master] Silence preg_match warnings on Shellbox

https://gerrit.wikimedia.org/r/1105786

Now waiting for deployment window, I think – once it’s deployed (via backport or train), we’ll want to put it back in development for creating a configuration change with common known-good regexes (e.g. based on this query).

Change #1105935 merged by jenkins-bot:

[mediawiki/extensions/WikibaseQualityConstraints@master] Check known-good regex patterns directly

https://gerrit.wikimedia.org/r/1105935

Change #1111949 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Audrey Penven):

[mediawiki/extensions/WikibaseQualityConstraints@wmf/1.44.0-wmf.12] Check known-good regex patterns directly

https://gerrit.wikimedia.org/r/1111949

Change #1111949 merged by jenkins-bot:

[mediawiki/extensions/WikibaseQualityConstraints@wmf/1.44.0-wmf.12] Check known-good regex patterns directly

https://gerrit.wikimedia.org/r/1111949

Mentioned in SAL (#wikimedia-operations) [2025-01-16T15:29:45Z] <lucaswerkmeister-wmde@deploy2002> Started scap sync-world: Backport for [[gerrit:1111949|Check known-good regex patterns directly (T380751)]]

Mentioned in SAL (#wikimedia-operations) [2025-01-16T15:36:21Z] <lucaswerkmeister-wmde@deploy2002> lucaswerkmeister-wmde: Backport for [[gerrit:1111949|Check known-good regex patterns directly (T380751)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2025-01-16T15:44:18Z] <lucaswerkmeister-wmde@deploy2002> Finished scap sync-world: Backport for [[gerrit:1111949|Check known-good regex patterns directly (T380751)]] (duration: 14m 32s)

Change #1112261 had a related patch set uploaded (by Audrey Penven; author: Audrey Penven):

[operations/mediawiki-config@master] Add known-good regexes for WikibaseQualityConstraints

https://gerrit.wikimedia.org/r/1112261

Change #1112708 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/WikibaseQualityConstraints@master] Make known-good regex check strict

https://gerrit.wikimedia.org/r/1112708

Change #1112708 merged by jenkins-bot:

[mediawiki/extensions/WikibaseQualityConstraints@master] Make known-good regex check strict

https://gerrit.wikimedia.org/r/1112708

Change #1112746 had a related patch set uploaded (by Audrey Penven; author: Audrey Penven):

[mediawiki/extensions/WikibaseQualityConstraints@master] Use more efficient checking for known-good regexes

https://gerrit.wikimedia.org/r/1112746

Change #1112771 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/WikibaseQualityConstraints@wmf/1.44.0-wmf.12] Make known-good regex check strict

https://gerrit.wikimedia.org/r/1112771

Change #1112771 merged by jenkins-bot:

[mediawiki/extensions/WikibaseQualityConstraints@wmf/1.44.0-wmf.12] Make known-good regex check strict

https://gerrit.wikimedia.org/r/1112771

Mentioned in SAL (#wikimedia-operations) [2025-01-20T15:29:11Z] <lucaswerkmeister-wmde@deploy2002> Started scap sync-world: Backport for [[gerrit:1112771|Make known-good regex check strict (T380751)]]

Mentioned in SAL (#wikimedia-operations) [2025-01-20T15:32:45Z] <lucaswerkmeister-wmde@deploy2002> lucaswerkmeister-wmde: Backport for [[gerrit:1112771|Make known-good regex check strict (T380751)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2025-01-20T15:39:46Z] <lucaswerkmeister-wmde@deploy2002> Finished scap sync-world: Backport for [[gerrit:1112771|Make known-good regex check strict (T380751)]] (duration: 10m 35s)

Change #1112746 merged by jenkins-bot:

[mediawiki/extensions/WikibaseQualityConstraints@master] Use more efficient checking for known-good regexes

https://gerrit.wikimedia.org/r/1112746

Change #1112261 merged by jenkins-bot:

[operations/mediawiki-config@master] Add known-good regexes for WikibaseQualityConstraints

https://gerrit.wikimedia.org/r/1112261

Mentioned in SAL (#wikimedia-operations) [2025-01-21T14:58:08Z] <lucaswerkmeister-wmde@deploy2002> Started scap sync-world: Backport for [[gerrit:1112261|Add known-good regexes for WikibaseQualityConstraints (T380751)]]

Mentioned in SAL (#wikimedia-operations) [2025-01-21T15:04:40Z] <lucaswerkmeister-wmde@deploy2002> lucaswerkmeister-wmde, audreypenven: Backport for [[gerrit:1112261|Add known-good regexes for WikibaseQualityConstraints (T380751)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2025-01-21T15:27:51Z] <lucaswerkmeister-wmde@deploy2002> Finished scap sync-world: Backport for [[gerrit:1112261|Add known-good regexes for WikibaseQualityConstraints (T380751)]] (duration: 29m 44s)