Page MenuHomePhabricator

Address Parsoid/PHP noise from cluttering mediawiki train deployments
Closed, ResolvedPublic

Description

Parsoid (JS or PHP) has known issues where large pages will either OOM or timeout. With Parsoid/PHP, the memory and request time limits are lower than with Parsoid/JS. We are addressing these issue separately in a number of ways. Given Parsoid's usage profile, these rendering or parse failures shouldn't block train deployments.

However, these failures are currently introducing artificial train blockers by triggering icinga or other train rollout alerts. So, this noisiness needs to be addressed so that they don't act as false positives wrt train rollout.

I am filing this task as a record of conversation in #wikimedia-operations channel. @Krinkle and @cscott are working on this already.

Not that these failures lead to WMFTimeout or OOM fatal errors that occur outside the Parsoid codebase and as such, aren't easily handled within the Parsoid codebase.

Event Timeline

ssastry created this task.

Change 554638 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[operations/puppet@production] Ensure fatal PHP7 errors from the Parsoid cluster don't spam mediawiki logs

https://gerrit.wikimedia.org/r/554638

Change 554639 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[operations/mediawiki-config@master] Keep $isParsoidCluster test consistent w/ operations/puppet

https://gerrit.wikimedia.org/r/554639

Change 554645 had a related patch set uploaded (by Cwhite; owner: Cwhite):
[operations/puppet@production] profile: rewrite log type to parsoid-php if php7.2-fpm err from wtp hosts

https://gerrit.wikimedia.org/r/554645

Change 554645 merged by Cwhite:
[operations/puppet@production] profile: rewrite log type to parsoid-php if php7.2-fpm err from wtp hosts

https://gerrit.wikimedia.org/r/554645

Change 554639 merged by jenkins-bot:
[operations/mediawiki-config@master] Keep $isParsoidCluster test consistent w/ operations/puppet

https://gerrit.wikimedia.org/r/554639

Change 554638 abandoned by C. Scott Ananian:
Ensure fatal PHP7 errors from the Parsoid cluster don't spam mediawiki logs

Reason:
Superceded by Idb10d1b39b8ce369995239a5fb97d1d80a2e52a2

https://gerrit.wikimedia.org/r/554638