Currently, all Parsoid/PHP fatal errors clog the production wiki fatal monitor channel. Given that Parsoid/PHP will run a lot and at least intially have a lot of errors, we should probably direct these logs to a different channel.
Need to look up what config to tweak where, and could possible go into tests/RTTestSettings.php once we figure that out.
Change 534889 had a related patch set uploaded (by Subramanya Sastry; owner: Subramanya Sastry):
[operations/mediawiki-config@master] Direct Parsoid/PHP rt-testing log events to a different target
Change 534889 merged by jenkins-bot:
[operations/mediawiki-config@master] Direct Parsoid/PHP rt-testing log events to a different target
Mentioned in SAL (#wikimedia-operations) [2019-09-10T17:58:17Z] <jforrester@deploy1001> Synchronized wmf-config/logging.php: T232042 Direct Parsoid/PHP rt-testing log events to a different target (duration: 01m 02s)
So, it looks like these logs disappeared from the "mediawiki" type, but they did not make an appearance under the "parsoid-tests" type in logstash (I searched in Kibana). Based on IRC conversations on MediaWiki-General, I learnt that there is puppet code involved in directing the log events from mediawiki to logstash. So, it looks like we'll need some puppet updates to these filters to ensure these events end up in logstash / kibana.
@fgiunchedi Something you can help us with?
Looks like adding this code at the end of modules/profile/files/logstash/filter-mediawiki.conf in the puppet repo might do the trick if we revert my mediawiki-config change patch?
# Scandium is a Parsoid test server and treat errors here as
# parsoid-tests errors and don't clog the main mediawiki channel
if [host] == "scandium" {
mutate {
replace => [ "type", "parsoid-tests" ]
}
}Yes sth like that should work, I'm not in love with hardcoding hostnames but overall it is best to have it in puppet than mediawiki-config when we'll replace scandium, so +1 from me!
Change 536208 had a related patch set uploaded (by Subramanya Sastry; owner: Subramanya Sastry):
[operations/puppet@production] Take #2: Redirect Parsoid/PHP rt-testing log events to "parsoid-tests"
Change 536208 merged by Giuseppe Lavagetto:
[operations/puppet@production] Take #2: Redirect Parsoid/PHP rt-testing log events to "parsoid-tests"
Joe deployed the revert and the puppet patch and I confirmed that this now directs the logs to 'parsoid-tests' type in logstash!