Page MenuHomePhabricator

Make ApiFeatureUsage extension easier to set up
Open, Needs TriagePublic

Description

AIUI the data flow for ApiFeatureUsage is:

  • ApiBase::logFeatureUsage()
  • wfDebugLog( 'api-feature-usage' )
  • Wikimedia's monolog setup
  • logstash
  • a sanitized version to the elasticsearch cluster used by CirrusSearch

I've been thinking for a while (ever since I tried to set up the extension locally for testing) that a db backed version of ApiFeatureUsage would be nice to have for small wikis, and would make it a lot easier to set up. We could use a Psr\Log\LoggerInterface logger in ApiBase::logFeatureUsage() that is overridable via a setter, use a hook to set a custom logger that does writes to the db if configured. That could also be used by the elasticsearch version to directly put the events inside the cluster we use to query instead of going through logstash (though maybe we do want them in logstash anyways?)

Event Timeline

That could also be used by the elasticsearch version to directly put the events inside the cluster we use to query instead of going through logstash (though maybe we do want them in logstash anyways?)

For WMF we probably do want them going through logstash, so they wind up both unsanitized in Kibana and sanitized in the ES data used by ApiFeatureUsage.

ApiBase::logFeatureUsage() exists largely to allow for global analysis of deprecated feature usage without having to parse things out of the api.log ocean-sized firehose. The ApiFeatureUsage extension is an add-on to allow clients to get a summary of their own use of deprecated features.

For your proposed version that just shoves the events into MySQL, making it easier to go through the logging infrastructure seems like the wrong way to go. What you'd really want would probably be a hook in ApiBase::logFeatureUsage() so you can just directly insert the events into your database table without messing with trying to parse the log message at all.