Summary

It would be useful to have some metrics about how well (or not) that the bloom filter is working in ReadingLists.

• How fast is the memcached lookup to retrieve the bloom filter? (with cache hit)
  • bloom filter with negative result (page definitely not bookmarked)
  • bloom filter with false positive or saved page, with the subsequent DB query to verify and retrieve metadata
• How fast is the saved page lookup via DB query? (for cases that bypass bloom filter)
• How fast is the query to retrieve the saved pages and build the bloom filter? and overall building the bloom filter
• How quickly are the bloom filter rebuild jobs processed? (from time the jobs are enqueued and run)
• How often there is bookmark status lookup, which results in cache status missing vs stale?

Things to measure:

• latency
• cache payload size
• DB query count (number of saved pages)

Other questions:

• How often do the lookups hit a cached bloom filter? (cache hit/miss rate)
• How often does the bloom filter produce a definite negative and avoid the DB lookup?
• How often does it produce a probable positive, thus requiring the DB lookup?
• How often are there false positives?
• How often are bloom filter rebuilds triggered?
• How often is the bloom filter bypassed because the user has more than the configured max number of pages?

For the metrics, we should use statslib that is available in MediaWiki:

• https://www.mediawiki.org/wiki/Manual:Stats
• https://www.mediawiki.org/wiki/Stats
• https://wikitech.wikimedia.org/wiki/Prometheus

and see example https://gerrit.wikimedia.org/g/mediawiki/extensions/DiscussionTools/+/4d8f74e8876e95dd5ee3802ff5fecd531b17b97b/includes/CommentFormatter.php#77

Follow up:

• Create a Grafana dashboard for these metrics and maybe for ReadingLists in general

Technical notes

This can be done in multiple patches:

• counter metrics for bloom filter and db lookups - done as part of this task
• async bloom filter rebuild via the job - to be done in https://phabricator.wikimedia.org/T423708
• timings and payload size - to be done in https://phabricator.wikimedia.org/T423709
• setup dashboard in Grafana - https://phabricator.wikimedia.org/T423699

Use counters for:

• cache miss
• definite negative
• probable positive
  • false positive
  • true positive
• user has too many pages saved, bypassing bloom filter
• bloom filter rebuild

Use timers:

• cache lookup latency
• DB lookup latency
• rebuild latency

Developer setup

• https://www.mediawiki.org/wiki/Manual:Stats#MediaWiki-Docker

Setup with docker (see above) and use the following settings:

$wgStatsdServer = 'statsd';  // might not be needed
$wgStatsTarget = 'udp://statsd-exporter:9125';
$wgStatsFormat = 'dogstatsd';

To view the stats events:

curl -s http://localhost:9112/metrics | grep -E 'readinglists'