Page MenuHomePhabricator

objectcache: Enable teams to measure their load in production by adding resident-inventory measurement for DB-backed MainStash
Open, In Progress, MediumPublic

Description

Different parts of Wikimedia production code write items to MainStash, the (DB-backed SqlBagOStuff running on x2). As a common resource shared by many teams, there's a risk of individual teams/products being irresponsible in how we use the common service. However, there's no built-in tooling to see what is actually resident for tenant teams (or SRE Data Persistence themselves), AFAIAA.

I suggest (if SRE Data Persistence agree!) adding some general-purpose resident-inventory measurement tooling to MediaWiki core to answer some questions: how many keys and how many bytes each keygroup (collection) occupies, the total and average footprint, and (by extracting this regularly) how any of this grows over time. I'm hoping to get agreement and input on shape and safety; if we're OK with this, I'm happy to lead the implementation.

Background

The idea is mainly about giving tenant teams (initially us in Abstract Wikipedia, but others too) the means to self-audit their own responsible use, as well as general operator visibility. The object-cache layer already emits bagostuff_* StatsFactory counters labelled by keygroup, but these are flow metrics (bytes read/written through the cache) — they never decrement on expiry or eviction, so they cannot answer "what is resident right now, and is it growing?". The BagOStuff interface exposes no enumeration, count, or size method; the only state-touching operation is deleteObjectsExpiringBefore(). To answer the proposed inventory questions will need aggregating the objectcache back-end directly, which no available tool does today.

Technical notes

Proposed shape, offered as a starting point for discussion rather than a fixed design:

  • Add a read-only public method on the concrete SqlBagOStuff class (e.g. getKeyGroupStats()), structurally a read-only twin of deleteObjectsExpiringBefore(), re-using its existing per-(server shard × table shard) iteration rather than re-deriving the layout. This method will return per-keygroup resident key count and byte totals. Grouping is normalised in PHP to match determinekeyGroupForStats() exactly, so buckets line up with the existing bagostuff_* counter labels in Grafana.
    • Query: Will this answer the questions we should be asking? Or are there other things we should measure too?
  • Add a maintenance script with two modes: default human-readable audit to stdout (one-off "are we storing too much?"), and a --report-to-stats emitting resident StatsFactory gauges (mainstash_resident_keys, mainstash_resident_bytes, labelled keygroup) for a timer-driven dashboard showing evolution over time, set up as a mwcron job running every 24 hrs at production load low point.
    • Query: Does this make sense? Also, would welcome bikeshedding on what we call the new StatsFactory counters here.
  • The reads would always target a replica (new read-index parameter), never the write primary, as a SUM(OCTET_LENGTH(value)) full scan per table shard must stay off the x2 primary.
    • Query: Ts it acceptable to run on a replica off-peak? Do we need shard-targeting/rate controls to spread the scan? (This is the main specific point on which SRE Data Persistence input is wanted.)
  • Reported bytes could be one or the other (or both) of the on-disk, gzdeflate-compressed footprint (what actually stresses storage, replication, and backups), or the logical payload size.
    • Query: Another check here that this sounds reasonable.

Acceptance criteria

  • SRE Data Persistence agree the approach (replica full-scan, cadence, any shard-targeting/rate limits) is safe to run against x2, or steer it to a safer shape.
  • Core gains a documented, maintenance/admin-only way to report resident MainStash inventory by keygroup (count and compressed bytes), consistent with existing keygroup metric labels.
  • We run this as an mwcron and see the metrication flow in.
  • AWT and other tenant teams are encouraged to review and track their inventory in Grafana from the StatsFactory gauges to monitor growth over time.

Event Timeline

Gehel subscribed.

Removing Data-Platform-SRE, I don't think we're involved in this in any way. If I misunderstood, please re-add us!

Hi, overall sounds good but we don't have x2 anymore, nor any replicas. We have three clusters now: ms1, ms2, ms3. All three are just one primary db per dc (https://orchestrator.wikimedia.org/web/cluster/ms1 for example). The data is replicated as in mw picks two out of three via consistent hashing so even if one goes out, you have a copy but it's not the traditional primary/replica setup. Shamelessly promoting this: https://wikitech.wikimedia.org/wiki/User:ASarabadani_(WMF)/Consistent_hashing_in_MediaWiki that goes in details of how it works.

We could do what we have done WAN and use stats system to send data to Prometheus) (while setting or retrieving). But I wonder if it would cover for all of your cases.

Hi, overall sounds good but we don't have x2 anymore, nor any replicas. We have three clusters now: ms1, ms2, ms3. All three are just one primary db per dc (https://orchestrator.wikimedia.org/web/cluster/ms1 for example). The data is replicated as in mw picks two out of three via consistent hashing so even if one goes out, you have a copy but it's not the traditional primary/replica setup. Shamelessly promoting this: https://wikitech.wikimedia.org/wiki/User:ASarabadani_(WMF)/Consistent_hashing_in_MediaWiki that goes in details of how it works.

Thanks Amir, that's really helpful, and quite neat! A re-stated potential approach, against ms* not x2, if we were to do this:

As there's no replicas, any queries would have to be gentle as they'd need to run on each of the live primaries to get the total data:

  • chunked by keyname primary-key ranges in small batches (not one big SUM(OCTET_LENGTH(value)) or GROUP BY) with a configurable sleep between batches to spread the load;
  • OCTET_LENGTH(value) computed server-side, so we ship only the key name and an int, never the blob;
  • short autocommit reads (no long-held snapshot on the primary);
  • one cluster at a time, reusing the existing per-tag targeting (T282761) and advisory-lock guard (T330377) so runs never overlap and can be spread across the day; and
  • off-peak, 24h cadence.

If you're OK with this idea, a steer from you on (rough estimates of) an acceptable batch size, inter-batch sleep, and time window would help.

I hesitate to run this on a live primary, but with this design the only alternative I see would be a set of temporarily depooled nodes for each. That feels massive piece of work (and we don't have automated pooling/depooling of DB primaries, AFAIAA).

We could do what we have done WAN and use stats system to send data to Prometheus) (while setting or retrieving). But I wonder if it would cover for all of your cases.

I think it's a great complementary approach, but it can't be the whole answer on its own: like the existing bagostuff_bytes_*_total that never decrements on expiry/eviction, and each appserver only sees its own traffic, so it can't answer "what's resident right now, and is it growing?".

Perhaps we could have a hybrid system: keep the existing per-keygroup flow counters for a cheap continuous trend, and layer a periodic scan-based gauge (bagostuff_resident_keys / bagostuff_resident_bytes) for residency calculated maybe weekly. The flow rate then explains the gauge's growth on the dashboard.

On double-counting: since each key lives on two of three clusters, I'd report per-cluster (labelled by cluster) to give the "what's on each box" number, and let Grafana derive logical footprint as total ÷ dataRedundancy.

Does the chunked-primary scan sound acceptable, or would you steer us to the depooled/backup shape? Or just give up?

Combining both approaches sound good to me. Basically on being set/retrieval, emit some data and on weekly basis via chunked scan of the keys, emit another set of data. As long as it's taking advantage of indexes (via range check of keys), it should be fine.

The only complexity or a puzzle to solve is that we might not have a section pooled during the check. Let me explain it a different way. We used to have a nightly maint script that cleaned up parsercache clusters. After we re-architectured it to be like this, when we had maintenance (hardware issues, reboots, etc.) and a cluster was out, the maint script would fail with fireworks. The solution I had was to use the normal purge mechanism which during writes, at random times it would just trigger a job to clean up expired entries.

So we can do two things:

  • Making sure the maint script doesn't break if a section is not found and have a graceful degradation.
  • Trigger a job at random to do the work (similar to parsercache expiry clean up). But making it be at reliable time intervals gonna be fun though. So I'm not sure whether this is a good idea.

FWIW, currently each cluster has around 5M rows. It's not small but it's rather easy to scan.

Another thing to note: Since the value is gzipped. So the result will be a different than the raw string and I was a bit worried the octet_length might not work. But it works just fine:

MariaDB [mainstash]> select OCTET_LENGTH(value) from objectstash limit 1;
+---------------------+
| OCTET_LENGTH(value) |
+---------------------+
|                 175 |
+---------------------+
1 row in set (0.000 sec)

Change #1309731 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/core@master] [WIP] objectcache: Add resident-inventory census script for DB-backed object stashes

https://gerrit.wikimedia.org/r/1309731

Thanks; I've pushed a proposed maintenance script to do that side of things. If that works, we can try running it manually and if it's appropriate schedule it.

Jdforrester-WMF changed the task status from Open to In Progress.Mon, Jul 13, 1:40 PM
Jdforrester-WMF claimed this task.
Jdforrester-WMF triaged this task as Medium priority.