Page MenuHomePhabricator

Set up Grafana board for data bridge openings by property datatype
Closed, ResolvedPublic

Description

This might not be possible before production deployment – @Addshore, is there a Grafana for Beta? It would be great if we can discuss this while you’re still in Berlin :)

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 23 2019, 9:44 AM

AFAIK there's no "beta" Grafana. We'd have to configure the board(s) and see if they work properly once the Bridge goes live.

Here's the board in progress https://grafana.wikimedia.org/d/pVG7xcAZz/wikidata-bridge-data-types?panelId=2&fullscreen&edit&orgId=1
We can also have a board per type. Not sure which is better.
Some explanation of the data source queries:

  • summarize(counter.MediaWiki.wikibase.client.databridge.datatype.string, '1day', 'sum', false)

interval = 1day; aggregation function = sum, align to form = false
Summarizes the datatype into interval buckets of 1 day.

  • sortByMaxima(<the result of the summarize function>)

Sorts the list of metrics in descending order by the maximum value across the time period specified.

  • consolidateBy(<the result of sortByMaxima>, 'sum')

When a graph is drawn where width of the graph size in pixels is smaller than the number of datapoints to be graphed, Graphite consolidates the values to prevent line overlap. The consolidateBy() function changes the consolidation function from the default of ‘average’ to one of ‘sum’.

  • aliasByNode(<the result of consolidateBy>, 3)

Uses an alias derived from a node in the metric name.
Honestly I'm not sure why it's 3 but all of wikimedia de boards use this number.

Source: Graphite Docs

So beta has a different metric prefix
https://github.com/wikimedia/operations-mediawiki-config/blob/c65fdd4947dd163d3068987bc5b8bffeb117b1c6/wmf-config/CommonSettings.php#L716
and also sends its data to a different server
https://github.com/wikimedia/operations-mediawiki-config/blob/c65fdd4947dd163d3068987bc5b8bffeb117b1c6/wmf-config/LabsServices.php#L15

AFAIK there's no "beta" Grafana. We'd have to configure the board(s) and see if they work properly once the Bridge goes live.

There is a grafana beta at https://grafana-beta.wikimedia.org, but im not sure this actually has access to the beta data.

Anyway, then I found T75882: Setup Grafana for beta cluster which links to https://grafana.wikimedia.org/d/000000112/labs-project-board?orgId=1 and that links to https://grafana-labs.wikimedia.org/d/000000018/labs-project-board

So, there is a place you can make a dashboard with data from beta :) https://grafana-labs.wikimedia.org
And then after knowing the URL i searched on wikitech and found https://wikitech.wikimedia.org/wiki/Grafana.wikimedia.org#Beta_cluster
Looks like you need this one for editing https://grafana-labs-admin.wikimedia.org/ (Log in with same old LDAP)
and remember you need the data source "Labs Graphite", and change MediaWiki to BetaMediaWiki in your metrics.

Pablo-WMDE renamed this task from Set up Grafana board for data bridge openings by property datatype to [stalled] Set up Grafana board for data bridge openings by property datatype.Oct 31 2019, 10:34 AM
Pablo-WMDE changed the task status from Open to Stalled.
Tonina_Zhelyazkova_WMDE renamed this task from [stalled] Set up Grafana board for data bridge openings by property datatype to Set up Grafana board for data bridge openings by property datatype.Nov 11 2019, 3:15 PM
Tonina_Zhelyazkova_WMDE changed the task status from Stalled to Open.

The bridge tracks the data types with a topic in the MediaWiki namespace.
The board on the beta Grafana queries data with the BetaMediaWiki namespace. Currently there's nothing showing even though I made an edit on beta enwp yesterday.
I'm really not sure what's broken cause we established the tracking is working, you can see that at https://graphite.wikimedia.org/ and https://graphite-labs.wikimedia.org/ respectively.
@LucasWerkmeister can you please take a look at this, as you said you've done some Grafana boards before. Do you see something wrong in the queries?

I think I’ve fixed the dashboard. Since we track under the MediaWiki namespace (hard-coded), that’s the one we need to use in the dashboard, regardless of what the PHP side is configured to use (BetaMediaWiki). Also, the counter. prefix is only a signal to WikimediaEvents on what kind of metric to send to Graphite; but we need to append some summary at the end, count probably being the most useful one (I think most of the others are more useful for timers); and since the * is the 5th part of the metric name (counting from 0), that’s the one we should alias by.

aliasByNode(consolidateBy(sortByMaxima(summarize(MediaWiki.wikibase.client.databridge.datatype.*.count, '1day', 'sum', false)), 'sum'), 5)

(I left the other parts – like summarize, sortByMaxima – alone, though I’m not sure if they’re correct.)

Thanks! The topic we send though is counter.MediaWiki.wikibase.client.databridge.datatype.${datatype}. I'm not sure if it makes a difference to have counter in the beginning or in the end.

No, counter. is only a signal to WikimediaEvents, it’s removed before being sent to statsv.

And the .count in the end is added by Graphite, I believe (count of all samples received in each 60 s interval). Actually, sum is probably the better one, though in theory I think we don’t expect any counters other than 1 (unless there’s another level of aggregation before Graphite that already combines n “1c” samples into one “nc” sample?).

Okayyy there is some data being displayed on the beta board now. Thanks. Lucas!

Tonina_Zhelyazkova_WMDE closed this task as Resolved.Nov 12 2019, 4:13 PM