Maniphest T117403

Track daily entity usage data in graphite for all wikidata clients
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Addshore
	Nov 2 2015, 11:31 AM

Tags

Referenced Files

None

Subscribers

Description

625 wikis
5 metrics per wiki
3125 data points per day

metric names: daily.wikidata.entityusage.{dbname}.{aspect}

Data to be set as a gauge through statsd using the replicated dbs on the analytics cluster.

Script will:

load https://noc.wikimedia.org/conf/wikidataclient.dblist
Run SELECT eu_aspect, count(*) FROm {dbname}.wbc_entity_usage GROUP BY eu_aspect for each db
Send data to statsd and thus graphite

Requires graphite retention periods to be poked

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		Addshore	T117201 [WD] Implement usage KPI
		Resolved		Addshore	T117405 Add entity usage graphs to wikidata dashboard
		Resolved		Addshore	T117403 Track daily entity usage data in graphite for all wikidata clients
		Resolved		Addshore	T117402 Enable retention of daily metrics for longer periods of time in Graphite
		Declined		Addshore	T117732 Create a Graphite instance in the Analytics cluster

Event Timeline

Addshore created this task.Nov 2 2015, 11:31 AM

Addshore claimed this task.

Addshore raised the priority of this task from to Medium.

Addshore updated the task description. (Show Details)

Addshore subscribed.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 2 2015, 11:31 AM

Addshore added projects: WMDE-Analytics-Engineering, Wikidata, Grafana.Nov 2 2015, 11:32 AM

Addshore set Security to None.

Addshore added a subtask: T117402: Enable retention of daily metrics for longer periods of time in Graphite.

Addshore added a parent task: T117201: [WD] Implement usage KPI.

Addshore moved this task from Incoming to ToDo on the WMDE-Analytics-Engineering board.Nov 2 2015, 11:36 AM

Addshore added a parent task: T117405: Add entity usage graphs to wikidata dashboard.

JanZerebecki subscribed.Nov 3 2015, 12:50 PM

Addshore added a subtask: T117732: Create a Graphite instance in the Analytics cluster.Nov 4 2015, 5:13 PM

Addshore changed the status of subtask T117732: Create a Graphite instance in the Analytics cluster from Open to Stalled.Nov 13 2015, 2:21 PM

Addshore closed subtask T117402: Enable retention of daily metrics for longer periods of time in Graphite as Resolved.Nov 17 2015, 3:16 PM

Addshore moved this task from ToDo to Done on the WMDE-Analytics-Engineering board.Nov 17 2015, 4:32 PM

Done in https://gerrit.wikimedia.org/r/#/c/253625/

Initial dashboard can be found on grafana although more work needs to take place to make this better...

https://grafana.wikimedia.org/dashboard/db/wikidata-entity-usage

Addshore closed subtask T117732: Create a Graphite instance in the Analytics cluster as Declined.Nov 20 2015, 4:17 PM