[Scraper] Gather available data which has been identified as valuable
Closed, ResolvedPublicGoal
Actions

Assigned To

Authored By

	jon_amar-WMDE
	Jul 12 2024, 11:04 PM

Description

context

we are taking a rather pragmatic approach to gathering the data we can find (freely available) regarding self-hosted wikibase instances.

story

as a person on the WBS team, I want to benefit from good data to enrich my understanding of our users' needs and to inform my decision making.

definition of done:

this is a container ticket to group all tickets related to acquiring data via the various methods including sparql, action api, and (where necessary) web scraping. when all sub tickets are completed then this ticket should be marked completed as well.

Related Objects
Search...

Status	Subtype	Assigned	Task
Resolved	Goal	RickiJay-WMDE	T369960 [Scraper] Gather available data which has been identified as valuable
Resolved		RickiJay-WMDE	T368763 Add language data to the metrics database
Resolved		RickiJay-WMDE	T368769 Import full list of known instances from Wikibase.world
Resolved		RickiJay-WMDE	T368776 Add key log data to database
Declined	Spike	None	T370410 Product/UX: Categorize Log Types
Resolved		RickiJay-WMDE	T369902 Key metrics for WMF convo
Resolved		RickiJay-WMDE	T369903 Flag known test instances in database
Resolved	Spike	None	T369967 [Scraper] Examine instances w a Wikibase Manifest and collect data where possible
Invalid		RickiJay-WMDE	T369969 Gather visitation stats via action API
Resolved		RickiJay-WMDE	T372758 gather data from special statistics page
Open		None	T374498 [Scraper] Analysis of Extension grouping

Event Timeline

jon_amar-WMDE created this task.Jul 12 2024, 11:04 PM

jon_amar-WMDE added a subtask: T368763: Add language data to the metrics database.

jon_amar-WMDE added a subtask: T368769: Import full list of known instances from Wikibase.world.Jul 12 2024, 11:07 PM

jon_amar-WMDE added a subtask: T368776: Add key log data to database.

jon_amar-WMDE added a subtask: T369902: Key metrics for WMF convo.

jon_amar-WMDE added a subtask: T369903: Flag known test instances in database.

RickiJay-WMDE closed subtask T368776: Add key log data to database as Resolved.Jul 18 2024, 1:23 PM

RickiJay-WMDE changed the status of subtask T369903: Flag known test instances in database from Open to In Progress.Jul 24 2024, 2:36 PM

RickiJay-WMDE closed subtask T369903: Flag known test instances in database as Resolved.Jul 24 2024, 3:08 PM

RickiJay-WMDE renamed this task from Gather available data which has been identified as valuable to Suite Scraper: Gather available data which has been identified as valuable.Aug 5 2024, 12:47 PM

RickiJay-WMDE removed a parent task: T366350: MVP WBS Metrics tooling from a list of known instances.

jon_amar-WMDE added a parent task: T371844: [Scraper] Metrics 1.1 - increase ease of access to data.Aug 5 2024, 6:29 PM

jon_amar-WMDE removed RickiJay-WMDE as the assignee of this task.Aug 5 2024, 6:32 PM

jon_amar-WMDE removed a parent task: T371844: [Scraper] Metrics 1.1 - increase ease of access to data.Aug 5 2024, 6:51 PM

RickiJay-WMDE moved this task from Sprint-∞ to Focus: Ecosystem Insights on the Wikibase Suite Team board.Aug 6 2024, 1:51 PM

RickiJay-WMDE edited projects, added Wikibase Suite Team; removed Wikibase Suite Team (Sprint-∞).

RickiJay-WMDE changed the status of subtask T372758: gather data from special statistics page from Open to In Progress.Aug 21 2024, 12:33 AM

RickiJay-WMDE changed the subtype of this task from "Spike" to "Goal".Aug 21 2024, 8:58 PM

RickiJay-WMDE closed subtask T372758: gather data from special statistics page as Resolved.Oct 15 2024, 1:17 PM

RickiJay-WMDE closed subtask T368763: Add language data to the metrics database as Resolved.May 28 2025, 10:51 AM

Leif_WMDE renamed this task from Suite Scraper: Gather available data which has been identified as valuable to [Scraper] Gather available data which has been identified as valuable.Jun 13 2025, 3:59 PM

Leif_WMDE moved this task from Focus: Ecosystem Insights to Needs Preparation on the Wikibase Suite Team board.

Leif_WMDE moved this task from Needs Preparation to Goals & Epics on the Wikibase Suite Team board.Aug 15 2025, 4:12 PM

Leif_WMDE triaged this task as High priority.Aug 15 2025, 4:15 PM

Leif_WMDE moved this task from Goals & Epics to Needs Preparation on the Wikibase Suite Team board.Sep 15 2025, 6:08 AM

Leif_WMDE moved this task from Needs Preparation to Ready for Refinement on the Wikibase Suite Team board.Sep 15 2025, 6:31 AM

Leif_WMDE changed the task status from Open to In Progress.Oct 9 2025, 8:25 AM

Leif_WMDE added a project: Epic.

Leif_WMDE moved this task from Ready for Refinement to Goals & Epics on the Wikibase Suite Team board.

Leif_WMDE moved this task from Goals & Epics to Sprint 6 (Oct 14 - Oct 28) on the Wikibase Suite Team board.Oct 22 2025, 10:51 AM

Leif_WMDE edited projects, added Wikibase Suite Team (Sprint 6 (Oct 14 - Oct 28)); removed Wikibase Suite Team.

Leif_WMDE moved this task from Backlog to EPICS on the Wikibase Suite Team (Sprint 6 (Oct 14 - Oct 28)) board.Oct 27 2025, 1:26 PM

conny-kawohl_WMDE edited projects, added Wikibase Suite Team (Sprint 7 (Oct 28 - Nov 11)); removed Wikibase Suite Team (Sprint 6 (Oct 14 - Oct 28)).Oct 28 2025, 12:17 PM