Page MenuHomePhabricator

Document data engineering items for Campaigns Product
Closed, ResolvedPublic

Description

Campaigns is working through questions/discovery for obtaining(T336603)/storing/handling grant_id (see notes).

The team is considering solutions including utilizing a per-wiki gadget which doesn't store grant_id within the campaigns database and instead stores grant_id and event_id elsewhere. The gadget cannot send data to Hive.

  • Can a gadget send data to Hive through Event Metrics or the Metrics Platform?
  • If we need to first store the gadet data somewhere (ancillary data storage), where should/can it be stored? How does the on-wiki working(s) come into play, and what should the team consider and keep in mind?

Generally, note, the ultimate goal for campaigns data is to ultimately move needed campaigns data into Hive, so it can be centralized and combined with existing product data for analysis.

Task:

  • Collect information on Event Platform a) Ticket to instrument simple usage tracking with Wikimedia Event Platform: T343163 b) see also this google doc.

Documents:
EM Instrumentation Document
Instrumentation Plan
see also T273235

Event Timeline

Note: The MP design is in flux and it might change over the next few months. It may be a few quarters before key work is complete.
As part of that or in addition to, there are platform issues that need to be resolve before analysts can access data.

Recommendation: decisions remain on how to architect the monoschema and make data available for analysts; if there's time, we should wait until MP is ready and move there.