Page MenuHomePhabricator

[Analytics] [Tech Debt] Remove unneeded filtered user agents columns from tables and charts
Closed, DeclinedPublic

Description

WMDE Analytics Request

This task was generated using the WMDE Analytics request form. Please use the task templates linked on our project page to create tasks for the team. Thank you!

Why (Context & Decision)

What problem are you trying to solve, and what decision will this analysis inform? Briefly explain the organisational or strategic context, why this matters now, and what action will be taken based on the outcome.

There's consistently been confusion about the "filtered user agent" fields in Superset charts including the following dashboards:

The original thinking here was a count of distinct user agents that are less likely to be bots, but my understanding is that this isn't in use for reporting.

This field is based on the following:

count(
    DISTINCT CASE
        WHEN
            user_agent
            NOT LIKE 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/% (KHTML, like Gecko) Chrome/% Safari/%'
        THEN user_agent
    END
) AS total_filtered_user_agents,

If this metric isn't being reported anywhere, then it should be removed from the tables, queries and charts.

What (Scope & Output)

Describe the specific question(s), metrics, segments, or deliverables (e.g., dashboard, deep dive, experiment analysis), including any relevant definitions or constraints.

  • Remove this field from queries
  • Remove the columns that contain this data from the following tables
    • wd_action_api_metrics_monthly
    • wd_rest_api_metrics_monthly
    • wd_dump_metrics_monthly
  • Update the tables on Superset and remove the field from the charts

By When (Timing & Priority)

Provide a clear deadline, any key milestones (e.g. launch, leadership review), and note if timing is flexible or fixed.

Tech debt, so relatively low priority and no deadline, but this would help make sure that information we're conveying to new WMDE employees is clear and up to date.


Information below this point is filled out by the analyst.

Sub Tasks

A breakdown of the steps to complete this task.

  • See output check boxes above

Estimation

Estimate: A few hours
Actual:

Event Timeline

As discussed with @karapayneWMDE, making this task and assigning it initially to @Lena_WMDE as it'd be great to have a decision on whether we should get rid of these fields. My understanding is that we don't need them, but CCing the others who I've discussed this with. @Ifrahkhanyaree_WMDE is the stakeholder for the dashboards linked, and @Lydia_Pintscher is CCed for a confirmation that they're not being reported. @Maria_Lechner_WMDE is CCed as we recently discussed what these fields mean (we figured a task made sense to move this along).

Thanks, all!

I defer this to Ifrah. Nothing to add from my side.

We can remove them, we have a new ticket to improve this https://phabricator.wikimedia.org/T416478 and can tackle that when we get to it

Declining this as this work will go through, but will be tracked in the more up to date T416478 :)