Page MenuHomePhabricator

[MEP] [BUG] Timestamp format changed in migrated client-side EventLogging schemas
Closed, ResolvedPublic

Description

The format of dt in client-side EventLogging schemas changed after migration to include millisecond precision. Here's a recent example from the migration of EditAttemptStep in T267343:

Run on https://superset.wikimedia.org/superset/sqllab:

WITH ranked_events AS (
  SELECT
    dt,
    client_dt,
    schema,
    meta.dt AS serverside_dt,
    meta.id,
    uuid,
    RANK() OVER (PARTITION BY hour ORDER BY dt DESC) AS rnk
  FROM event.editattemptstep 
  WHERE year = 2021
  AND month = 3
  AND day = 8 AND hour IN(20, 21)
)
SELECT dt, client_dt, schema, serverside_dt, id, uuid
FROM ranked_events
WHERE rnk <= 5
dtclient_dtschemaserverside_dtiduuid
2021-03-08T20:59:59ZEditAttemptStep14fd8bc5da7c577d83f150256294fa22
2021-03-08T20:59:59ZEditAttemptStep3ec891312b015007b1a4ddaa57ac2dad
2021-03-08T20:59:59ZEditAttemptStep8b79cd562d0f5c70a5ee5edf947ede4a
2021-03-08T20:59:58ZEditAttemptStepceafeb50ad8a5ba09d0c87bd527bc572
2021-03-08T20:59:58ZEditAttemptStep43573c9a473d5eada62cde7f6dff7a35
2021-03-08T20:59:58ZEditAttemptStep37bddd6f9f995dd5bf4d43b58f596f45
2021-03-08T20:59:58ZEditAttemptStep0e28daf0b1465809a051aa3933cb983f
2021-03-08T20:59:58ZEditAttemptStep67792ae611455f9eb867f8ca2ec86b57
2021-03-08T20:59:58ZEditAttemptStep7cc87007fd865d9ea9922e269df88ab2
2021-03-09T21:27:22.975Z2021-03-09T21:27:22.975ZEditAttemptStep2021-03-08T21:27:07.897Z985ee03e-92df-4dfe-8cd7-8bf69ad851c7
2021-03-09T21:27:18.120Z2021-03-09T21:27:18.120ZEditAttemptStep2021-03-08T21:27:07.897Z2581fb75-e5ee-48b9-98c1-63b3225a2474
2021-03-09T21:27:18.119Z2021-03-09T21:27:18.119ZEditAttemptStep2021-03-08T21:27:07.897Z5f52c868-501e-4cdc-b1ab-328dda81f353
2021-03-09T21:27:17.814Z2021-03-09T21:27:17.814ZEditAttemptStep2021-03-08T21:27:07.895Z7ac7e503-67af-404e-bb4e-56b4fddefe16
2021-03-09T09:37:36.477Z2021-03-09T09:37:36.477ZEditAttemptStep2021-03-08T21:41:31.208Zff5ff12e-105f-455b-8b43-f20cd95be5db

Event Timeline

At the moment it's unclear in the Product Analytics team whether the millisecond timestamp is a welcome feature or a bug that needs fixing. I think it warrants more discussion.

The Product Analytics team are going to discuss this in our upcoming planning meeting on March 16 to determine what our preferences around this is.

fdans moved this task from Incoming to Event Platform on the Analytics board.
LGoto moved this task from Doing to To Do on the Better Use Of Data board.

Per our discussion with Product Analytics, Andrew Otto, Jason Linehan, and Michael Holloway in the Product Analytics team sharing meeting, we have agreed to move forward with ms precision. Moving this task to signoff, as the ms precision is now considered a feature, rather than a bug.