Page MenuHomePhabricator

Refactor mobile schemas
Closed, ResolvedPublic

Description

Our schema code has evolved into a giant ... well, code. Let's make these changes to make the maintenance easier:

  • Email wikitech-l about forthcoming change to remove logging if non-sendBeacon
  • Use mw.eventLog.Schema
  • Do not bundle all schema files as a single module (mobile.loggingSchemas). Create separate modules for each schema and add as a dependance to modules where each schema is used
  • Do not trick schema files into thinking that Schema.x module exists when eventLogging is not installed. If EventLogging is not installed, no schema file should be loaded in the front end.
  • Remove code related to logging past events. Rely on sendBeacon if it's supported or don't log at all. There has been multiple discussions in the mailing thread and gerrit about recording events from previous sessions.
    • [phuedx] N.B. that this is implied by the use of mw.eventLog.Schema but there's nothing wrong with being explicit.

Anything else I missed?

  • Do the above for Gather, Thanks, and any other extension that depends on MobileFrontend Schema class.

Details

Related Gerrit Patches:
mediawiki/extensions/MobileFrontend : masterRemove MobileOptionsTracking event logging
mediawiki/extensions/MobileFrontend : masterHygiene: Cleanup old schema related code
mediawiki/extensions/MobileFrontend : masterHygiene: Remove an unused schema file
mediawiki/extensions/MobileFrontend : masterHygiene: remove some MobileWebClickTracking schemas
mediawiki/extensions/MobileFrontend : masterHygiene: Use mw.eventLog.Schema for logging MobileWebMainMenuClickTracking events
mediawiki/extensions/Thanks : masterRemove mobile event logging code
mediawiki/extensions/Gather : masterHygiene: Refactor schemas
mediawiki/extensions/MobileFrontend : masterHygiene: use mw.eventLog.Schema for logging MobileWebLanguageSwitcher schema events
mediawiki/extensions/MobileFrontend : masterHygiene: use mw.eventLog.Schema for logging MobileWebSearch schema events
mediawiki/extensions/MobileFrontend : masterHygiene: MobileWebWatching schema is no longer in use
mediawiki/extensions/MobileFrontend : masterHygiene: remove MobileWebUIClickTracking schema
mediawiki/extensions/MobileFrontend : masterHygiene: use mw.eventLog.Schema for logging Edit schema events
mediawiki/extensions/MobileFrontend : masterHygiene: make MobileWebWatching schema inherit from mw.eventLog.Schema

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Jdlrobson set Security to None.Feb 5 2016, 12:18 AM
Jdlrobson moved this task from Backlog to Tech debt on the MobileFrontend board.Feb 18 2016, 6:44 PM

Change 272921 had a related patch set uploaded (by Bmansurov):
Hygiene: make Edit schema inherit from mw.eventLog.Schema

https://gerrit.wikimedia.org/r/272921

Change 272923 had a related patch set uploaded (by Bmansurov):
Hygiene: make MobileWebWatching schema inherit from mw.eventLog.Schema

https://gerrit.wikimedia.org/r/272923

Change 272966 had a related patch set uploaded (by Bmansurov):
Hygiene: make MobileWebUIClickTracking schema inherit from mw.eventLog.Schema

https://gerrit.wikimedia.org/r/272966

Change 272968 had a related patch set uploaded (by Bmansurov):
Hygiene: make MobileWebSearch schema inherit from mw.eventLog.Schema

https://gerrit.wikimedia.org/r/272968

Change 273456 had a related patch set uploaded (by Bmansurov):
Hygiene: make MobileWebClickTracking schemas inherit from mw.eventLog.Schema

https://gerrit.wikimedia.org/r/273456

bmansurov updated the task description. (Show Details)Feb 26 2016, 8:26 PM

Change 273514 had a related patch set uploaded (by Bmansurov):
Update mobile event logging code

https://gerrit.wikimedia.org/r/273514

Change 273516 had a related patch set uploaded (by Bmansurov):
Hygiene: Remove an old file

https://gerrit.wikimedia.org/r/273516

Please start merging the initial patches if they look good as it will make it easier to rebase the subsequent ones.

Change 274115 had a related patch set uploaded (by Bmansurov):
Refactor schemas

https://gerrit.wikimedia.org/r/274115

Change 274123 had a related patch set uploaded (by Bmansurov):
Hygiene: make MobileWebLanguageSwitcher schemas inherit from mw.eventLog.Schema

https://gerrit.wikimedia.org/r/274123

Change 274126 had a related patch set uploaded (by Bmansurov):
Hygiene: Cleanup old schema files

https://gerrit.wikimedia.org/r/274126

I am not going to review this until the language switcher and lazy loading images are wrapped up as it will take away too much of my focus. Hopefully we'll get some time at the end of the sprint.

Change 272923 abandoned by Bmansurov:
Hygiene: make MobileWebWatching schema inherit from mw.eventLog.Schema

Reason:
See the alternative patch above.

https://gerrit.wikimedia.org/r/272923

Let's get the base patch merged and I'll rebase the remaining ones afterward.

phuedx added a subscriber: phuedx.Mar 29 2016, 9:50 AM

Let's get the base patch merged...

Some work…

... and I'll rebase the remaining ones afterward.

A lot more work… ;)

Agree. I was highlighting that we should pay all our attention to the base patch for now. I know the remaining patches need a rebase, but I want the base to be stable first.

And I was – mostly – kidding.


Quoting from my most recent review:

@Jdlrobson: I think that @bmansurov is right to change only the mechanism by which the Edit events are logged. Any issues that we have with the code that has just been moved should be fixed in a follow-on change.

Neil_P._Quinn_WMF raised the priority of this task from Medium to High.Mar 29 2016, 11:14 PM
Neil_P._Quinn_WMF moved this task from Backlog to Blocked on the Contributors-Analysis board.

Change 272921 merged by jenkins-bot:
Hygiene: use mw.eventLog.Schema for logging Edit schema events

https://gerrit.wikimedia.org/r/272921

Change 280493 had a related patch set uploaded (by Jdlrobson):
Hygiene: MobileWebWatching schema is no longer in use

https://gerrit.wikimedia.org/r/280493

Email titled "MobileFrontend schema code (EventLogging) changes" has been sent to wikitech-l.

bmansurov updated the task description. (Show Details)Mar 30 2016, 7:52 PM

Change 272966 merged by jenkins-bot:
Hygiene: remove MobileWebUIClickTracking schema

https://gerrit.wikimedia.org/r/272966

Change 280493 merged by jenkins-bot:
Hygiene: MobileWebWatching schema is no longer in use

https://gerrit.wikimedia.org/r/280493

Change 272968 merged by jenkins-bot:
Hygiene: use mw.eventLog.Schema for logging MobileWebSearch schema events

https://gerrit.wikimedia.org/r/272968

phuedx updated the task description. (Show Details)Apr 4 2016, 9:53 AM
phuedx added a comment.EditedApr 4 2016, 6:34 PM

Remember to mark the MobileWebDiffClickTracking schema as inactive once 273516: Hygiene: Remove an unused schema file is merged.

phuedx updated the task description. (Show Details)Apr 4 2016, 6:35 PM
phuedx updated the task description. (Show Details)

Change 274123 merged by jenkins-bot:
Hygiene: use mw.eventLog.Schema for logging MobileWebLanguageSwitcher schema events

https://gerrit.wikimedia.org/r/274123

Change 281828 had a related patch set uploaded (by Bmansurov):
Hygiene: remove some MobileWebClickTracking schemas

https://gerrit.wikimedia.org/r/281828

Change 274115 merged by jenkins-bot:
Hygiene: Refactor schemas

https://gerrit.wikimedia.org/r/274115

Change 273456 merged by jenkins-bot:
Hygiene: Use mw.eventLog.Schema for logging MobileWebMainMenuClickTracking events

https://gerrit.wikimedia.org/r/273456

Change 273514 merged by jenkins-bot:
Remove mobile event logging code

https://gerrit.wikimedia.org/r/273514

Change 281828 merged by jenkins-bot:
Hygiene: remove some MobileWebClickTracking schemas

https://gerrit.wikimedia.org/r/281828

phuedx updated the task description. (Show Details)Apr 7 2016, 11:24 AM

@bmansurov: What else needs to happen now? What's your plan?

Oh great, a couple of clean-up patches left and that's it. I'll rebase them now.

Change 273516 abandoned by Bmansurov:
Hygiene: Remove an unused schema file

Reason:
Done in Icdc5447a4e68ec006dbd1362c84f81c4e7df46e9 now.

https://gerrit.wikimedia.org/r/273516

bmansurov added a comment.EditedApr 7 2016, 12:16 PM

I've squashed the last two patches into one and here it is: https://gerrit.wikimedia.org/r/#/c/274126/

Change 282182 had a related patch set uploaded (by Phuedx):
Remove MobileOptionsTracking event logging

https://gerrit.wikimedia.org/r/282182

Change 274126 merged by jenkins-bot:
Hygiene: Cleanup old schema related code

https://gerrit.wikimedia.org/r/274126

Change 282182 abandoned by Phuedx:
Remove MobileOptionsTracking event logging

Reason:
This schema allows us to provide a rough estimate of the number of MFβ users.

https://gerrit.wikimedia.org/r/282182

phuedx updated the task description. (Show Details)Apr 8 2016, 8:34 AM
phuedx removed bmansurov as the assignee of this task.Apr 8 2016, 8:46 AM
phuedx added a comment.Apr 8 2016, 8:57 AM

This is a technical task so any of the Readers-Web-Backlog software engineers can sign it off.

As well as the Cucumber scenarios and QUnit tests passing, you should be looking out for the following:

  • a reduced payload when the EventLogging extension isn't loaded;
  • events being successfully logged when the EventLogging extension is loaded
    • N.B. that you'll have to tweak the default bucketing values for the schemas
  • no modifications to localStorage or cookies as these sendBeacon-esque mechanisms have been removed from the codebase

I try to keep our part of the list of active schemas up to date, but here's the schemas that you should expect to see events being logged under on the client:


If you didn't already know, there's also a list of inactive schemas.

Jdlrobson closed this task as Resolved.Apr 13 2016, 6:07 PM
Jdlrobson claimed this task.

I can confirm all the above. Am seeing events for Edit, Menu and Language click tracking.

Neil_P._Quinn_WMF raised the priority of this task from High to Needs Triage.Mar 29 2018, 9:06 AM
Neil_P._Quinn_WMF moved this task from Blocked to Radar on the Contributors-Analysis board.