Page MenuHomePhabricator

Mobile editors do not log new revision IDs on saveSuccess
Closed, ResolvedPublic

Description

In order to look at the size of edits and whether they were reverted, saveSuccess events need to have the new revision ID in the revision_id field. Since an attempt's events before saveSuccess should have the parent revision ID in that field, a quick way to check this is to make sure that all saveSuccess events have a larger revision ID than all previous events in the session.

with saves as (
    select
        event.editing_session_id as attempt_id,
        event.revision_id as revision_id,
        event.platform as platform,
        event.editor_interface as editor
    from event.editattemptstep
    where
        event.action = "saveSuccess" and
        year = 2019 and month = 6 and
        -- Remove Flow and other non-standard edits
        event.integration = "page"
),
pre_saves as (
    select
        event.editing_session_id as attempt_id,
        max(event.revision_id) as max_revision_id
    from event.editattemptstep
    where
        event.action != "saveSuccess" and
        year = 2019 and month = 6 and
        -- Remove Flow and other non-standard edits
        event.integration = "page"
    group by event.editing_session_id
)
select
    platform,
    editor,
    concat(round((
        sum(cast(saves.revision_id > pre_saves.max_revision_id as int)) * 100 / count(*)
    ), 1), "%") as save_has_greater_revision_id
from saves 
left join pre_saves
on saves.attempt_id = pre_saves.attempt_id
group by
    platform,
    editor

And it looks like like we don't always log the new revision ID on desktop wikitext and never log it on other platforms.

platformeditorsaveSuccess has larger revision ID
desktopvisualeditor0.0%
desktopwikitext96.4%
desktopwikitext-20170.0%
phonevisualeditor0.0%
phonewikitext0.0%

The phone editors are the ones that are important for the A/B test, although while we're looking at it it would be nice to fix this on desktop too so that in the future we can correlate details about the edit attempt (e.g. time taken, number of save attempts) with details of the published edit (e.g. length, whether reverted).

Details

Related Gerrit Patches:
mediawiki/extensions/MobileFrontend : masterOn page edit save, pass new revision ID through for logging
mediawiki/extensions/VisualEditor : masterPass newRevId through to MobileFrontend
mediawiki/extensions/VisualEditor : masterArticleTarget: saveSuccess wasn't receiving expected arguments

Event Timeline

nshahquinn-wmf renamed this task from Editing interfaces do not log new revision IDs on saveSuccess to Mobile editors do not log new revision IDs on saveSuccess.Jun 28 2019, 6:41 PM
nshahquinn-wmf updated the task description. (Show Details)

Open question

  • What would be involved to fix this issue, so we could measure edit revert rates?

Context relevant to the question above. From chat:

  • The issue is potentially unfixable b/c the data might not be accessible to the client when it's time to send the event

Change 519738 had a related patch set uploaded (by DLynch; owner: DLynch):
[mediawiki/extensions/VisualEditor@master] ArticleTarget: saveSuccess wasn't receiving expected arguments

https://gerrit.wikimedia.org/r/519738

DLynch claimed this task.Jun 29 2019, 5:08 AM
DLynch moved this task from Incoming to Code review on the VisualEditor (Current work) board.

TODO: more patches for mobile.

Change 520119 had a related patch set uploaded (by DLynch; owner: DLynch):
[mediawiki/extensions/VisualEditor@master] Pass newRevId through to MobileFrontend

https://gerrit.wikimedia.org/r/520119

Change 520120 had a related patch set uploaded (by DLynch; owner: DLynch):
[mediawiki/extensions/MobileFrontend@master] On page edit save, pass new revision ID through for logging

https://gerrit.wikimedia.org/r/520120

Change 519738 merged by jenkins-bot:
[mediawiki/extensions/VisualEditor@master] ArticleTarget: saveSuccess wasn't receiving expected arguments

https://gerrit.wikimedia.org/r/519738

Change 520119 merged by jenkins-bot:
[mediawiki/extensions/VisualEditor@master] Pass newRevId through to MobileFrontend

https://gerrit.wikimedia.org/r/520119

Change 520120 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] On page edit save, pass new revision ID through for logging

https://gerrit.wikimedia.org/r/520120

ppelberg closed this task as Resolved.Jul 3 2019, 3:35 PM