We want to be able to answer these questions going forward,
- Are 'non-conflicting' edits causing the total number of chunks to be so large that is it easier to abandon?
- Track total number of chunks and correlation to success/abandonment
- What percentage of chunks have direct conflicts/overlapping edits in the same paragraph?
- Do users understand that they can edit?
- Track if editing before submission, track 'edit other' and 'edit yours' separately
- What is causing abandonment?
- When is it happening? How long are people spending on the page / how complete is the form?
Ensure that we have metrics to produce this data, and build the postprocessing ahead of when we'll need it.
Resources
The numbers below are calculated using this join script and this notebook.
Results
We have exit data beginning April 6th:
+-----------+-----+-------+ |exit_action|count|percent| +-----------+-----+-------+ | cancel| 36| 2.2| | unknown| 888| 53.3| | save| 742| 44.5| +-----------+-----+-------+ Registered user exits +-----------+-----+----------+ |exit_action|count|percentage| +-----------+-----+----------+ | cancel| 34| 2.1| | unknown| 858| 54.2| | save| 690| 43.6| +-----------+-----+----------+ Anonymous user exits +-----------+-----+----------+ |exit_action|count|percentage| +-----------+-----+----------+ | cancel| 2| 2.4| | unknown| 30| 35.7| | save| 52| 61.9| +-----------+-----+----------+ Average experience level of users according to outcome +-----------+--------------+-------------+ |exit_action|mean_editcount|std_editcount| +-----------+--------------+-------------+ | cancel| 14792| 29569| | unknown| 38723| 55944| | save| 29938| 51139| +-----------+--------------+-------------+
Total elapsed time spent in the conflict workflow, grouped by outcome. I'm very suspicious of how short the times are, I'll verify later using another data source. There are negative values and other smells.
+-----------+--------------+-------------+-------------+-------------+ |exit_action|mean_elapsed_s|std_elapsed_s|min_elapsed_s|max_elapsed_s| +-----------+--------------+-------------+-------------+-------------+ | cancel| 139.0| 499.0| -80| 3016| | unknown| 98.0| 706.0| -211| 16267| | save| 86.0| 953.0| -2876| 25658| +-----------+--------------+-------------+-------------+-------------+
Overlapping chunk summary. These are the count of server-calculated actual conflicts, not related to the number of UI rows.
+-------+-----------------+------------------+------------------+ |summary| user_editcount| conflict_chunks| conflict_chars| +-------+-----------------+------------------+------------------+ | count| 443| 443| 443| | mean|27603.18284424379|1.5914221218961626| 964.8081264108353| | stddev|55763.25081415555|2.1213456222435263|1452.0718079693656| | min| 0| 1| 0| | max| 519415| 25| 14600| +-------+-----------------+------------------+------------------+ +-----------+-----+-----------------------+----------------------+----------------------+---------------------+ |exit_action|count|mean_overlapping_chunks|std_overlapping_chunks|mean_overlapping_chars|std_overlapping_chars| +-----------+-----+-----------------------+----------------------+----------------------+---------------------+ | cancel| 19| 1.6| 2.5| 697.4| 559.7| | unknown| 197| 1.9| 2.9| 1134.2| 1832.6| | save| 227| 1.3| 1.0| 840.1| 1067.5| +-----------+-----+-----------------------+----------------------+----------------------+---------------------+
TODO:
- UI rows counts and interactions are still pending T248601: Adapt existing column metrics for talk page use case
- A second data stream, EditAttemptStep should become available once a patch is deployed which lets us easily join with the TwoColConflict events.