Pull in data gathered since going into our "small default" phase. This will have many more anonymous and new users than our previous sample.

• Transfer conflict-query repo to WMDE ownership and set up code review.
• Review shared notes and catalog which questions remain unanswered. Document conclusions here. If open questions require more work, outline that work here but try to limit the scope of this task.
• Write analytics function to summarize actual, displayed number of conflict rows. Don't get into the more advanced uses for this data, yet.
• Run conflict metrics for April 2020 and refresh notebooks.

Refreshed notebooks can be found here.

Open questions

These will not be investigated as part of the task, but should be documented as future directions for work, ideally as a wiki page somewhere.

• RQ1: How does displayed row count relate to workflow outcome? What are the risk factors causing high row count?
  • H1.1: Row count positively correlates with dropout rate.
  • H1.2: Row count is similar across namespaces.

• RQ2: What impact does the TwoColConflict interface have on users?
  • H2.1: Anonymous users can resolve a higher proportion of conflicts in our tool vs. the legacy workflow.
  • H2.2: Experienced users can resolve a higher proportion of conflicts in our tool vs. the legacy workflow.
  • H2.3: The percentage improvement for our tool will increase following a sigmoid learning curve, as users gain proficiency.
  • H2.4: The success percentage is roughly the same across all wikis.

• RQ3: Are success rates measured correctly?
  • H3.1: Proportion of exits by successful save vs. abandonment (minus duplicates due to page reload) is the same as the statsd success counts displayed in Grafana.

• RQ4: Is there a quality difference between legacy and TwoColConflict successes?
  • H4.1: Some legacy resolutions are simply submitting the textbox unchanged, losing their own edits (should even be considered failure), but we record as a success.
    • Yes. This degenerate case is recorded as a successful resolution in statsd. TODO: It's possible to find exactly how often this happens, by joining with EditAttemptStep and checking whether saveAttempt and saveSuccess have the same revision_id.

• RQ5: Is no-javascript browsing common for edit conflicts?
  • H5.1: Same as historical rate of ~6% of requests.

• RQ6: Can we learn anything about abandonment?
  • H6.1: Most abandonment is through the back button.
  • H6.2: Clicking "Cancel" is rare.

• RQ7: What causes the shadow 2/3 of conflicts coming from T246439 and T246440? Do users even see the conflict interface when this happens?
  • H7.1: Shadow new article conflicts result in either 100% abandonment (no interface displayed), or a high success percentage (trivial).
  • H7.2: Shadow self-conflicts have a very high successful outcome percentage.