Change Details

Pull in data gathered since going into our "small default" phase. This will have many more anonymous and new users than our previous sample. * Transfer conflict-query repo to WMDE ownership and set up code review. * Review [[ https://etherpad.wikimedia.org/p/TwoColConflict_post-deployment | shared notes ]] and catalog which questions remain unanswered. Document conclusions here. If open questions require more work, outline that work here but try to limit the scope of this task. * Write analytics function to summarize actual, displayed number of conflict rows. Don't get into the more advanced uses for this data, yet. * Run conflict metrics for April 2020 and refresh notebooks.

Pull in data gathered since going into our "small default" phase. This will have many more anonymous and new users than our previous sample. * Transfer conflict-query repo to WMDE ownership and set up code review. * Review [[ https://etherpad.wikimedia.org/p/TwoColConflict_post-deployment | shared notes ]] and catalog which questions remain unanswered. Document conclusions here. If open questions require more work, outline that work here but try to limit the scope of this task. * Write analytics function to summarize actual, displayed number of conflict rows. Don't get into the more advanced uses for this data, yet. * Run conflict metrics for April 2020 and refresh notebooks. == Questions * RQ1: How does displayed row count relate to workflow outcome? What are the risk factors causing high row count? * `H1.1`: Row count positively correlates with dropout rate. * `H1.2`: Row count is similar across namespaces. * RQ2: Is the TwoColConflict interface helpful? * `H2.1`: Anonymous users can resolve a higher proportion of conflicts in our tool vs. the legacy workflow. * `H2.2`: Experienced users can resolve a higher proportion of conflicts in our tool vs. the legacy workflow. * `H2.3`: The percentage improvement for our tool will increase following a sigmoid learning curve, as users gain proficiency. * `H2.4`: The success percentage is roughly the same across all wikis. * RQ3: Are success rates measured correctly? * `H3.1`: Proportion of exits by successful save vs. abandonment (minus duplicates due to page reload) is the same as the statsd success counts displayed in Grafana. * RQ4: Is there a quality difference between legacy and TwoColConflict successes? * `H4.1`: Some legacy resolutions are simply submitting the textbox unchanged, losing their own edits (should even be considered failure), but we record as a success. * RQ5: Is no-javascript browsing common for edit conflicts? * `H5.1`: Same as historical rate of ~6% of requests. * RQ6: Can we learn anything about abandonment? * `H6.1`: Most abandonment is through the back button. * `H6.2`: Clicking "Cancel" is rare. * RQ7: What causes the shadow 2/3 of conflicts coming from T246439 and T246440? Do users even see the conflict interface when this happens? * `H7.1`: Shadow new article conflicts result in either 100% abandonment (no interface displayed), or a high success percentage (trivial). * `H7.2`: Shadow self-conflicts have a very high successful outcome percentage.

Pull in data gathered since going into our "small default" phase. This will have many more anonymous and new users than our previous sample. * Transfer conflict-query repo to WMDE ownership and set up code review. * Review [[ https://etherpad.wikimedia.org/p/TwoColConflict_post-deployment | shared notes ]] and catalog which questions remain unanswered. Document conclusions here. If open questions require more work, outline that work here but try to limit the scope of this task. * Write analytics function to summarize actual, displayed number of conflict rows. Don't get into the more advanced uses for this data, yet. * Run conflict metrics for April 2020 and refresh notebooks. == Questions * RQ1: How does displayed row count relate to workflow outcome? What are the risk factors causing high row count? * `H1.1`: Row count positively correlates with dropout rate. * `H1.2`: Row count is similar across namespaces. * RQ2: Is the TwoColConflict interface helpful? * `H2.1`: Anonymous users can resolve a higher proportion of conflicts in our tool vs. the legacy workflow. * `H2.2`: Experienced users can resolve a higher proportion of conflicts in our tool vs. the legacy workflow. * `H2.3`: The percentage improvement for our tool will increase following a sigmoid learning curve, as users gain proficiency. * `H2.4`: The success percentage is roughly the same across all wikis. * RQ3: Are success rates measured correctly? * `H3.1`: Proportion of exits by successful save vs. abandonment (minus duplicates due to page reload) is the same as the statsd success counts displayed in Grafana. * RQ4: Is there a quality difference between legacy and TwoColConflict successes? * `H4.1`: Some legacy resolutions are simply submitting the textbox unchanged, losing their own edits (should even be considered failure), but we record as a success. * RQ5: Is no-javascript browsing common for edit conflicts? * `H5.1`: Same as historical rate of ~6% of requests. * RQ6: Can we learn anything about abandonment? * `H6.1`: Most abandonment is through the back button. * `H6.2`: Clicking "Cancel" is rare. * RQ7: What causes the shadow 2/3 of conflicts coming from T246439 and T246440? Do users even see the conflict interface when this happens? * `H7.1`: Shadow new article conflicts result in either 100% abandonment (no interface displayed), or a high success percentage (trivial). * `H7.2`: Shadow self-conflicts have a very high successful outcome percentage.