This task is about investigating what – if any – technical issues could be contributing to the drop-off between the saveintent and savesuccess steps in the Reply Tool comment funnels. E.g. might people be encountering errors when attempting to save the comment they posted with the Reply Tool? Might the tool/server timeout when people are attempting to save the comment they posted with the Reply Tool? etc.
In T247139, we are looking at how people are engaging with the Reply Tool at the Arabic, Czech and Hungarian Wikipedias...wikis where the tool is available to everyone by default.
As part of the analysis we are doing in T247139 [i], we noticed that a non-trivial percentage of people who reached the saveintent step within the Reply Tool comment funnel (read: they clicked the Reply button) did not reach the savesuccess step (read: the comment they drafted in earlier steps was successfully published to the talk page).
You can observe this for yourself by reviewing the following:
|Reply Tool mode||saveintent --> savesuccess drop-off range||Data|
|source||Drop-off ranges from 0% (0 edits group) to 7% (1-4 edits group)||Comments made with wikitext|
|visual||Drop-off ranges from 0% (0 edits group) to 6% (1-4 edits group)||Comments made with VisualEditor|
- Document whether there are any observable technical issues that could be contributing to the drop-off rates mentioned in the ===Background section above
i. You can view the full report here: https://nbviewer.jupyter.org/github/wikimedia-research/Discussion-tools-analysis-2020/blob/master/Engagement-Metrics/Reply-Tool-Workflow-Engagement-Metrics.ipynb#By-Wiki