@NRodriguez I'm trying to understand how we measure realtime preview success. Is it a calculation from the data gathered in T298218?

To my mind, it seems like we should be looking at two primary numbers: how often RTP is used (i.e. the preview-realtime-on action), and how many edits are made. The hypothesis being that if people preview more they'll save less. Perhaps it'd be interesting to also look at the number of small edits, or those made very close together (idea being that these are more likely to be 'previewing' edits).

Given that the majority of previews currently are full-page reloads, it sort of makes sense that people don't bother with them and instead just hit publish. The preview time in both cases is the same, and if there's no further edits to make they've saved themselves a click.

Logging extra data is tracked in T306176 (but I'm not sure any of those extra bits are 100% required for a basic success metric).

To my mind, it seems like we should be looking at two primary numbers: how often RTP is used (i.e. the preview-realtime-on action), and how many edits are made. The hypothesis being that if people preview more they'll save less. Perhaps it'd be interesting to also look at the number of small edits, or those made very close together (idea being that these are more likely to be 'previewing' edits).

@Samwilson that's a good stab at a number that could be meaningful

I think if possible, it would be great to see:

Of the edits that published an edit after using real-time preview, how many did NOT get reverted?

Hi @MNeisler! We need your expertise for this task where we are trying to measure success for real-time preview.

• What data do you think we should be looking at?
• How should we look at the data?
• Is there any additional data that we should track besides T298218?

Thank you again for your support with this project!

Hi @HMonroy! I've tagged the Product-Analytics on this request. We will review and prioritize the request during our next board review meeting on Tuesday, 24 May 2022.

@HMonroy we don't have availability to take on work related to this, so I'm moving it to our tracking column.

We can help address your questions during our consultation hours; feel free to book 2 back-to-back sessions with Megan or someone else from our team if needed.