Decide on better wording for "users were 1.07 times more likely to do X"
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• ksmith
	Jul 12 2016, 11:34 PM

Description

In https://www.mediawiki.org/wiki/Topic:T7cyg64hzu6aop69 someone pointed out that the sentence "We found that the test group had a 3.64% higher probability of interacting with the page and was 1.07 times more likely to engage with the page than the control group." can be confusing to people who don't specialize in math.

People not well-versed in math will have a hard time translating the 1.07 into a recognition that engagement increased by 75. I proposed replacing "was 1.07 times more likely to" with "was 7% more likely to", but @mpopov expressed concerns that the percentage wording would mislead people into thinking it was a probability.

The goal of this task is to find wording that is:

Correct
Accessible to people who are not specialists in math
Unlikely to lead to misinterpretations

Perhaps this has already been "solved" by folks in the Analytics team?

Event Timeline

• ksmith created this task.Jul 12 2016, 11:34 PM

Restricted Application added subscribers: Zppix, Aklapper. · View Herald TranscriptJul 12 2016, 11:34 PM

moving to the sprint board as this topic is undergoing discussion now.

We are going to wait until our new analyst starts working because they will probably have very valuable insights on this topic.

In T140187#2482425, @mpopov wrote:

We are going to wait until our new analyst starts working because they will probably have very valuable insights on this topic.

Marking as stalled based on this.

@chelsyx, @debt, and I talked about this and decided to try referring to things in terms of "out of a hundred" e.g.

if P(Clickthrough in Control Group) = 0.7, P(Clickthrough in Test Group) = 0.75,

then "in the control group without the new feature, we can expect 70 out of 100 sessions to click on a search result, while we can expect 75 out of 100 sessions with the new feature to click on a search result"

this is also credible interval-friendly: "we can expect 2-9 more sessions to click on a search result when they have the new feature"

@debt: it's done but don't resolve this yet until we end up using this phrasing in a report and people find it better (more accessible without losing accuracy)

I look forward to seeing this in context, but based on that fragment it looks awesome so far.

Great! :)

debt closed this task as Resolved.Sep 9 2016, 6:51 PM

debt claimed this task.

Updating assignment for clarity.

Decide on better wording for "users were 1.07 times more likely to do X"Closed, ResolvedPublicActions

Description

Event Timeline

Decide on better wording for "users were 1.07 times more likely to do X"
Closed, ResolvedPublic
Actions