Change Details

T283505 will introduce a fixed "sticky" site header to the desktop reading experience. To start, the sticky header will //NOT// contain editing functionality, per T294383. This task represents the work with running an A/B test to evaluate the impact introducing an edit affordance within desktop reading experience's fixed "sticky" site header has on the following: 1. The speed and ease with which contributors, across experience levels, can begin making a change to the content they are wanting to affect 2. How likely people are to publish the edits they start making 3. Peoples' awareness of their ability to edit the content they are consuming 4. The rate at which people make destructive changes to wikis === Decision to be made This A/B test will help us make the following decision: **Should edit affordance(s) within the sticky header be made available to more people?** //Note: we have yet to define what "more people" means in this context and defining it depends on knowing all of the people who have access to the sticky header at the time the test is run.// === Hypotheses To help evaluate the impact of the impact of introducing an edit affordance within desktop reading experience's fixed "sticky" site header, **we would like to know if doing so...** |ID|Hypothesis|Metric(s) for evaluation |---|---|--- |KPI|**...causes** people, across experience levels, to publish a greater number of edits on desktop because they will need to expel less effort to start an edit to fix something they noticed while reading. | //TBD. Metrics will be defined in T296264.// |Curiosity #1| **...causes** people to begin making a change to content more quickly and with greater ease because they will be able to locate the content they opened the editing interface seeking to affect with less effort.| //TBD. Metrics will be defined in T296264.// |Curiosity #2| **...causes** a greater number of people to open/explore editing because there will be an omnipresent edit affordance reminding them of the ability they have to affect the content they are consuming.| //TBD. Metrics will be defined in T296264.// |Guardrail #1|**...does not cause** a sustained decrease in the rate at which people publish the edits they initiate. | //TBD. Metrics will be defined in T296264.// |Guardrail #2|**...does not cause** a sustained increase in the rate at which people publish destructive edits. | //TBD. Metrics will be defined in T296264.// === Decision Matrix |ID|Scenario|Plan of action |---|---|--- |1.| People with access to the edit affordance within the sticky header publish a significantly **higher number** of edits than those who do not have access to the affordance | //TBD// | 2. |People with access to the edit affordance within the sticky header publish a significantly **lower number** of edits than those who do not have access to the affordance | //TBD// | 3. | People with access to the edit affordance within the sticky header publish a **nominally different number** of edits than those who do not have access to the affordance | //TBD// //Note: all of the scenarios above assume there is NOT a sustained decrease in the rate at which people publish the edits they initiate or a sustained increase in the rate at which people publish destructive edits.// === Participating Wikis //See: T298280.// NOTE: all wikis listed in T298280 //EXCEPT// for th.wiki and vi.wiki will have access to the sticky header without editing functionality included prior to the A/B test beginning. //More context in T298280#7641884.// === Done - [ ] A report is published that evaluates the `===Hypotheses` above

T283505 will introduce a fixed "sticky" site header to the desktop reading experience. To start, the sticky header will //NOT// contain editing functionality, per T294383. This task represents the work with running an A/B test to evaluate the impact introducing an edit affordance within desktop reading experience's fixed "sticky" site header has on the following: 1. The speed and ease with which contributors, across experience levels, can begin making a change to the content they are wanting to affect 2. How likely people are to publish the edits they start making 3. Peoples' awareness of their ability to edit the content they are consuming 4. The rate at which people make destructive changes to wikis === Decision to be made This A/B test will help us make the following decision: **Should edit affordance(s) within the sticky header be made available to more people?** //Note: we have yet to define what "more people" means in this context and defining it depends on knowing all of the people who have access to the sticky header at the time the test is run.// === Metrics We do **not** think this series of changes is likely to have a notable impact on a **single** metric that has a clear directional interpretation. //Said another way: we do not think there is a single metric that is likely to A) move in response to these changes *and* B) for the direction of that movement to indicate a clear improvement or degradation in peoples' user experience. As such, we will evaluate a collection of metrics and evaluate them holistically to decide whether the impact of this intervention was positive or negative.// |Impact| Metric | |---|--- |Content disruption|Proportion of published edits that are reverted within 48 hours of being made | Peoples' awareness of their ability to edit| Proportion of people who initiated at least one edit session during the course of the A/B test | Reduction in //overall// effort required to publish an edit| **i.** Average number of edits each person makes throughout the course of the A/B test //and// **ii.** Average rate at which people publish the edits they start | Reduction in effort required to //start// an edit| Average time required between when the editing interface is ready and people make a change to the document. === Decision Matrix //#TODO: fill out decision matrix using a similar format to T221198 where we did not depend on a single KPI.// |ID|Scenario|Plan of action |---|---|--- |1.| | === Participating Wikis //See: T298280.// NOTE: all wikis listed in T298280 //EXCEPT// for th.wiki and vi.wiki will have access to the sticky header without editing functionality included prior to the A/B test beginning. //More context in T298280#7641884.// === Done - [ ] A report is published that evaluates the `===Hypotheses` above

T283505 will introduce a fixed "sticky" site header to the desktop reading experience. To start, the sticky header will //NOT// contain editing functionality, per T294383. This task represents the work with running an A/B test to evaluate the impact introducing an edit affordance within desktop reading experience's fixed "sticky" site header has on the following: 1. The speed and ease with which contributors, across experience levels, can begin making a change to the content they are wanting to affect 2. How likely people are to publish the edits they start making 3. Peoples' awareness of their ability to edit the content they are consuming 4. The rate at which people make destructive changes to wikis === Decision to be made This A/B test will help us make the following decision: **Should edit affordance(s) within the sticky header be made available to more people?** //Note: we have yet to define what "more people" means in this context and defining it depends on knowing all of the people who have access to the sticky header at the time the test is run.// === Hypotheses To help evaluate the impact of the impact of introducing an edit affordance within desktop reading experience's fixed "sticky" site headerMetrics We do **not** think this series of changes is likely to have a notable impact on a **single** metric that has a clear directional interpretation. //Said another way: we do not think there is a single metric that is likely to A) move in response to these changes *and* B) for the direction of that movement to indicate a clear improvement or degradation in peoples' user experience. As such, **we would like to know if doing so...**we will evaluate a collection of metrics and evaluate them holistically to decide whether the impact of this intervention was positive or negative.// |ID|Hypothesis||Impact| Metric(s) for evaluation |---|---|--- | |KPI|**...causes** people, across experience levels, to publish a greater number of edits on desktop because they will need to expel less effort to start an edit to fix something they noticed while reading. | //TBD. Metrics will be defined in T296264.//---|--- |Curiosity #1| **...causes** people to begin making a change to content more quickly and with greater ease because they will be able to locate the content they opened the editing interface seeking to affect with less effort.| //TBD. Metrics will be defined in T296264.//ontent disruption|Proportion of published edits that are reverted within 48 hours of being made |Curiosity #2| **...causes** a greater number of people to open/explore editing because there will be an omnipresent edit affordance reminding them of the ability they have to affect the content they are consuming.| //TBD. Metrics will be defined in T296264.// Peoples' awareness of their ability to edit| Proportion of people who initiated at least one edit session during the course of the A/B test |Guardrail #1|**...does not cause** a sustained decrease in the rate at which people publish the edits they initiate. | //TBD. Metrics will be defined in T296264.// Reduction in //overall// effort required to publish an edit| **i.** Average number of edits each person makes throughout the course of the A/B test //and// **ii.** Average rate at which people publish the edits they start |Guardrail #2|**...does not cause** a sustained increase in the rate at which people publish destructive edits. | //TBD. Metrics will be defined in T296264.// Reduction in effort required to //start// an edit| Average time required between when the editing interface is ready and people make a change to the document. === Decision Matrix //#TODO: fill out decision matrix using a similar format to T221198 where we did not depend on a single KPI.// |ID|Scenario|Plan of action |---|---|--- |1.| People with access to the edit affordance within the sticky header publish a significantly **higher number** of edits than those who do not have access to the affordance | //TBD// | 2. |People with access to the edit affordance within the sticky header publish a significantly **lower number** of edits than those who do not have access to the affordance | //TBD// | 3. | | People with access to the edit affordance within the sticky header publish a **nominally different number** of edits than those who do not have access to the affordance | //TBD// //Note: all of the scenarios above assume there is NOT a sustained decrease in the rate at which people publish the edits they initiate or a sustained increase in the rate at which people publish destructive edits.// === Participating Wikis //See: T298280.// NOTE: all wikis listed in T298280 //EXCEPT// for th.wiki and vi.wiki will have access to the sticky header without editing functionality included prior to the A/B test beginning. //More context in T298280#7641884.// === Done - [ ] A report is published that evaluates the `===Hypotheses` above