Change Details

Goal: design and implement an experiment to explore editor willingness to balance personalization with content equity in edit recommender systems. == Subtasks This is an attempt at a semi-complete listing -- we may choose to decline some of these steps: === Documentation [] Update article importance project meta page to summarize CSCW paper findings: https://meta.wikimedia.org/wiki/Research:Prioritization_of_Wikipedia_Articles/Importance [] Create new section on that meta page for this phase of the project (alternatively can create a subpage and link to it -- e.g., https://meta.wikimedia.org/wiki/Research:Prioritization_of_Wikipedia_Articles/Importance/SuggestBot [] Start Overleaf project for this work and write up offline analyses / motivation === Offline Analysis [x] Initial SuggestBot analysis [x] Initial Newcomer Tasks analysis: https://meta.wikimedia.org/wiki/Research:Prioritization_of_Wikipedia_Articles/Recommendation#Newcomer_Tasks [] Extend SuggestBot offline analysis to other languages [] Extend SuggestBot offline analysis to other recommendation systems (Newcomer Tasks) === Experiment [x] SuggestBot onboarding -- get code running fully [x] Power analysis: should be able to see small effects even with three experimental conditions across several months on enwiki [] Experiment design [] Get experimental code running on test instance [] Offline analysis of simulation of experiment on test instance to verify assumptions / power analysis [] Design review w/ Morten + Loren [] Coordinate with Morten for keeping Mo's fork and main branch in alignment so deploy/undeploy is easy [] Coordinate with Morten/Isaac to be maintainers for the code in case of issues [] Dry-run of code on SuggestBot official where we log what we would have done but don't change any recommendations for a week or two to make sure matching what we saw with Mo's fork [] Deployment to small pilot group to make sure no errors [] Deployment to all for at least one month and ideally a few months (decide in advance how many rec sets to cut off at) [] IRB approval [] Survey/Interviews

Goal: design and implement an experiment to explore editor willingness to balance personalization with content equity in edit recommender systems. == Subtasks This is an attempt at a semi-complete listing -- we may choose to decline some of these steps: === Documentation [x] Update article importance project meta page to summarize CSCW paper findings: https://meta.wikimedia.org/wiki/Research:Prioritization_of_Wikipedia_Articles/Importance/Vital_Articles [x] Create new section on that meta page for this phase of the project (alternatively can create a subpage and link to it: https://meta.wikimedia.org/wiki/Research:Prioritization_of_Wikipedia_Articles/Importance/SuggestBot [] Start Overleaf project for this work and write up offline analyses / motivation === Offline Analysis [x] Initial SuggestBot analysis [x] Initial Newcomer Tasks analysis: https://meta.wikimedia.org/wiki/Research:Prioritization_of_Wikipedia_Articles/Recommendation#Newcomer_Tasks [x] Extend SuggestBot offline analysis to other languages [] Extend SuggestBot offline analysis to other recommendation systems (Newcomer Tasks) === Experiment [x] SuggestBot onboarding -- get code running fully [x] Power analysis: should be able to see small effects even with three experimental conditions across several months on enwiki [x] Experiment design [x] Get experimental code running on test instance [] Offline analysis of simulation of experiment on test instance to verify assumptions / power analysis [x] Design review w/ Morten + Loren [x] Coordinate with Morten for keeping Mo's fork and main branch in alignment so deploy/undeploy is easy [x] Coordinate with Morten/Isaac to be maintainers for the code in case of issues [] Dry-run of code on SuggestBot official where we log what we would have done but don't change any recommendations for a week or two to make sure matching what we saw with Mo's fork [] Deployment to small pilot group to make sure no errors [] Deployment to all for at least one month and ideally a few months (decide in advance how many rec sets to cut off at) [] IRB approval [] Survey/Interviews

Goal: design and implement an experiment to explore editor willingness to balance personalization with content equity in edit recommender systems. == Subtasks This is an attempt at a semi-complete listing -- we may choose to decline some of these steps: === Documentation [x] Update article importance project meta page to summarize CSCW paper findings: https://meta.wikimedia.org/wiki/Research:Prioritization_of_Wikipedia_Articles/Importance/Vital_Articles [x] Create new section on that meta page for this phase of the project (alternatively can create a subpage and link to it -- e.g.,it: https://meta.wikimedia.org/wiki/Research:Prioritization_of_Wikipedia_Articles/Importance/SuggestBot [] Start Overleaf project for this work and write up offline analyses / motivation === Offline Analysis [x] Initial SuggestBot analysis [x] Initial Newcomer Tasks analysis: https://meta.wikimedia.org/wiki/Research:Prioritization_of_Wikipedia_Articles/Recommendation#Newcomer_Tasks [x] Extend SuggestBot offline analysis to other languages [] Extend SuggestBot offline analysis to other recommendation systems (Newcomer Tasks) === Experiment [x] SuggestBot onboarding -- get code running fully [x] Power analysis: should be able to see small effects even with three experimental conditions across several months on enwiki [x] Experiment design [x] Get experimental code running on test instance [] Offline analysis of simulation of experiment on test instance to verify assumptions / power analysis [x] Design review w/ Morten + Loren [x] Coordinate with Morten for keeping Mo's fork and main branch in alignment so deploy/undeploy is easy [x] Coordinate with Morten/Isaac to be maintainers for the code in case of issues [] Dry-run of code on SuggestBot official where we log what we would have done but don't change any recommendations for a week or two to make sure matching what we saw with Mo's fork [] Deployment to small pilot group to make sure no errors [] Deployment to all for at least one month and ideally a few months (decide in advance how many rec sets to cut off at) [] IRB approval [] Survey/Interviews