We launched the experiment on August 29, 2022. On September 7, 2022, we updated the experiment to also include single-request users (not just regular subscribers). We have been monitoring SuggestBot to make sure everything is working as intended, and have noticed no issues so far. The experiment is expected to run for approximately 3 months. At the 3-month mark, we will see how many recommendation sets have been served and plug that into the power analysis code I set up before the experiment. If we determine the sample size to be large enough, we will stop the experiment. Otherwise, we will update here with a new timeline.

• @nettrom_WMF and I met last week to set up a limited deployment. I am currently getting modified/experiment recommendations and all other users are getting the normal recommendations. As of right now, there are no issues and SuggestBot is functioning as intended.
• IRB asked for one final change to the protocol. I resubmitted this morning, so we can probably expect approval by early next week. Full deployment of the experiment will happen ASAP after that.

Weekly updates:

Weekly updates:

Weekly updates:

• After some discussion, we decided to go with a much simplified algorithm for re-ranking items. The current implementation basically has a random chance (50%) of adding an additional filtering criterion (e.g., article must be a biography of a woman, or article must pertain to the global south).
• Did some testing of the aforementioned filtering. Had to decide on an appropriate fallback for when an article that meetings the additional filtering criteria cannot be found. Tested 1) falling back to recommending a random article that meets all filter criteria and 2) falling back to the normal, less stringent, filtering criteria. Decided on the latter.
• With the above completed, we have a (mostly) settled experimental design. The one big missing part is a filtering feature that is tied to campaigns as opposed to explicit criteria like gender and region. Brainstorming possibilities here and will test any plausible approaches.

Weekly updates:

• Spent a good amount of time thinking about the logic of the various parameters in Sonboli et al.'s re-ranking algorithm (esp. feature weights). Had a chat with Nasim Sonboli where she confirmed my intuitions about directionality of feature weights.
• Decided to go with a more heavily modified version of the Sonboli et al. algorithm.
• Still conducting offline analysis of re-ranking outputs. I've tried various weighting schemes, none of which have been great (very little effect on recs' composition), but am on the right track to finding reasonable algorithm parameters for the experiment by early next week.

Weekly updates:

Weekly updates: