Hi! Interesting and useful questions, for sure... Adding Fundraising tag just in case we can provide some very general guidance, or maybe refer you to someone who could actually work on this.

@AndyRussG it would be great if you could help lay the connection with the right people :)

This request was the result from a thread between community organizers, that had some doubts about the effectiveness of a campaign with certain diets. Hopefully with this kind of digging, we get a feeling for what the impact of a diet would be on effectiveness of campaigns.

In T290387#7333165, @Effeietsanders wrote:

This request was the result from a thread between community organizers, that had some doubts about the effectiveness of a campaign with certain diets. Hopefully with this kind of digging, we get a feeling for what the impact of a diet would be on effectiveness of campaigns.

Yes, that makes a lot of sense. :) Just in case this is useful, I can mention that Fundraising has found that only the first few banners shown to a user have much likelihood of getting a response. However, reactions could certainly be different for a campaign like WLM.

Yeah, one of the main reasons why people think this might be different, is that people may have to take a photo before they can actually upload them. So they may see the banner, go take a picture, and then come back to the banner to upload.
Not sure if that actually happens in practice, but that may be one of the plausible reasons I heard why things may be different. There is some intuition that there is a diminishing return though - where the optimum lies, is not obvious to me :) .

adding @Sadads for visibility. I hear he is fielding community campaign questions these days.

@Effeietsanders this is not something we (Product Analytics) will be able to look into in the foreseeable future, but I'm adding @Iflorez and @ifried as subscribers so they can keep this need in mind as they work on Campaigns-Product-Team

(Disclaimer: I have volunteered for Wiki Loves Monuments for multiple years and understand and empathize with the needs of the international team and local organizers first-hand.)

@Effeietsanders I discussed multiple options with the different teams within WMF who could potentially support your request and here is what I can offer:

• Our team, Research, would be happy to set up an experiment to see how we can support WLM's needs in this case.
• We do this as part of our commitment to improve equity in decision making.
• We won't be able to answer the specific questions you asked as we expect you all to run into more questions as you learn more from the data and we feel it's important to enable you all to answer them directly. Of course, this won't be super easy, hence the Experiment.

Below are more details about the experiment:

• We ask that you nominate (privately to me) 1 person from your community who will enter into a formal relationship with WMF. This means, they will have to sign a Memorandum of Understanding (valid for 6 months) and Non-disclosure agreement with WMF. (We will be closely following the process that we have in place for Formal Collaborations.)
• We will give your approved nominated person access to the relevant data and @MGerlach, a research scientist in our team, will be their Point of Contact for the duration of the six months.
• Martin will help the individual learn how to process very large data on our servers. He will also support them in developing their research skills as it relates to the request.
• I'll send you some requirements for the individual you can nominate.
• If we go with the above, I expect that in six months time we review and decide whether to extend the relationship.

For transparency,

• At this point, we will only be able to pick up this experiment with one group, WLM.
• At this moment, our team is not equipped to scale this kind of access and relationship should we find the experiment to be mutually desirable and successful. If we see this is the kind of service we want to offer to the communities at large, we need to find answers to many questions that will arise and will have impact on the work of many teams within WMF: Legal, Security, Research, Data Engineering and more.
• The above being said, we have an alignment of stars at this point and I believe it's important to seize the moment, experiment, and learn to make more informed decisions.

Sooo, if the above sounds good to you, let me know here and let's pick up the nomination discussion over email.

Thanks @leila for this helpful response. We will discuss internally and follow up with you over email following that.

Update:

• I heard back from Effeietsanders and they confirmed that the WLM international team has nominated them as the 1 person who will work on the team's questions.
• MGerlach will be Effeietsanders' point of contact for the duration of their access and research.
• I have added the Research tag to this task so we don't lose track of it on our team's end.
• MGerlach is coordinating the next steps with Effeietsanders over email. Sub-tasks may be created for this task to track that work or not. I will leave that decision to MGerlach and Effeietsanders.
• To all other teams/projects: feel free to remove your tags if this task is no longer of interest to you. I didn't touch existing tags. :)

Thanks all!

@Effeietsanders am I correct that the paper on this research is now published here and I can close this task as resolved?

No, this is unrelated.

@Effeietsanders can you give an update on the status of this task? thanks!

After much back and forth (sorry for the delays!) I finally published the write-up here: https://meta.wikimedia.org/wiki/Research:Effectiveness_of_the_new_participant_pipeline_for_Wiki_Loves_campaigns#Outcomes

@Effeietsanders thanks and congrats! Is there anything else left in this task? if not, can you "Resolve" it? thanks!

As for the status:

• As far as the research project is concerned, this is probably how far it'll go.
• It did not answer the questions we set out to answer to the level of detail we hoped for, but gave some insights.

There is some potential to continue this task. With the script available in Github, it should now be much easier to adapt it, and repeat the data collection for an entire campaign. The meta page includes some specific pointers for future steps.

Specifically:

• To establish the effect of a diet, a randomized controlled trial (A/B test) would be the most logical avenue.
• The data analysis pipeline would need to be edited to establish how many banners someone has seen before arriving on the landing page. This may also require a slightly different set of assumptions with regards to UTC-days.
• We could also do a better job in counting how often someone visited the landing page before making their first attempt at uploading

We have observed stark differences between countries/landing pages. To better understand this, a more qualitative approach might be called for (survey/interviews).

Thanks for expanding @Effeietsanders . I'm going to remove the task from Research backlog (we're changing the way we work on Phabricator and we're keeping tasks in the backlog lane when we expect to work on them in the coming 6 months). When you conclude the conversations/deliberations and if you need our support, please add Research back as a tag. We will then review and triage.