Update: Bob, Ramtin, and I talked about the details of how the experiment can look like, and we need around a week or two to sort out details. I'm off on the week on May 1st, so the experiment itself can realistically start in 3 weeks. In the mean time, of course, there is work to do:

• Choice of language: it makes most sense to run this test in English. Running it in another language right now requires updating the code base to handle text in other languages and this needs some days of coding and quality check. We should not do this for this very first experiment. So, let's fix the language for this to en.

• We will be doing user testing and that's in the realm of Abbey and Jonathan's work. I'll reach out to Jonathan to see if he has time to talk with Bob, Ramtin and I about details next week.

@MMiller_WMF if you're interested, please review the task description, especially the section on Description of the experiment. The details are sorted out now and we are close to the experiment point. I'll wait until 2018-05-31 for your feedback and then set the start time of the experiment accordingly. Can you also let us know if you can support us with sending emails to the recipients?

@bmansurov can you look into how we can send emails to enwiki users who have registered a Wikimedia account in a specific time period, excluding users that have opted out from being contacted by other users? Just FYI, the last time we send these emails was from Wikimedia Research [mailto:recommender-feedback@wikimedia.org]. For context, we want to send emails out every n hours, and as users register in a period of a week on enwiki. @MMiller_WMF feel free to jump in if you already have a solution for this, of course. :)

@leila, I've looked into sending emails to recently registered users. We can get the list by running an SQL query:

echo "select user_name, user_registration from enwiki.user order by user_id desc limit 60000;" | mysql -h analytics-store.eqiad.wmnet -A > users.txt

Note that filtering by user_registration is unbearably slow because the field is not indexed. We can post-process data to only get the users within a certain time frame. We can dry run SendBulkEmails.php to get the number of users who don't want to receive emails or blocked recommender-feedback@wikimedia.org. Depending on the number we can feed more/fewer users to the script.

In T190776#4239845, @bmansurov wrote:

@leila, I've looked into sending emails to recently registered users. We can get the list by running an SQL query:

echo "select user_name, user_registration from enwiki.user order by user_id desc limit 60000;" | mysql -h analytics-store.eqiad.wmnet -A > users.txt

Note that filtering by user_registration is unbearably slow because the field is not indexed. We can post-process data to only get the users within a certain time frame.

Ok. So in practice we should be able to do this, for example, twice a day and send emails out, right?

We can dry run SendBulkEmails.php to get the number of users who don't want to receive emails or blocked recommender-feedback@wikimedia.org. Depending on the number we can feed more/fewer users to the script.

Let's give it a try and do a dry-run, maybe with our own volunteer accounts.

Ok. So in practice we should be able to do this, for example, twice a day and send emails out, right?

Yes.

Let's give it a try and do a dry-run, maybe with our own volunteer accounts.

OK!

In T190776#4240327, @leila wrote:

Let's give it a try and do a dry-run, maybe with our own volunteer accounts.

Count me in. :)

FYI, I'm waiting to get access to send emails (although I have access to log into the server, I don't have access to send emails yet): T189285#4240614.

I've got the access and here are the results of the dry-run (i.e. you shouldn't have received any mails):

WARNING - User Bmansurov_(WMF) can't receive mail
INFO - Emailing LZia_(WMF) <leila@wikimedia.org>
INFO - Emailing Trizek_(WMF) <bevellin@wikimedia.org>
[2018-06-01 14:28:55] processed: 3 (0.3/sec); ok: 2 (66.7%); failed: 0 (0.0%); missing: 0 (0.0%); noreceive: 1 (33.3%); optedout: 0 (0.0%);
done.

This seems correct to me because I've opted out of receiving emails. I suppose you two didn't.

@leila, do we have access to the recommender-feedback@wikimedia.org account? We need to create a wiki account with this email address if we want it to be the sender. We can then create a subpage under this user's account that lists the usernames that opt out of emails. Once we create this account, I'll send test emails to us all.

In T190776#4248895, @bmansurov wrote:

I've got the access and here are the results of the dry-run (i.e. you shouldn't have received any mails):

WARNING - User Bmansurov_(WMF) can't receive mail
INFO - Emailing LZia_(WMF) <leila@wikimedia.org>
INFO - Emailing Trizek_(WMF) <bevellin@wikimedia.org>
[2018-06-01 14:28:55] processed: 3 (0.3/sec); ok: 2 (66.7%); failed: 0 (0.0%); missing: 0 (0.0%); noreceive: 1 (33.3%); optedout: 0 (0.0%);
done.

This seems correct to me because I've opted out of receiving emails. I suppose you two didn't.

yup. looks right.

@leila, do we have access to the recommender-feedback@wikimedia.org account?

@ori set that account. Not sure if he still has access to it.

@Aklapper o/ @ori is no longer with us. Would you be able to make one of us an admin of the recommender-feedback@wikimedia.org?

@Trizek-WMF is it possible that we send the announcement to enwiki while bmansurov is finalizing the email component? We know that we can start no later than 2 weeks from now, so we might as well notify the community to save some time with parallelization. What do you think?

You can send the announcement on en.wp anytime, if you are ready to potentially spend time there to reply to questions.

Focus on the fact that it will not impact active users and what are the future benefits for active users.

In T190776#4260685, @Trizek-WMF wrote:

You can send the announcement on en.wp anytime, if you are ready to potentially spend time there to reply to questions.

I'm ready. Which section is the most appropriate?

Focus on the fact that it will not impact active users and what are the future benefits for active users.

yup.

https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(miscellaneous) is the most appropriate place.

On it. Thanks!

@leila, while we're waiting on T196556, I've sent test emails to @leila and @Trizek-WMF. Let me know if we should proceed with using my email as the sender and be done with the emails.

@bmansurov I confirm that I've received your email.

It's better to wait for T196556 as we need a more formal looking account, ideally. Let's wait for that.

Village pump announcement at https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(miscellaneous)#Two_experiments:_Eliciting_New_Editor_Interest_and_Team_effect

prepared the form for privacy statement request. The form is with Ramtin for his review.

@leila, I've sent a test email. Unsubscribe workflow also seems to work. Waiting on the subject of the email and a couple of other things we talked about today.

The experiment started today.

I sent an update to enwiki VP about this experiment: https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(miscellaneous)#Experiment%3A_Eliciting_New_Editor_Interest

@leila, Here's the histogram of the minutes to first edit after registration from 07/05/2018 till 08/05/2018. During this time 43,436 users made their first edit. To briefly summarize, about 41,000 users made their first edit within the first 3 days. I'm also attaching the raw data below.

Context: https://meta.wikimedia.org/wiki/Research_talk:Voice_and_exit_in_a_voluntary_work_environment/Elicit_new_editor_interests

Is my support still needed? T191675: CL support for increasing contributor diversity experiments

@Trizek-WMF just responded on that task. sorry about the delay. I somehow missed the ping there.

I sent an update to EN-VP that we are done with the first stage of the experiment and starting the second stage: https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(miscellaneous)#Experiment:_Eliciting_New_Editor_Interests_[Update]

That update hasn't raised pulblic feedback: https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(miscellaneous)/Archive_59#Experiment:_Eliciting_New_Editor_Interests_[Update]
I haven't heard anything concerning your project.

We have a new experiment based on the learnings from the previous one. I put a note on enwiki's VP about it https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(miscellaneous)#Experiment:_Eliciting_New_Editor_Interests_(Update)

VP and the research talk page are quiet after the first batch of emails that went out yesterday. Let's proceed with the second batch after coordinating on IRC about a few questions. Stats of this round are being updated daily at https://meta.wikimedia.org/wiki/Research:Voice_and_exit_in_a_voluntary_work_environment/Elicit_new_editor_interests#Stats