Page MenuHomePhabricator

Request creation of Gratitude VPS project
Closed, ResolvedPublic

Description

Project Name: gratitude

Wikitech Usernames of requestors: Maximilianklein, Epenn-cs, Rubberpaw

Purpose: This will host the production data-pipeline for the CivilServant gratitude study. https://meta.wikimedia.org/wiki/CivilServant's_Wikimedia_studies#Gratitude_prompts

Brief description:
That is it will monitor several wikis for users who are eligible for an approved study on the effects of giving extra thanks, and index their recent edits. It will include python, multiple connections to the database replicas, a dedicated mysql-server, redis, and cron.
All software will MIT Licensed. https://pypi.org/project/civilservant/.

Estimated Resources: 8-16GB Ram, 2-4 CPU, 100-200GB Storage.

How soon you are hoping this can be fulfilled: 2 weeks would be acceptable. We are hoping to Launch the application in June 2019, so would require need time.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 16 2019, 11:41 PM
bd808 added a subscriber: bd808.Apr 17 2019, 12:36 AM

If the redis usage can be pseudo-namespaced by prepending all the keys with a random string, I think this could run as a tool in Toolforge. The wiki replicas are there as well as per-tool databases on ToolsDB. Cron and redis are also well supported in that environment. Toolforge cron jobs should run on the job grid which may or may not complicate configuration and deployment.

bd808 added a comment.Apr 23 2019, 4:18 PM

Discussed in WMCS team meeting, approved for creation, but still suggesting Toolforge if possible. Waiting for feedback from @notconfusing about preferred next steps.

Hello @bd808 , I would prefer to get the VPS if possible. The reason being is that our stack is built on top of mysql8, and I'm not sure what issues lie ahead with using the Maria-DB per-tool instances . Our stack was built 2-years ago without toolforge compatibility in mind. I appreciate your response that it would be possible to run toolforge, and was going to roll-up my sleeves to try just that, until I saw your response about approval in the WMCS meeting. Here is another argument I would make for needing the VPS: (A) if our experiment is successful ("does nudged-thanking improve editor retention and performance?") then we would convert the experimental software to be a tool that would run for all users, which definitely need lots of disk for cacheing, probably necessitating a VPS. Otherwise, if (B) the experiment is not successful we would just spin down the VPS anyway.

So if it doesn't bother you, the VPS would save me a lot of headache. Thanks for your consideration.

Just to be clear - this system is just going to be processing publicly available data from the wikis, right? You won't be processing anything directly from users with it? There won't be surveys hosted on it or anything

The reason being is that our stack is built on top of mysql8, and I'm not sure what issues lie ahead with using the Maria-DB per-tool instances . Our stack was built 2-years ago without toolforge compatibility in mind. I appreciate your response that it would be possible to run toolforge, and was going to roll-up my sleeves to try just that, until I saw your response about approval in the WMCS meeting. Here is another argument I would make for needing the VPS: (A) if our experiment is successful ("does nudged-thanking improve editor retention and performance?") then we would convert the experimental software to be a tool that would run for all users, which definitely need lots of disk for cacheing, probably necessitating a VPS. Otherwise, if (B) the experiment is not successful we would just spin down the VPS anyway.

The operating system currently available is Debian Stretch, which has a package for MariaDB 10.1. Is the plan to install from external repositories?

Hello @Krenair ,
Thanks for your response:

  1. We will only be processing publicly available data from the wikis, correct. There will be no survey data, or other data collected directly from the user on it.
  2. Yes, we would install mysql8 from external repos on the VPS.

I hope that clarifies your questions, and thanks again for your diligence.

Andrew added a subscriber: Andrew.May 14 2019, 4:16 PM

Approved -- we'll create this in the next few days.

Andrew closed this task as Resolved.May 16 2019, 5:22 PM
Andrew claimed this task.
Andrew added subscribers: Rubberpaw, Maximilianklein.

I've created this project and made @Maximilianklein and @Rubberpaw project admins. I'm unable to locate a dev account for 'Epenn-cs' but one of the others can add them if they have an account.