Page MenuHomePhabricator

Puppetize the meet account manager
Closed, ResolvedPublic

Description

The account manager of Wikimedia meet is two flask applications that are currently run on screen which is... suboptimal. Having them run as uwsgi and using puppet would be awesome.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 25 2020, 7:55 PM
Dzahn added a subscriber: Dzahn.Apr 27 2020, 8:06 AM

I'd be happy to help with puppetizing. A first step would be to move the repo from github to Gerrit. Then we can have puppet clone the source.

Should i start with requesting the project?

I'd be happy to help with puppetizing.

Thank you! You also helped with codesearch too, thank you <3

> A first step would be to move the repo from github to Gerrit. Then we can have puppet clone the source.

Should i start with requesting the project?

Is that really needed? This is CloudVPS and not in production. puppet can talk to the outside AFAIK. I agree if we want to move it to production one day™️, we should move it to gerrit. I don't mind either way though.

The longer we wait moving it the more it will be an issue to lose git history. Things that are temporary have a habit of becoming permanent.

Would like to learn some puppet so happy to help too, if I will be useful in any way

Dzahn added a comment.Apr 29 2020, 9:28 AM

@Majavah I am happy to add you as reviewer or CC to any puppet changes in Gerrit. Have you logged in there before? See https://www.mediawiki.org/wiki/Gerrit

@Dzahn I might have done something sometimes.

Dzahn added a comment.Apr 29 2020, 9:32 AM

@Majavah Very cool! I will add you so you can see the puppet changes and get notified.

@Dzahn (for the record, the intended meaning of T251034#6092621 was to tell that I've been around for a while and learning Puppet has been on my bucket list for a while)

jbond added a subscriber: jbond.Apr 29 2020, 9:36 AM

Change 593233 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] puppetize meet-accountmanager (WIP)

https://gerrit.wikimedia.org/r/593233

Dzahn added a comment.Jun 9 2020, 9:51 AM

@Ladsgroup Trying to move forward with this I went and SSHed to the instance called "jitsi" in the meet project.

I found that the checkout of meet-accountmanager repo seems to be in /srv/meet-auth.

I then compared git log with the contents of the Gerrit repo and saw differences. git config showed me the origin of the deployed repo is github, not gerrit. They do have a common history but some newer merges have not been deployed.

If i merge this change puppet would start pulling from Gerrit instead.

Can we delete /srv/meet-auth and let puppet re-clone it?

@Ladsgroup Trying to move forward with this I went and SSHed to the instance called "jitsi" in the meet project.

I found that the checkout of meet-accountmanager repo seems to be in /srv/meet-auth.

I then compared git log with the contents of the Gerrit repo and saw differences. git config showed me the origin of the deployed repo is github, not gerrit. They do have a common history but some newer merges have not been deployed.

Yeah because jitsi is a client of the account manger. The one that really matters is "meet-auth".

If i merge this change puppet would start pulling from Gerrit instead.

Can we delete /srv/meet-auth and let puppet re-clone it?

Yeah, it should be fine.

Dzahn added a comment.Jun 9 2020, 11:54 AM

Yeah because jitsi is a client of the account manger. The one that really matters is "meet-auth".

But on the "meet-auth" instance there is nothing in /srv/. It's just empty. While the jitsi instance has /srv/meet-auth. That confuses me now if meet-auth is the important one. Where would the git clone directory be then?

If i merge this change puppet would start pulling from Gerrit instead.

Can we delete /srv/meet-auth and let puppet re-clone it?

Yeah, it should be fine.

Ok, cool.

Change 593233 merged by Dzahn:
[operations/puppet@production] puppetize meet-accountmanager, step 1

https://gerrit.wikimedia.org/r/593233

Dzahn renamed this task from Puppetize the account manager to Puppetize the meet account manager.Jun 10 2020, 7:42 AM

Change 604409 had a related patch set uploaded (by Majavah; owner: Majavah):
[operations/puppet@production] Add wsgi-file for meet-accountmanager

https://gerrit.wikimedia.org/r/604409

Change 604409 merged by Dzahn:
[operations/puppet@production] Add wsgi-file for meet-accountmanager

https://gerrit.wikimedia.org/r/604409

Dzahn added a comment.Jun 10 2020, 3:14 PM

The role meet::accountmanager has been applied to the instance meet-auth.

Puppet ran and created the user/group "meet-auth".

Then I moved the /srv/meet-auth directory out of the way to /root and ran puppet again and it git-cloned the meet-accountmanager repo into /srv/meet-auth.

There are now 3 instances of

/usr/bin/uwsgi --die-on-term --ini /etc/uwsgi/apps-enabled/meet-accountmanager.ini

running.

But the process "python3 server.py" that is running since Apr 29 (so pre-puppet) is still the only one listening on any port.

Change 604434 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/puppet@production] meet: Change the account manager socket

https://gerrit.wikimedia.org/r/604434

Change 604434 merged by Dzahn:
[operations/puppet@production] meet: Change the account manager socket

https://gerrit.wikimedia.org/r/604434

Change 604444 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/puppet@production] meet: Use python3 in uwsgi

https://gerrit.wikimedia.org/r/604444

Change 604444 merged by Dzahn:
[operations/puppet@production] meet: Use python3 in uwsgi

https://gerrit.wikimedia.org/r/604444

Change 604773 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/puppet@production] meet: Add ferm rule to open port 5000 to the cloud proxy

https://gerrit.wikimedia.org/r/604773

Change 604773 merged by Dzahn:
[operations/puppet@production] meet: Add ferm rule to open port 5000 to the cloud proxy

https://gerrit.wikimedia.org/r/604773

Ladsgroup closed this task as Resolved.Jun 11 2020, 11:00 PM
Ladsgroup assigned this task to Dzahn.

Except the secrets, this is done. The secrets seems to be not easy to do in cloud. We will try to do it once we are in prod