Create a basic puppet role for Sentry. This won't include any of the high-perfomance options (noSQL backend, Redis buffer, worker queue etc.)
T84957 has a primitive puppet role but it needs improvements:
See also the Guidance on creating Debian packages for puppet ops thread from January. The consensus there seemed to be that apt packages are an overkill for this (even more true since then as upstream apparently switched gears with 7.0 and is moving very fast now) and trebuchet would be the best solution. (That said, if there is an easy way to do proper packaging, it would be valuable for 3rd party MediaWiki installs. getsentry.com has reasonably cheap hosted options, so not a big deal, just a nice extra.)
I know this isn't the main direction we've decided on, but I've tried my hand at packaging the django-* things with debianize.sh. It was reasonably easy, but if the resulting packages were incorrect, I'm not sure I would know :)
Here are the few debs I've put together so far:
https://dl.dropboxusercontent.com/u/109867/debs/python-django-crispy-forms_1.4.0_all.deb
https://dl.dropboxusercontent.com/u/109867/debs/python-django-paging_0.2.5_all.deb
https://dl.dropboxusercontent.com/u/109867/debs/python-django-recaptcha_1.0.3_all.deb
https://dl.dropboxusercontent.com/u/109867/debs/python-django-statsd-mozilla_0.3.14_all.deb
https://dl.dropboxusercontent.com/u/109867/debs/python-django-sudo_1.1.3_all.deb
https://dl.dropboxusercontent.com/u/109867/debs/python-django-templatetag-sugar_1.0_all.deb
If this is a route worth pursuing, even if we don't manage to package them all, should I create a repo for each deb?
Depending on how much manual work is required, it might be too much effort to keep it up - every time Sentry is upgraded, some or all of these packages would need to be upgraded as well.
I dicussed this with ops and recently with @hashar and there are two approaches that don't require one-by-one packaging:
So far it was very little work and updating the packages I've put together could be fully automated. The rest of the packages might be more complicated, though. I guess I'll try and see if I run into issues, but that shouldn't stop you from pursuing one of the two alternative solutions.
The use of dh-virtualenv is for Zuul T48552 . It is not validated by ops yet though. Note that whenever you update a python module package, it is updated on the whole cluster. The update can thus have interesting back compatibility side effects or be too risky.
I've turned what I had done manually into a shell script that will automatically package the latest version of these django extensions/apps dependencies: P421
Now I'll start looking at the other packages.
I've updated P421 and it now works with all missing packages except MySQL-python. I haven't tried the "sentry" package itself yet.
I almost forgot, but in production we should use pgsql, not mysql. The Sentry devs said mysql is not really supported - it works, but performance issues are fixed for pgsql only. (And psycopg2 has a Debian package, so no packaging needed for that.)
Excellent, P421 updated and sentry itself builds ok. Next I'm going to submit these debs for review. I expect that there might be some version mismatch for some of the dependencies that you found to be available, in which case we might have to package a few more this way, but it's straightforward. Fingers crossed on these automatically generated debs being alright by Ops!
Of course, I overlooked one very important issue: versions. Looking at Debian Jessie, I would have to downgrade 10 stock jessie python packages and upgrade 3 of them to make this work. That doesn't look like a future-proof plan, as anything sharing dependencies with Sentry will have problems if it needs entirely different versions. Oh well, at least now I fully understand why this route for python dependencies is a bad idea.
Posted the request for repo creation here https://www.mediawiki.org/wiki/Git/New_repositories/Requests
Repo is live at gerrit.wikimedia.org/r/operations/software/sentry and is a clone of yesterday's state of sentry on github
I'll target Jessie. The following packages have their version on Jessie match what the latest Sentry expects:
python-beautifulsoup
python-cssutils
python-django-crispy-forms
python-django-jsonfield
python-django-picklefield
python-ipaddr
python-mock
python-progressbar
python-pytest
python-redis
python-six
python-setproctitle
Those can be installed by puppet, I'll build a virtualenv for the rest that expects the above to be provided by the system.
My attempt at a venv referencing these local packages when available is sitting under /srv/deployment/sentry/sentry on the sentry-packaging.eqiad.wmflabs machine (a fresh Jessie instance). I've asked Christian if operations/software/sentry can be nuked (he created it by cloning Sentry and its git history) to store the venv there for Trebuchet.
@Gilles and I had a quick conf call yesterday. Seems the Debian packaging is going to be messy and it is much simpler to use a virtualenv deployed via git-deploy.
Python modules for which there are packages could be provided by puppet. Then a venv could be created based on Debian/Jessie + those python packages and then using virtualenv --system-site-packages. This way, it will not include / download from pypi modules that are packaged.
The end result can be then send to a git repo and git-deployed. That is similar to how we deploy Parsoid, npm being run and the result sent to a /deploy.git repo.
Not sure what will happen if some python-dev lib is updated on the system, that might cause compiled modules to have issues.
Right, I figured that if the system-provided packages end up being updated and cause problems, we might have to move some of these into the venv.
@dduvall and I are the point of contact for release engineering.
@Gilles I guess we can get Alexandros in the loop to validate the deployment model. Seems you will want to end up with a full virtual env that brings all the dependencies, does not seem to be too much of a problem to me since it is supported by a single server and the app has few links with other systems.
I removed the scrum of scrums and blocked on releng tags. I'll work with @Tgr to come up with a basic plan on how to move this forward but I don't think it is fair for us to try and stick it all on another group at this point.
Change 224987 had a related patch set uploaded (by Gergő Tisza):
Add salt configuration for Sentry
Change 225554 had a related patch set uploaded (by Gergő Tisza):
Update deployment_server rule to support Apache 2.4
Change 225554 merged by Yuvipanda:
trebuchet: Update deployment_server to support Apache 2.4
Change 227927 had a related patch set uploaded (by Gergő Tisza):
Private hiera keys for Sentry
Only code review of gerrit 199598. The patch does include a puppet role to provision a DB.
@Tgr I would like to merge https://gerrit.wikimedia.org/r/#/c/434539/ and I saw this role is not in production but it is used in deployment-prep -> https://tools.wmflabs.org/openstack-browser/puppetclass/role::sentry
Is that used?
@Dzahn it's experimental, and currently unmaintained. I hope to get back to it eventually.