Page MenuHomePhabricator

[WMDE-Fundraising] Set up server for continuous deployment
Closed, DeclinedPublic

Description

This task consists of two steps

  1. set up the server/virtual machine with a domain, web server, PHP 7, composer and MySQL. SSH permissions for all the team members.
  2. Set up an Ansible deployment script that
    • Checks out the latest version from the repository
    • Makes sure all the necessary files and directories (cache, log, tmp, etc) are there and have the right permissions
    • Install dependencies with composer

Event Timeline

  1. is done, the server is at deployment.wikimedia.de. If you did not get a SSH login, poke me when I'm in the office

The code for 2) is in the webhook repo. Before that can be used on the server, two additional things have to be implemented for the deployment shell scripts:

  • add a LoggingDeployerDecorator that checks the command success and alerts via email if the deploy failed.
  • add a FailureDetectionDeployerDecorator that checks the number of tries of a release and only deploys if date_added + (RETRY_INTERVAL_IN_SECONDS^num_tries) < NOW(). That way, failed deployments won't be tried every minute by the deployment scripts, spamming us with the failure notices. The goal is to have exponential timeouts between retries, something like 5 minutes, 10 minutes, 20 minutes, 30 minutes, 1 hour, 2 hours, etc. The formula probably has to be tweaked for that.

Thanks for describing what still needs doing! I'm having a look at this now.

Not sure a decorator is the best option for the logging. The public method is just getting a Process, so kinda hard to log anything meaningful then. Could just put in the logger interface... Or decouple a bit more and have an onDeploymentFailed handler. Will try out the later.

Did not get very far yet: https://github.com/wmde/github-webhooks/pull/8 Will continue with that tomorrow if no one beats me to it.

I like the idea of having a callback. If you use monolog as a logging library, you can use several loggers - a one that logs to a file and one that notifies via email.

Do you know how to send nice mails using Monolog? If so, probably good if you do that part of the task so we save the time of me looking up the stuff. I'll already do the file logging part now.

With https://github.com/wmde/github-webhooks/pull/12 we now have a logger called with the needed info. Still need to create the real logger that does disk and mail stuff. Calling this quits for today, so feel free to do that part (or the retry thing).

Why do we need this retrying thing? Why would it fail? And if it does, why would it work on a successive try? And if there is good reason to have this, can't we already start using the app without having this done, and track it via a dedicated task?

The deployment could fail due to external (Github down, npm down, composer)
and internal factors (missing config entry, ssh key changed). For all those
things it would be nice to have an automatic retry instead of having to
manually log in to the deployment server and do a deploy.

You're right that this is a nonessential feature.

This task is now broken down and removed from the FUN-Team workboard.

If I understand correctly we no longer plan to do this, so let's close these tasks?