Page MenuHomePhabricator

Set up maintenance routine for servers and websites
Open, HighPublic

Description

During the server migration we've had some turbulence with respect to who should keep an eye on what and as a consequence some of our websites have gone unpatched (and there is at least one confirmed security incidence).

Historically this has had severe negative impact on a variety of our Wordpress sites, our Piwik installation, SSL certificates as well as our main Drupal site.

To ensure this does not happen again we should
a) Inventory all

  • websites/servers we maintain
  • components on these which may need monitoring for updates/patches
  • ensure there is clear documented info for HOW these are updated patched
  • ensure there is one person assigned with the main responsibility for keeping these up-to-date

b) For each component identified above. Ensure that drift@ is subscribed to the relevant feeds/lists for security announcements
c) Ensure that there is

  • a schedule for regularly checking in to all of the identified components/websites/servers and updating them
  • A protocol so that we can follow up on this having been done
  • Time/budget set aside to ensure that this maintenance can be performed without competing for time with other responsibilities.

d) Ensure we have someone we can turn to (with rather short notice) when shit hits the fan and fixing the problem is beyond the skill set of our in-house staff.

Event Timeline

Lokal_Profil triaged this task as High priority.EditedMay 8 2018, 1:38 PM

I created this after

  • last weeks incident,
  • the fact that GDPR increases the demands on us,
  • the variety of new services we are getting through Ola which we will take over maintenance of
  • the lack of documentation discovered during T192556: Inventory of FSdata and Glesys of what we are running where, how it was installed and how it should be maintained.

The inventory could probably be initialised by @Sebastian_Berlin-WMSE (I'm happy to do some initial discussions on structure).

The last two points need input from @Jopparn, a discussion regarding that could ideally be scheduled for some point in May after the 25th

We need to look into this shortly.

Lokal_Profil renamed this task from Set up maintenance routine for servers and webbsites to Set up maintenance routine for servers and websites.Sep 25 2019, 1:03 PM

ensure there is clear documented info for HOW these are updated patched

I started a page on the wiki for this (I couldn't find one from before): https://se.wikimedia.org/wiki/Underhåll_av_servrar. I added a quick description of Matomo.