Page MenuHomePhabricator

Upgrade bohrium (piwik/matomo) to Debian Stretch
Closed, ResolvedPublic8 Estimated Story Points

Description

Bohrium is a Ganeti VM so we'd need a new one to properly upgrade to Debian Stretch. A high level plan would be:

  1. Create a new Ganeti VM with Debian Stretch
  2. Deploy Piwik/matomo to it and test that everything runs as expected.
  3. Plan a controlled outage in which piwik is stopped, and the database is dumped and recreated in the new host.
  4. Point piwik.wikimedia.org's Varnish director to the new host.

Event Timeline

elukey triaged this task as Medium priority.Aug 28 2018, 7:09 AM
elukey created this task.
elukey changed the task status from Open to Stalled.Aug 28 2018, 7:16 AM

VM requested in T202963

There seems to be a new upstream version - 3.6.0 - that contains some security fixes but it is not available via debian.matomo.org's mirror. https://github.com/matomo-org/matomo-package/issues/81 was opened to track the issue. It would be great to deploy on the new VM 3.6.0 directly to avoid more downtime for the upgrade.

Change 463762 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] Add matomo1001 to puppet

https://gerrit.wikimedia.org/r/463762

Change 463762 merged by Elukey:
[operations/puppet@production] Add matomo1001 to puppet

https://gerrit.wikimedia.org/r/463762

Change 463781 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] profile::piwik: move to mariadb

https://gerrit.wikimedia.org/r/463781

Change 463781 abandoned by Elukey:
profile::piwik: move to mariadb

https://gerrit.wikimedia.org/r/463781

Change 463783 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] profile::piwik: move to mariadb

https://gerrit.wikimedia.org/r/463783

Change 463783 merged by Elukey:
[operations/puppet@production] profile::piwik: move to mariadb

https://gerrit.wikimedia.org/r/463783

Change 463797 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] profile::piwik::webserver: install php7.0-mbstring on Stretch

https://gerrit.wikimedia.org/r/463797

Change 463797 merged by Elukey:
[operations/puppet@production] profile::piwik::webserver: install php7.0-mbstring on Stretch

https://gerrit.wikimedia.org/r/463797

Change 463810 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] matomo: replace last piwik occurrences

https://gerrit.wikimedia.org/r/463810

Change 463810 merged by Elukey:
[operations/puppet@production] matomo: replace last piwik occurrences

https://gerrit.wikimedia.org/r/463810

Change 463906 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] Add prometheus mysql exporter for analytics-meta and matomo

https://gerrit.wikimedia.org/r/463906

Change 463906 merged by Elukey:
[operations/puppet@production] Add prometheus mysql exporter for analytics-meta and matomo

https://gerrit.wikimedia.org/r/463906

Change 463915 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] role::prometheus::analytics: add mysqld exporter references

https://gerrit.wikimedia.org/r/463915

Change 463915 merged by Elukey:
[operations/puppet@production] role::prometheus::analytics: add mysqld exporter references

https://gerrit.wikimedia.org/r/463915

Change 463918 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] role::prometheus::analytics: add mysql jobs to the config

https://gerrit.wikimedia.org/r/463918

Change 463918 merged by Elukey:
[operations/puppet@production] role::prometheus::analytics: add mysql jobs to the config

https://gerrit.wikimedia.org/r/463918

Change 463919 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] role::prometheus::analytics: fix mysql exporter config

https://gerrit.wikimedia.org/r/463919

Change 463919 merged by Elukey:
[operations/puppet@production] role::prometheus::analytics: fix mysql exporter config

https://gerrit.wikimedia.org/r/463919

Change 463920 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] profile::piwik::database: deploy the mysql exporter on jessie

https://gerrit.wikimedia.org/r/463920

Change 463920 merged by Elukey:
[operations/puppet@production] profile::piwik::database: deploy the mysql exporter on jessie

https://gerrit.wikimedia.org/r/463920

Change 463944 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] profile::piwik: increase innodb buffer pool

https://gerrit.wikimedia.org/r/463944

Change 463944 merged by Elukey:
[operations/puppet@production] profile::piwik: tune mariadb configs

https://gerrit.wikimedia.org/r/463944

Change 463947 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] matomo: remove plugin not compatible with 3.5.1

https://gerrit.wikimedia.org/r/463947

Change 463952 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] profile::piwik::webserver: deploy php7.0-xml

https://gerrit.wikimedia.org/r/463952

elukey changed the task status from Stalled to Open.Oct 2 2018, 1:17 PM
elukey edited projects, added Analytics-Kanban; removed Patch-For-Review.
elukey moved this task from Next Up to In Progress on the Analytics-Kanban board.

Change 463947 merged by Elukey:
[operations/puppet@production] matomo: remove plugin not compatible with 3.5.1

https://gerrit.wikimedia.org/r/463947

Change 463952 merged by Elukey:
[operations/puppet@production] profile::piwik::webserver: deploy php7.0-xml

https://gerrit.wikimedia.org/r/463952

Change 463957 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] role::cache::text: add matomo1001 backend for piwik.wikimedia.org

https://gerrit.wikimedia.org/r/463957

matomo1001 is ready to take traffic! The switch procedure should be:

  1. announce the downtime
  2. put piwik on bohrium in read only mode (no inserts in the db) and dump the piwik database
  3. copy the dump to matomo1001 and load it
  4. verify that everything works as expected
  5. merge https://gerrit.wikimedia.org/r/463957 and wait for bohrium to drain.

Change 463957 abandoned by Elukey:
role::cache::text: add matomo1001 backend for piwik.wikimedia.org

Reason:
Going to split the change in two parts as suggested by Brandon

https://gerrit.wikimedia.org/r/463957

Change 464110 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] role::cache::text: add a backed for matomo1001

https://gerrit.wikimedia.org/r/464110

Change 464112 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] Replace bohrium with matomo1001 in cache text configuration

https://gerrit.wikimedia.org/r/464112

Change 464113 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] Clean up bohrium's references in cache text

https://gerrit.wikimedia.org/r/464113

Mentioned in SAL (#wikimedia-operations) [2018-10-04T07:31:17Z] <elukey> move Piwik/Matomo from bohrium to matomo1001 - T202962

Change 464110 merged by Elukey:
[operations/puppet@production] role::cache::text: add a backend for matomo1001

https://gerrit.wikimedia.org/r/464110

Change 464112 merged by Elukey:
[operations/puppet@production] Replace bohrium with matomo1001 in cache text configuration

https://gerrit.wikimedia.org/r/464112

Change 464113 merged by Elukey:
[operations/puppet@production] Clean up bohrium's references in cache text

https://gerrit.wikimedia.org/r/464113

Change 464799 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/dns@master] Add matomo1001's IPv6 PTR

https://gerrit.wikimedia.org/r/464799

Change 464799 merged by Elukey:
[operations/dns@master] Add matomo1001's IPv6 PTR

https://gerrit.wikimedia.org/r/464799

elukey moved this task from In Progress to Done on the Analytics-Kanban board.
elukey set the point value for this task to 8.