Page MenuHomePhabricator

Update piwik to latest stable
Closed, ResolvedPublic

Description

Event Timeline

elukey renamed this task from update piwik to latest stable to Update piwik to latest stable.Apr 16 2018, 3:42 PM
elukey claimed this task.
elukey triaged this task as Medium priority.
elukey added a project: User-Elukey.
elukey updated the task description. (Show Details)
fdans lowered the priority of this task from Medium to Low.Apr 16 2018, 4:03 PM
fdans moved this task from Incoming to Operational Excellence on the Analytics board.
elukey raised the priority of this task from Low to Medium.Jun 26 2018, 12:38 PM
elukey added a project: Analytics-Kanban.
elukey removed a subscriber: Aklapper.

I tested today in labs an upgrade from the current version 2.17 to 3.2.1, the latest stable before the rebrand to matomo. Everything worked fine, but the update function will probably take 1/2 hours to make all the invasive database changes. It should be doable very soon, so we'll get a version that was released last December. The next step will be to upgrade to matomo as well.

Piwik maintenance just completed, the website is back in service with the new 3.2.1 version.

The last version of Matomo (the new Piwik name) contains a super interesting feature, namely use TLS for connections to the database. This is not important now that we co-locate Piwik with its db on bohrium, but for future usage (for example, if we'll want to move the db out of bohrium).

I tried to install it in labs, this is what I get:

elukey@piwik:~$ sudo dpkg -i matomo_3.5.1-1_all.deb
Selecting previously unselected package matomo.
(Reading database ... 60845 files and directories currently installed.)
Preparing to unpack matomo_3.5.1-1_all.deb ...
  * Piwik is now Matomo. Matomo configuration files are stored in /etc/matomo/
  * Piwik has been previously installed on this system and its configuration folder
  * remains. We're moving Piwik configuration folder from '/etc/piwik' to '/etc/piwik.orig'
  * so this package can install correctly.
  * Review the configuration changes accordingly.
Unpacking matomo (3.5.1-1) ...
Setting up matomo (3.5.1-1) ...
Enabling conf matomo.
To activate the new configuration, you need to run:
  service apache2 reload
  * Check Matomo web configuration in /etc/apache2/conf-available/matomo.conf
  * Copying Piwik configuration (/etc/piwik.orig/config.ini.php) to Matomo configuration folder (/etc/matomo/)
  * Matomo main configuration files are stored in /etc/matomo/
  * Matomo crontab file is stored in /etc/cron.d/matomo-archive
  * When installing/using matomo, make sure you disable any ads blocker
  * To finish Matomo's upgrade, please run: /usr/bin/php /usr/share/matomo/console core:update

And then, there are db upgrades to do:

Executing CREATE TABLE `piwik_privacy_logdata_anonymizations` (`idlogdata_anonymization` BIGINT UNSIGNED NOT NULL AUTO_INCREMENT, `idsites` TEXT NULL DEFAULT NULL, `date_start` DATETIME NOT NULL, `date_end` DATETIME NOT NULL, `anonymize_ip` TINYINT(1) UNSIGNED NOT NULL DEFAULT 0, `anonymize_location` TINYINT(1) UNSIGNED NOT NULL DEFAULT 0, `anonymize_userid` TINYINT(1) UNSIGNED NOT NULL DEFAULT 0, `unset_visit_columns` TEXT NOT NULL DEFAULT '', `unset_link_visit_action_columns` TEXT NOT NULL DEFAULT '', `output` MEDIUMTEXT NULL DEFAULT NULL, `scheduled_date` DATETIME NULL, `job_start_date` DATETIME NULL, `job_finish_date` DATETIME NULL, `requester` VARCHAR(100) NOT NULL DEFAULT '', PRIMARY KEY ( `idlogdata_anonymization` )) ENGINE=InnoDB DEFAULT CHARSET=utf8;... Done. [1 / 6]
Executing ALTER TABLE `piwik_privacy_logdata_anonymizations` ADD INDEX index_job_start_date (`job_start_date`);... Done. [2 / 6]
Executing ALTER TABLE `piwik_plugin_setting` ADD COLUMN `json_encoded` TINYINT UNSIGNED NOT NULL DEFAULT 0;... Done. [3 / 6]
Executing ALTER TABLE `piwik_site_setting` ADD COLUMN `json_encoded` TINYINT UNSIGNED NOT NULL DEFAULT 0;... Done. [4 / 6]
Executing ALTER TABLE `piwik_log_visit` MODIFY COLUMN `location_region` char(3) DEFAULT NULL;... Done. [5 / 6]
Executing ALTER TABLE `piwik_log_conversion` MODIFY COLUMN `location_region` char(3) DEFAULT NULL;... Done. [6 / 6]

They might take a while since piwik_log_visit is involved, but surely less than the last time (~8h).

Just to be super sure, triple checked that the matomo's tracking JS code didn't change:

<!-- Matomo -->
<script type="text/javascript">
  var _paq = _paq || [];
  /* tracker methods like "setCustomDimension" should be called before "trackPageView" */
  _paq.push(['trackPageView']);
  _paq.push(['enableLinkTracking']);
  (function() {
    var u="//localhost:1080/";
    _paq.push(['setTrackerUrl', u+'piwik.php']);
    _paq.push(['setSiteId', '1']);
    var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0];
    g.type='text/javascript'; g.async=true; g.defer=true; g.src=u+'piwik.js'; s.parentNode.insertBefore(g,s);
  })();
</script>
<!-- End Matomo Code -->

Change 442806 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] Upgrade the piwik module to matomo

https://gerrit.wikimedia.org/r/442806

Piwik is currently archiving tables, so I am going to postpone the maintenance until it finishes.

Change 442806 merged by Elukey:
[operations/puppet@production] Upgrade the piwik module to matomo

https://gerrit.wikimedia.org/r/442806

Mentioned in SAL (#wikimedia-operations) [2018-06-28T14:46:40Z] <elukey> upgrade piwik 3.2.1 to matomo (new name/package) 3.5.1 - T192298

Going back to in progress, some issues are coming up after the upgrade:

  1. Nuria had to restart Matomo (apachectl restart) due to slowness and timeouts. I can see the action in auth.log but not specific errors in the logs, will try to check latencies in apache logs as well:
Jun 28 18:03:21 bohrium sudo:    nuria : TTY=pts/0 ; PWD=/usr/share/matomo ; USER=root ; COMMAND=/etc/init.d/apache2 stop
Jun 28 18:03:26 bohrium sudo:    nuria : TTY=pts/0 ; PWD=/usr/share/matomo ; USER=root ; COMMAND=/etc/init.d/apache2 start
  1. There are periodical/recurrent errors in the apache error log:
[Fri Jun 29 16:43:45.560500 2018] [:error] [pid 30120] [client 10.64.32.103:13001] Error in Matomo (tracker): Custom timestamp is 962307 seconds old, requires &token_auth...

If I try to correlate it with an access log request, I can see a big POST that looks from iOS with a weird timestamp:

url?QUERY_STRING_OMITTED&cdt=2018-06-28%2014:49:57

UI: WikipediaApp/5.7.2.1267 (iPhone OS 9.3.5; Tablet)

A bit weird..

I bet that error comes from me using a session in piwik that predated the upgrade

It happens even now, and httpd returns application/x-httpd-php/400. The requests are look the same, GET with long query strings.. (not all the same client IPs).

Let's keep an eye on this, not sure if it is a new bug or feature :D

The only error i see on ui is: "curl_exec: Failed to connect to plugins.matomo.org port 443: Connection timed out. Hostname requested was: plugins.matomo.org"

In /var/log/apache/error.log there are still errors like:

[Tue Jul 03 05:52:59.776050 2018] [:error] [pid 5552] [client 10.64.32.103:7423] Error in Matomo (tracker): Custom timestamp is 283052 seconds old, requires &token_auth...

That corresponds to application/x-httpd-php/400 errors for UA WikipediaApp* (various versions). These agents are doing simple GET requests, so not POSTing events...