Page MenuHomePhabricator

Piwik puppet configuration refactoring and updates
Closed, ResolvedPublic8 Estimated Story Points

Description

In T154558 several changes to Piwik were performed manually to mitigate various problems:

  • SET GLOBAL max_connections 300 to the mysql instance (apache error logs for no more connections to use)
  • disabled apache mod_deflate (probably this one wasn't needed)
  • bump to 256M memory_limit in /etc/piwik/global.ini.php and /etc/php5/apache2/php.ini
  • set bulk_requests_use_transaction=0 to /etc/piwik/global.ini.php (https://github.com/piwik/piwik/issues/6398#issuecomment-91093146) that was causing entries in the apache error log for locks already in use.
  • Bump the maximum Apache connections allowed (hence the number of processes due to mpm_prefork) from 150 to 250, reducing also the churn due to process kill/re-spawn with MaxSpareServers set to 150 as well (and Min to 50):
# prefork MPM
# StartServers: number of server processes to start
# MinSpareServers: minimum number of server processes which are kept spare
# MaxSpareServers: maximum number of server processes which are kept spare
# MaxRequestWorkers: maximum number of server processes allowed to start
# MaxConnectionsPerChild: maximum number of requests a server process serves

<IfModule mpm_prefork_module>
	# prev: 5
	StartServers	          250
	# prev: 5
	MinSpareServers		  50
	# prev: 10
	MaxSpareServers		  250
	# prev: 150
	MaxRequestWorkers	  250
	MaxConnectionsPerChild    0
</IfModule>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

What would be needed:

  1. Puppetization of the above fixes.
  2. Refactor of piwik's role code into modules and profile for a better maintenance over time.

Security upgrades will be handled in T158322

Database improvements will be handled in T164073

Event Timeline

Change 342007 had a related patch set uploaded (by Elukey):
[operations/puppet] Remove Piwik backend probe from Varnish Misc backends

https://gerrit.wikimedia.org/r/342007

Nuria raised the priority of this task from Medium to High.Mar 13 2017, 5:09 PM
elukey lowered the priority of this task from High to Medium.Apr 14 2017, 2:23 PM
elukey added a parent task: T163000: Piwik improvements.
elukey removed a project: Patch-For-Review.

Change 348938 had a related patch set uploaded (by Elukey):
[operations/puppet@production] Refactor role::piwik in multiple profiles

https://gerrit.wikimedia.org/r/348938

Change 349950 had a related patch set uploaded (by Elukey):
[labs/private@master] Piwik fake passwords

https://gerrit.wikimedia.org/r/349950

Change 349950 merged by Elukey:
[labs/private@master] Piwik fake passwords

https://gerrit.wikimedia.org/r/349950

Change 349954 had a related patch set uploaded (by Elukey):
[labs/private@master] Correct hiera path for piwik role

https://gerrit.wikimedia.org/r/349954

Change 349954 merged by Elukey:
[labs/private@master] Correct hiera path for piwik role

https://gerrit.wikimedia.org/r/349954

Change 348938 merged by Elukey:
[operations/puppet@production] Refactor role::piwik in multiple profiles

https://gerrit.wikimedia.org/r/348938

Change 350567 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] Add quotes to the Piwik configuration file

https://gerrit.wikimedia.org/r/350567

Change 350567 merged by Elukey:
[operations/puppet@production] Add quotes to the Piwik configuration file

https://gerrit.wikimedia.org/r/350567

Change 350573 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] Fix Piwik erb config template

https://gerrit.wikimedia.org/r/350573

Change 350573 merged by Elukey:
[operations/puppet@production] Fix Piwik erb config template

https://gerrit.wikimedia.org/r/350573

Change 350576 merged by Elukey:
[operations/puppet@production] Set trusted hosts for Piwik in its profile

https://gerrit.wikimedia.org/r/350576

elukey moved this task from In Progress to Done on the Analytics-Kanban board.
elukey set the point value for this task to 8.May 4 2017, 3:20 PM