Page MenuHomePhabricator

Toolviews data loading from Toolforge front proxy access log stopped on 2019-10-28
Closed, ResolvedPublic

Description

The post-rotate script implemented in rOPUPfece35d5ddb4: toolforge: process dynamicproxy access logs has not added new rows to s53734__toolviews_p.daily_raw_views since 2019-10-28. Likely an unintended side effect of T235627: Toolforge: upgrade main proxy servers to Debian Buster

Event Timeline

$ sudo journalctl /usr/sbin/logrotate --no-pager
-- Logs begin at Thu 2019-10-31 21:57:55 UTC, end at Fri 2019-11-01 02:17:24 UTC. --
Nov 01 00:00:55 tools-proxy-05 logrotate[13051]: Traceback (most recent call last):
Nov 01 00:00:55 tools-proxy-05 logrotate[13051]:   File "/usr/local/bin/toolviews.py", line 242, in <module>
Nov 01 00:00:55 tools-proxy-05 logrotate[13051]:     main()
Nov 01 00:00:55 tools-proxy-05 logrotate[13051]:   File "/usr/local/bin/toolviews.py", line 237, in main
Nov 01 00:00:55 tools-proxy-05 logrotate[13051]:     stats = ToolViews(config, args.dry_run)
Nov 01 00:00:55 tools-proxy-05 logrotate[13051]:   File "/usr/local/bin/toolviews.py", line 63, in __init__
Nov 01 00:00:55 tools-proxy-05 logrotate[13051]:     self.tools = self.get_tools()
Nov 01 00:00:55 tools-proxy-05 logrotate[13051]:   File "/usr/local/bin/toolviews.py", line 71, in get_tools
Nov 01 00:00:55 tools-proxy-05 logrotate[13051]:     ], ldap3.POOLING_STRATEGY_ROUND_ROBIN, active=True, exhaust=True),
Nov 01 00:00:55 tools-proxy-05 logrotate[13051]: AttributeError: module 'ldap3' has no attribute 'POOLING_STRATEGY_ROUND_ROBIN'
Nov 01 00:00:55 tools-proxy-05 logrotate[13051]: run-parts: /etc/logrotate.d/nginx-postrotate/toolviews exited with return code 1
Nov 01 00:00:55 tools-proxy-05 logrotate[13051]: error: error running shared postrotate script for '/var/log/nginx/*.log '
$ python3
Python 3.7.3 (default, Apr  3 2019, 05:39:12)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ldap3
>>> ldap3.__version__
'2.4.1'

Our first confirmation of T214541: python3-ldap3 mixed versions and future traps!

Change 547700 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[operations/puppet@production] toolforge: Update toolviews.py for ldap3 v2.4.1

https://gerrit.wikimedia.org/r/547700

bd808 triaged this task as High priority.
bd808 moved this task from Inbox to Doing on the cloud-services-team (Kanban) board.

Change 547700 merged by Phamhi:
[operations/puppet@production] toolforge: Update toolviews.py for ldap3 v2.4.1

https://gerrit.wikimedia.org/r/547700