Page MenuHomePhabricator

webservicemonitor: AttributeError: 'module' object has no attribute 'utcnow'
Closed, ResolvedPublic

Description

I would have sworn @scfc created a bug for this already, but I can't find it :/

2015-10-11 19:12:29,370 Starting webservice for tool kmlexport failed
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/tools/manifest/webservicemonitor.py", line 67, in run
    if self._start_webservice(manifest):
  File "/usr/lib/python3/dist-packages/tools/manifest/webservicemonitor.py", line 19, in _start_webservice
    manifest.record_starting()
  File "/usr/lib/python3/dist-packages/tools/manifest/manifest.py", line 31, in record_starting
    self.start_times.append(datetime.utcnow())
AttributeError: 'module' object has no attribute 'utcnow'
2015-10-11 19:12:29,459 Service monitor run completed, 0 webservices restarted

this is causing several tools to not come back up:

2015-10-11 19:09:56,836 Starting webservice for tool nlwikibots failed
2015-10-11 19:09:56,855 Starting webservice for tool wikiviz failed
2015-10-11 19:09:56,891 Starting webservice for tool quick-intersection failed
2015-10-11 19:09:56,959 Starting webservice for tool dplbot failed
2015-10-11 19:09:57,027 Starting webservice for tool veblenbot failed
2015-10-11 19:09:57,090 Starting webservice for tool random-featured failed
2015-10-11 19:09:57,119 Starting webservice for tool russbot failed
2015-10-11 19:09:57,289 Starting webservice for tool wikiwatchdog failed
2015-10-11 19:09:57,301 Starting webservice for tool suggestbot failed
2015-10-11 19:09:57,319 Starting webservice for tool dewikinews-rss failed
2015-10-11 19:09:57,328 Starting webservice for tool enwp10 failed
2015-10-11 19:09:57,379 Starting webservice for tool kmlexport failed

and was the cause behind T115221 (2015-10-11 18:44:21,627 Starting webservice for tool wsexport failed).

Event Timeline

valhallasw raised the priority of this task from to Needs Triage.
valhallasw updated the task description. (Show Details)
valhallasw added a project: Toolforge.
valhallasw added subscribers: valhallasw, scfc.
Restricted Application added a project: Cloud-Services. · View Herald TranscriptOct 11 2015, 7:14 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

I have now manually restarted these webservices with webservice restart.

yuvipanda triaged this task as Unbreak Now! priority.Oct 11 2015, 9:32 PM
yuvipanda added subscribers: coren, yuvipanda.

This breaks webservicemonitor. /cc @coren

Change 245192 had a related patch set uploaded (by Yuvipanda):
Call utcnow on datetime class, not module

https://gerrit.wikimedia.org/r/245192

This patch should be good enough, need to merge, rebuild package and deploy it.

Change 245192 merged by jenkins-bot:
Call utcnow on datetime class, not module

https://gerrit.wikimedia.org/r/245192

I merged, rebuilt package and hand copied it to tools-services-02 and restarted it. Things seem to work now.

Tested by killing the wikibugs webservice manually, it got restarted.

Still left to do is to add the built package to aptly and verify that it installs from there properly. You can find the newly built package as ~/tools-manifest_0.5_all.deb on my homedir.

Err - As I said on IRC I was on it already; with a ~/tools-manifest_0.6_all.deb in my homedir.

coren lowered the priority of this task from Unbreak Now! to Medium.Oct 13 2015, 1:26 PM

This should be hotfixed; keeping the ticket open for tracking cleanup.

Is there any cleanup still to be done?

valhallasw closed this task as Resolved.May 27 2016, 12:20 PM
valhallasw assigned this task to coren.