Page MenuHomePhabricator

Still issues with node.js webservice
Closed, DeclinedPublic

Description

T160241 has been fixed (thanks!), but I have still trouble with the tool. It runs unusually slow, and restarting the webservice works, but throws a scary error message:

webservice restart
Restarting webservice...............Traceback (most recent call last):
  File "/usr/local/bin/webservice", line 152, in <module>
    start(job, '')
  File "/usr/local/bin/webservice", line 34, in start
    wait_for(lambda: job.get_state() == Backend.STATE_RUNNING, message)
  File "/usr/lib/python2.7/dist-packages/toollabs/common/utils.py", line 20, in wait_for
    if predicate():
  File "/usr/local/bin/webservice", line 34, in <lambda>
    wait_for(lambda: job.get_state() == Backend.STATE_RUNNING, message)
  File "/usr/lib/python2.7/dist-packages/toollabs/webservice/backends/kubernetesbackend.py", line 372, in get_state
    pod = self._find_obj(pykube.Pod, self.webservice_label_selector)
  File "/usr/lib/python2.7/dist-packages/toollabs/webservice/backends/kubernetesbackend.py", line 191, in _find_obj
    selector=selector
  File "/usr/lib/python2.7/dist-packages/pykube/query.py", line 75, in get
    raise ValueError("get() more than one object; use filter")
ValueError: get() more than one object; use filter

Also, previously the kubernetes config file would remember that it's node.js, but apparently it doesn't, so I now need to say

webservice --backend kubernetes nodejs start

instead of

webservice start

every time.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

webservice restart (as opposed to start) should use the contents of the service.manifest file. Not ideal, but this is due to the original design where webservice start would do the same thing for all users.

I'm not sure about the slowness (and am also not sure how to debug that), so someone else will have to chime in there :-)

bd808 subscribed.
  • The python stack trace problem is tracked in T156626: k8s webservice restart failure with `ValueError: get() more than one object; use filter`
  • Related to @valhallasw's comment about using webservice restart, webservice stop clears the contents of the "service.manifest" file which is where the settings for backend and type are stored. Changing the default backend to kubernetes is tracked in T154504: Make webservice backend default to kubernetes. That was blocked on some needed Kubernetes configuration changes that have been recently resolved, so we might see progress on that in the coming weeks/months.
  • "It runs unusually slow" is not an easily actionable report. Have you done any profiling or other investigation to attempt to determine what specifically is slower about the Kubernetes hosted service compared to the grid engine hosted service?

part duplicate and part unactionable