An error in Fireflytools
Closed, DuplicatePublic
Actions

Assigned To

None

Authored By

	Adithyak1997
	Apr 19 2019, 2:08 PM

Description

Traceback (most recent call last):
  File "/usr/bin/job", line 47, in <module>
    proc = subprocess.Popen(['/usr/bin/qstat', '-xml'], stdout=subprocess.PIPE)
  File "/usr/lib/python3.5/subprocess.py", line 676, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.5/subprocess.py", line 1221, in _execute_child
    restore_signals, start_new_session, preexec_fn)
OSError: [Errno 12] Cannot allocate memory

I have got an email recently with this message. I would like to know how this issue can be solved. I guess the memory might have become full.

Related Objects

Mentioned Here: T222255: tools-sgecron-01 virtual memory allocation error at midnight and noon UTC
T183436: Add memory limit configuration for Kubernetes pods

Event Timeline

Adithyak1997 created this task.Apr 19 2019, 2:08 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 19 2019, 2:08 PM

I would like to know how this issue can be solved.

Is this a failure of a webservice process or a job? Was it running on the grid engine job grid or in a Kubernetes container?

Broadly you need to either adjust your application so that it uses less total memory at runtime or adjust the runtime that the application is running under so that more memory is available to the process. For grid engine job submitted with jsub/jstart/qsub see https://wikitech.wikimedia.org/wiki/Help:Toolforge/Grid#Allocating_additional_memory for information on allocating more memory. Similar adjustments can be made for webservices running on the grid engine backend. For Kubernetes webservices we have a feature request open (T183436: Add memory limit configuration for Kubernetes pods), but no implementation at this time.

I have a small doubt. Is there any method by which .sh files which contain dump values can be cleared through Putty?

This bug seems similar to T222255. I don't think it has anything to do with -mem

@bd808

In T221451#5127795, @bd808 wrote:

you need to either adjust your application so that it uses less total memory at runtime or adjust the runtime that the application is running under so that more memory is available to the process.

Please check the page https://tools.wmflabs.org/sge-status/. In that, when I searched for the jobs run by my tool, it showed large memory space {I guess}. So I think it has something to do with Kubernetes.

@Adithyak1997 Who is the sender of the email you are referring to by 'I have got an email recently with this message.'?

It's from Cron Daemon with the sender <root@tools.wmflabs.org>. I usually get error messages when something went wrong with the tool.

Then it has nothing to do with k8s. Cron doesn't do k8s.

An error in FireflytoolsClosed, DuplicatePublicActions

Description

Related Objects

Event Timeline

An error in Fireflytools
Closed, DuplicatePublic
Actions