allow zotero container nodejs server to define the amount of heap used instead of the fixed limit of 1.7Gi
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• fsero
	Jan 10 2019, 12:26 PM

Description

we had a briefly outage of one minute when zotero was unresponsive, our current believe is that nodejs heap size was filled up and hence the server wasn't responding

https://grafana.wikimedia.org/d/000000620/xxxx-zotero-debugging-kubernetes?orgId=1&from=1547060027804&to=1547062295521

In order to change that we need to change the docker image to allow an adjustable heap size and modify also the helm chart to allow to define the max heap size which should be Pod Mem Limit size - X, where X is the amount of RAM the nodejs app is using besides heap.

Details

	Subject	Repo	Branch	Lines +/-
	Added defaults of node heap size to match the new one introduced.	operations/deployment-charts	master	+3 -2
	Added NODE_V8_HEAP_SIZE_IN_MB env variable and made node honor it.	mediawiki/services/zotero	master	+3 -2

Customize query in gerrit

Related Objects

Mentioned In: rDEPLOYCHARTSebd47d34549c: Added defaults of node heap size to match the new one introduced.
T213693: Zotero service crashes and pages multiple times.
rDEPLOYCHARTS1687d6d0c09a: Added defaults of node heap size to match the new one introduced.

Event Timeline

• fsero created this task.Jan 10 2019, 12:26 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 10 2019, 12:26 PM

jijiki added a project: SRE.Jan 10 2019, 12:28 PM

jijiki subscribed.

Change 483395 had a related patch set uploaded (by Fsero; owner: Fsero):
[mediawiki/services/zotero@master] Added NODE_V8_HEAP_SIZE_IN_MB env variable and made node honor it.

https://gerrit.wikimedia.org/r/483395

gerritbot added a project: Patch-For-Review.Jan 10 2019, 12:30 PM

Change 483398 had a related patch set uploaded (by Fsero; owner: Fsero):
[operations/deployment-charts@master] Added defaults of node heap size to match the new one introduced.

https://gerrit.wikimedia.org/r/483398

Diffusion mentioned this in rDEPLOYCHARTS1687d6d0c09a: Added defaults of node heap size to match the new one introduced..Jan 10 2019, 12:46 PM

Change 483395 merged by jenkins-bot:
[mediawiki/services/zotero@master] Added NODE_V8_HEAP_SIZE_IN_MB env variable and made node honor it.

https://gerrit.wikimedia.org/r/483395

greg moved this task from Active investigation to Follow-up prevention on the Wikimedia-Incident board.Jan 11 2019, 11:28 PM

• fsero moved this task from Incoming 🐫 to Stalled 🐌 on the serviceops board.Jan 14 2019, 10:08 AM

• fsero moved this task from Stalled 🐌 to Incoming 🐫 on the serviceops board.

• fsero moved this task from Incoming 🐫 to Doing 😎 on the serviceops board.

• fsero mentioned this in T213693: Zotero service crashes and pages multiple times..Jan 14 2019, 12:11 PM

An image that allows overriding the node heap has now been deployed.

• Phabricator_maintenance moved this task from Backlog to Acknowledged on the SRE board.Jan 26 2019, 11:02 PM

• fsero added a project: User-fsero.Feb 7 2019, 6:14 AM

Mvolz moved this task from Backlog to Production on the Citoid board.Feb 26 2019, 9:43 AM

jijiki mentioned this in rDEPLOYCHARTSebd47d34549c: Added defaults of node heap size to match the new one introduced..Mar 18 2019, 4:34 PM

• fsero closed this task as Resolved.Jun 20 2019, 2:05 PM

Change 483398 abandoned by Fsero:
Added defaults of node heap size to match the new one introduced.

https://gerrit.wikimedia.org/r/483398

Maintenance_bot removed a project: Patch-For-Review.Jul 16 2019, 10:10 AM

Krinkle edited projects, added Sustainability (Incident Followup); removed Wikimedia-Incident.Apr 28 2020, 9:50 PM

allow zotero container nodejs server to define the amount of heap used instead of the fixed limit of 1.7GiClosed, ResolvedPublicActions

Description

Details

Related Objects

Event Timeline

allow zotero container nodejs server to define the amount of heap used instead of the fixed limit of 1.7Gi
Closed, ResolvedPublic
Actions