Page MenuHomePhabricator

Issues with OCG service in production
Closed, ResolvedPublic

Description

Users have been reporting strange issues with OCG (the PDF download service) in #wikimedia-tech, and the graphs don't look normal:
https://ganglia.wikimedia.org/latest/graph.php?r=week&z=xlarge&c=PDF+servers+eqiad&m=cpu_report&s=descending&mc=2&g=cpu_report

Event Timeline

Krenair raised the priority of this task from to Needs Triage.
Krenair updated the task description. (Show Details)
Krenair added subscribers: Krenair, Nemo_bis, cscott.
{
  "_index": "logstash-2015.07.03",
  "_type": "mw-ocg-service",
  "_id": "j_8xKLSSQYuVvrfvHiKqFA",
  "_score": null,
  "_source": {
    "host": "ocg1002",
    "level": 3,
    "file": "/srv/deployment/ocg/ocg/mw-ocg-service/lib/threads/backend.js",
    "line": "560",
    "version": "1.0",
    "@version": "1",
    "@timestamp": "2015-07-03T15:18:07.838Z",
    "source_host": "10.64.48.42",
    "pid": 14689,
    "channel": "backend.bundler.error",
    "job.id": "e0a61224d61a300ff111c1dee3f3dde37fa0222d",
    "job.writer": "rdf2latex",
    "err.message": "Bundling process died with non zero code: 1",
    "err.name": "BundlerError",
    "err.stack": "BundlerError: Bundling process died with non zero code: 1\n    at fork.catch.then.console.info.channel (/srv/deployment/ocg/ocg/mw-ocg-service/lib/threads/backend.js:560:11)\n    at run (/srv/deployment/ocg/ocg/node_modules/core-js/modules/es6.promise.js:89:39)\n    at /srv/deployment/ocg/ocg/node_modules/core-js/modules/es6.promise.js:100:28\n    at process._tickCallback (node.js:415:13)",
    "type": "mw-ocg-service",
    "tags": [
      "es",
      "gelf",
      "normalized_message_untrimmed"
    ],
    "message": "Bundling process died with non zero code: 1",
    "normalized_message": "Bundling process died with non zero code: 1"
  },
  "sort": [
    1435936687838
  ]
}
Nemo_bis renamed this task from Issues with OCG service in production? to Issues with OCG service in production.Jul 3 2015, 6:15 PM
Nemo_bis triaged this task as Unbreak Now! priority.
Nemo_bis set Security to None.
This comment was removed by TheDJ.

I just got this on officewiki though?

I just got this on officewiki though?

Based on T73849 and T101052 I'm not sure it ever worked at private wikis.

Yes, it never worked on private wikis, since we don't do the cookie-forwarding thing needed to make that work. The old pre-OCG collection extension didn't handle this either, I don't think.