Users have been reporting strange issues with OCG (the PDF download service) in #wikimedia-tech, and the graphs don't look normal:
https://ganglia.wikimedia.org/latest/graph.php?r=week&z=xlarge&c=PDF+servers+eqiad&m=cpu_report&s=descending&mc=2&g=cpu_report
Description
Description
Related Objects
Related Objects
Event Timeline
Comment Actions
{
"_index": "logstash-2015.07.03",
"_type": "mw-ocg-service",
"_id": "j_8xKLSSQYuVvrfvHiKqFA",
"_score": null,
"_source": {
"host": "ocg1002",
"level": 3,
"file": "/srv/deployment/ocg/ocg/mw-ocg-service/lib/threads/backend.js",
"line": "560",
"version": "1.0",
"@version": "1",
"@timestamp": "2015-07-03T15:18:07.838Z",
"source_host": "10.64.48.42",
"pid": 14689,
"channel": "backend.bundler.error",
"job.id": "e0a61224d61a300ff111c1dee3f3dde37fa0222d",
"job.writer": "rdf2latex",
"err.message": "Bundling process died with non zero code: 1",
"err.name": "BundlerError",
"err.stack": "BundlerError: Bundling process died with non zero code: 1\n at fork.catch.then.console.info.channel (/srv/deployment/ocg/ocg/mw-ocg-service/lib/threads/backend.js:560:11)\n at run (/srv/deployment/ocg/ocg/node_modules/core-js/modules/es6.promise.js:89:39)\n at /srv/deployment/ocg/ocg/node_modules/core-js/modules/es6.promise.js:100:28\n at process._tickCallback (node.js:415:13)",
"type": "mw-ocg-service",
"tags": [
"es",
"gelf",
"normalized_message_untrimmed"
],
"message": "Bundling process died with non zero code: 1",
"normalized_message": "Bundling process died with non zero code: 1"
},
"sort": [
1435936687838
]
}Comment Actions
Caused by https://gerrit.wikimedia.org/r/#/c/220036/ - fixed by https://gerrit.wikimedia.org/r/#/c/222615/
I just deployed it and it looks okay now.
This comment was removed by TheDJ.
Comment Actions
Yes, it never worked on private wikis, since we don't do the cookie-forwarding thing needed to make that work. The old pre-OCG collection extension didn't handle this either, I don't think.