Users have been reporting strange issues with OCG (the PDF download service) in #wikimedia-tech, and the graphs don't look normal:
https://ganglia.wikimedia.org/latest/graph.php?r=week&z=xlarge&c=PDF+servers+eqiad&m=cpu_report&s=descending&mc=2&g=cpu_report
Description
Description
Related Objects
Related Objects
Event Timeline
Comment Actions
{ "_index": "logstash-2015.07.03", "_type": "mw-ocg-service", "_id": "j_8xKLSSQYuVvrfvHiKqFA", "_score": null, "_source": { "host": "ocg1002", "level": 3, "file": "/srv/deployment/ocg/ocg/mw-ocg-service/lib/threads/backend.js", "line": "560", "version": "1.0", "@version": "1", "@timestamp": "2015-07-03T15:18:07.838Z", "source_host": "10.64.48.42", "pid": 14689, "channel": "backend.bundler.error", "job.id": "e0a61224d61a300ff111c1dee3f3dde37fa0222d", "job.writer": "rdf2latex", "err.message": "Bundling process died with non zero code: 1", "err.name": "BundlerError", "err.stack": "BundlerError: Bundling process died with non zero code: 1\n at fork.catch.then.console.info.channel (/srv/deployment/ocg/ocg/mw-ocg-service/lib/threads/backend.js:560:11)\n at run (/srv/deployment/ocg/ocg/node_modules/core-js/modules/es6.promise.js:89:39)\n at /srv/deployment/ocg/ocg/node_modules/core-js/modules/es6.promise.js:100:28\n at process._tickCallback (node.js:415:13)", "type": "mw-ocg-service", "tags": [ "es", "gelf", "normalized_message_untrimmed" ], "message": "Bundling process died with non zero code: 1", "normalized_message": "Bundling process died with non zero code: 1" }, "sort": [ 1435936687838 ] }
Comment Actions
Caused by https://gerrit.wikimedia.org/r/#/c/220036/ - fixed by https://gerrit.wikimedia.org/r/#/c/222615/
I just deployed it and it looks okay now.
This comment was removed by TheDJ.
Comment Actions
Yes, it never worked on private wikis, since we don't do the cookie-forwarding thing needed to make that work. The old pre-OCG collection extension didn't handle this either, I don't think.