Outbound proxy doesn't seem to be working in citoid production
Closed, ResolvedPublic1 Estimated Story Points
Actions

Assigned To

Authored By

	Jdforrester-WMF
	Mar 19 2015, 12:16 AM

Description

Getting errors when making any outbound request with citoid.

This is encountered any time we try to follow a redirect or use the native scraper; basically any URL that doesn't go straight to Zotero from the getgo.

Details

	Subject	Repo	Branch	Lines +/-
	Citoid: Set the protocol for the proxy	operations/puppet	production	+1 -1

Customize query in gerrit

Related Objects

Mentioned In: rOPUP4f970a92256d: Citoid: Set the protocol for the proxy

Event Timeline

Jdforrester-WMF created this task.Mar 19 2015, 12:16 AM

Jdforrester-WMF assigned this task to Mvolz.

Jdforrester-WMF raised the priority of this task from to High.

Jdforrester-WMF updated the task description. (Show Details)

Jdforrester-WMF added a project: Citoid.

Jdforrester-WMF added subscribers: Jdforrester-WMF, Mooeypoo.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 19 2015, 12:16 AM

Mvolz added a project: Services.Mar 19 2015, 7:07 AM

Mvolz set Security to None.

Mvolz added a subscriber: • mobrovac.

Looks like we might have been IP banned by springer. Will investigate more.

Nope, nevermind. :). That's not it. It's an actual bug, which has since been fixed. @mobrovac, I'll run through an update to /deploy.

Mvolz moved this task from Site specific issues to Production on the Citoid board.Mar 19 2015, 10:37 AM

Nevermind again!

I remember a bug exactly like that, where request from doi would fail, and somehow we ended up with scrape failing too because of it, but I'll damned if I remember exactly why that happened.

But what's running on deploy right now should have been okay.

Got into the logs, here what they have to say:

{"name":"citoid","hostname":"sca1001","pid":35542,"level":30,"msg":"Using native scraper on http://dx.doi.org/10.1007/11926078_68","time":"2015-03-19T11:33:25.229Z","v":0}
{"name":"citoid","hostname":"sca1001","pid":35542,"level":50,"err":{"message":"Invalid protocol: url-downloader.wikimedia.org:","name":"Error","stack":"Error: Invalid protocol: url-downloader.wikimedia.org:\n at Request.init (/srv/deployment/citoid/deploy/node_modules/request/request.js:619:31)\n at new Request (/srv/deployment/citoid/deploy/node_modules/request/request.js:283:8)\n at request (/srv/deployment/citoid/deploy/node_modules/request/index.js:50:10)\n at Scraper.scrape (/srv/deployment/citoid/deploy/src/lib/Scraper.js:40:2)\n at CitoidService.scrape (/srv/deployment/citoid/deploy/src/lib/CitoidService.js:204:3)\n at /srv/deployment/citoid/deploy/src/lib/CitoidService.js:94:12\n at Request._callback (/srv/deployment/citoid/deploy/src/lib/unshorten.js:16:5)\n at self.callback (/srv/deployment/citoid/deploy/node_modules/request/request.js:373:22)\n at Request.EventEmitter.emit (events.js:95:17)\n at Request.init (/srv/deployment/citoid/deploy/node_modules/request/request.js:619:17)"},"msg":"Invalid protocol: url-downloader.wikimedia.org:","time":"2015-03-19T11:33:25.231Z","v":0}
{"name":"citoid","hostname":"sca1001","pid":35542,"level":30,"msg":"Unable to scrape resource at http://dx.doi.org/10.1007/11926078_68","time":"2015-03-19T11:33:25.231Z","v":0}

Mvolz renamed this task from The example URL at citoid.wikimedia.org 520s to Outbound proxy doesn't seem to be working in citoid production,.Mar 19 2015, 11:36 AM

Mvolz renamed this task from Outbound proxy doesn't seem to be working in citoid production, to Outbound proxy doesn't seem to be working in citoid production.

Mvolz updated the task description. (Show Details)

Mvolz removed Mvolz as the assignee of this task.Mar 19 2015, 11:39 AM

• mobrovac claimed this task.Mar 19 2015, 11:41 AM

Change 197884 had a related patch set uploaded (by Mobrovac):
Citoid: Set the protocol for the proxy

https://gerrit.wikimedia.org/r/197884

gerritbot added a project: Patch-For-Review.Mar 19 2015, 11:50 AM

Change 197884 merged by Alexandros Kosiaris:
Citoid: Set the protocol for the proxy

https://gerrit.wikimedia.org/r/197884