Page MenuHomePhabricator

"Preview with PediaPress" defunct
Closed, ResolvedPublic

Description

A click on "Preview with PediaPress" on https://en.wikipedia.org/wiki/Special:Book always leads to the following error message:

Book rendering failed
There was an error while attempting to render your book.
Return to Main Page.

Bug also confirmed on de.wikipedia.org and es.wikipedia.org.


Version: unspecified
Severity: normal

Details

Reference
bz71675

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 3:46 AM
bzimport set Reference to bz71675.
Ckepper created this task.Oct 5 2014, 6:32 PM
saper added a comment.Oct 6 2014, 10:32 PM

Confirmed on plwiki as well.

But MediaWiki at fd9173e with extension Collection also at its git master installed locally do work (4b4bc6c05883ff2d8061d88bd5e7b861725a16ba); one gets redirected to the PediaPress website.

cscott added a comment.Oct 7 2014, 3:51 PM

It appears that the 'zip_post' command is going to the OCG servers (who can't handle it) instead of the public pediapress servers.

(In reply to Marcin Cieślak from comment #1)

Confirmed on plwiki as well.
But MediaWiki at fd9173e with extension Collection also at its git master
installed locally do work (4b4bc6c05883ff2d8061d88bd5e7b861725a16ba); one
gets redirected to the PediaPress website.

If OCG is not configured at all, you mean? It's conceivable that some (few?) REL1_24 users will set up OCG, so I'd rather have the fix in REL1_24 as well.

Change 165234 had a related patch set uploaded by Cscott:
Allow overriding the default server for POD functions.

https://gerrit.wikimedia.org/r/165234

Change 165239 had a related patch set uploaded by Cscott:
Disable PediaPress POD function.

https://gerrit.wikimedia.org/r/165239

cscott added a comment.Oct 7 2014, 4:34 PM

The PediaPress link should be re-added to [[:en:Template:Saved_book]] once this function works again.

Change 165234 merged by jenkins-bot:
Allow overriding the default server for POD functions.

https://gerrit.wikimedia.org/r/165234

Change 165239 merged by jenkins-bot:
Disable PediaPress POD function.

https://gerrit.wikimedia.org/r/165239

Well - turning the book export off completely prevents the error message from popping up, but it also does not make anything better. Our customers are complaining that they cannot export their books.
Was there a problem with using tools.pediapress.com?

cscott added a comment.Oct 8 2014, 7:05 PM

Yes, tools.pediapress.com is giving "Server returned error: system overloaded. please try again later." errors when I test it locally.

But we need to land/deploy https://gerrit.wikimedia.org/r/165234 before we can redirect post_zip requests to tools.pediapress.com. Hopefully once that lands you'll have tools.pediapress.com ready to handle the load and we can turn the POD function back on.

Note that gerrit 165234 pushes the post_zip requests over to tools.pediapress.com. The OCG bundle format is largely compatible, but there's no code written to take OCG zip bundles and post them to pediapress. So pediapress needs to be rolling the zip bundles itself.

cscott added a comment.Oct 8 2014, 7:06 PM

Oh, and Christoph: feel free to jump into #wikimedia-tech on freenode today (2014-10-08) or #mediawiki-parsoid any time to discuss the technical details; my IRC nick is 'cscott'.

That changeset landed on WP last week, yes? If so, how are things on the PP side of things - can we re-enable soon? We're still expecting to phase this out eventually, but for now it'd be nice to have it back :-)

Hi Erik, I talked to Nemo and CScott last week during the bug triage. As you can see above the PediaPress public toolserver could not handle the load from Wikipedia post requests and so CScott decided to turn it off..

In the meantime we set up another endpoint specifically for Wikimedia. CScott wanted to test the new endpoint and re-enable PediaPress afterwards. As we are getting quite a few complaints from would-be customers right now, we would also like to have it back ;-)

Yup. I was on vacation last week, but I'm back now. I'll try to test this new endpoint out today.

Change 167806 had a related patch set uploaded by Cscott:
Update PediaPress endpoint; re-enable PediaPress in labs for testing.

https://gerrit.wikimedia.org/r/167806

Change 167806 merged by jenkins-bot:
Update PediaPress endpoint; re-enable PediaPress in labs for testing.

https://gerrit.wikimedia.org/r/167806

This was deployed early, it's live on beta now.

Initial testing looks good. Christoph, could you give it a try on http://en.wikipedia.beta.wmflabs.org/ before I turn it on in production?

Change 167866 had a related patch set uploaded by Cscott:
Re-enable PediaPress POD in production.

https://gerrit.wikimedia.org/r/167866

Thank you very much Scott. I checked the upload and the preview. Looks good to me :-)

Ok. I've scheduled the patch to turn this on in production for the same Wednesday morning window.

Change 167866 merged by jenkins-bot:
Re-enable PediaPress POD in production.

https://gerrit.wikimedia.org/r/167866

So we've deployed the patch and turned it on, but I'm getting a "Book rendering failed: There was an error while attempting to render your book." when I try to use PediaPress on enwiki. Rendering to PediaPress on beta still seems to work fine.

PediaPress folks: can you take a look at the logs on your side and see if you're getting what you expect?

OK - I will try to figure out what is going on.

Change 168091 had a related patch set uploaded by Cscott:
Send Extension:Collection logs to logstash.

https://gerrit.wikimedia.org/r/168091

Change 168091 merged by jenkins-bot:
Send Extension:Collection logs to logstash.

https://gerrit.wikimedia.org/r/168091

Looks like outgoing connections are firewalled on the WMF side (well, not firewalled exactly, more like "not routed from our VLAN"). It works from beta because the labs environment is open.

We can use WMF's url-downloader proxy to work around this, but I need to right a patch to allow Extension:Collection to use a proxy when making its requests.

marktraceur: we can either SWAT the revert patch ( https://gerrit.wikimedia.org/r/168082 ) and then wait for the patch to Collection to ride the usual deploy train before trying again, or else we can SWAT the new patch for Collection and a tweak to the mediawiki config later today.

Change 168114 had a related patch set uploaded by Cscott:
Use url-downloader proxy to reach PediaPress POD service.

https://gerrit.wikimedia.org/r/168114

Change 168082 had a related patch set uploaded by Cscott:
Revert "Re-enable PediaPress POD in production."

https://gerrit.wikimedia.org/r/168082

Change 168114 merged by jenkins-bot:
Use url-downloader proxy to reach PediaPress POD service.

https://gerrit.wikimedia.org/r/168114

Ok, new patch using url-downloader to proxy out to pediapress is deployed.

Please test. Seems like it works to me!

Change 168082 abandoned by Cscott:
Revert "Re-enable PediaPress POD in production."

Reason:
Abandoned in favor of https://gerrit.wikimedia.org/r/168114

https://gerrit.wikimedia.org/r/168082

<3 Thanks for fixing this so quickly! :)