Deploy ElectronPdfService Extension to production
Open, NormalPublic

Description

The ElectronPdfService extension provides access to the Electron Service for browser based PDF rendering.

https://www.mediawiki.org/wiki/Extension:ElectronPdfService
Electron-PDFs

Current Schedule:

  • to beta cluster T150945 DONE
  • to testwikis and mediawikiwiki T150944 DONE
  • to metawiki T150943 - The week of the 30th of Jan
  • to dewiki T150942 - The week of the 30th of Jan
Addshore created this task.Nov 7 2016, 4:13 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 7 2016, 4:13 PM
Addshore updated the task description. (Show Details)Nov 7 2016, 4:17 PM
Addshore updated the task description. (Show Details)Nov 7 2016, 4:22 PM
Addshore moved this task from Incoming to Tables in pdfs on the TCB-Team board.Nov 17 2016, 9:30 AM

Subtasks for each step created (and linked to this ticket)

Change 322083 had a related patch set uploaded (by Addshore):
Add ElectronPdfService to make-wmf-branch

https://gerrit.wikimedia.org/r/322083

Change 322083 merged by jenkins-bot:
Add ElectronPdfService to make-wmf-branch

https://gerrit.wikimedia.org/r/322083

Addshore updated the task description. (Show Details)Nov 18 2016, 11:48 AM
Joe added a subscriber: Joe.Nov 29 2016, 11:07 AM

I have just noticed that this extension goes through restbase to request the PDFs; this doesn't make sense at all as restbase is doing no caching on these requests.

This means we're just adding a level of indirection for the sake of it. This should NOT be deployed to production in the current form.

Joe added a comment.Nov 29 2016, 11:10 AM

Also, I find pretty baffling that htere is no "operations" tag attached to this ticket.

Joe changed the task status from Open to Stalled.Nov 29 2016, 11:10 AM
Joe added a project: Operations.

Changed the status to "stalled" until this is fixed and mediawiki calls the service directly.

Joe changed the status of subtask T150942: Deploy ElectronPdfService Extension to dewiki from Open to Stalled.Nov 29 2016, 11:18 AM

I have just noticed that this extension goes through restbase to request the PDFs; this doesn't make sense at all as restbase is doing no caching on these requests.

This means we're just adding a level of indirection for the sake of it. This should NOT be deployed to production in the current form.

Ahh, after all of the discussions we thought this was the direction the WMF was expecting of the extension, thus we made and marged https://gerrit.wikimedia.org/r/#/c/322121/.
Reverting this will result in mediawiki calling the service directly and saving the file locally and serving from there.

Joe added a comment.Nov 29 2016, 11:46 AM

Ahh, after all of the discussions we thought this was the direction the WMF was expecting of the extension, thus we made and marged https://gerrit.wikimedia.org/r/#/c/322121/.
Reverting this will result in mediawiki calling the service directly and saving the file locally and serving from there.

Yeah I saw some of the discussion, and at least to me going through restbase seems an unnecessary hop in the call stack; I also couldn't find any compelling argument for going through restbase, but the discussion is quite spread around multiple tickets, so I might just have missed the point.

Happy to know it's just matter of reverting a change for us to be able to switch to calling directly, though.

Thanks for the patience.

Joe added a comment.Nov 29 2016, 12:10 PM

Looking at the code, there has been a misunderstanding:

MediaWiki is just issuing a redirect to the rest api, not proxying the request through it.

Which is still doing an unnecessary hop (IMHO) and caches twice as many objects in varnish, but is not flawed as a call pattern within the cluster.

I'm still not convinced this is a good idea, but it's not what I first understood. We can deploy this version if we think it's a good idea to have only one entrypoint to the PDF service.

Joe added a comment.Nov 29 2016, 12:22 PM

So, as discussed with @Addshore on irc:

  1. we could get rid of the unnecessary redirect by just linking to the restbase url directly from the pages when we have to request the PDF
  2. The only downside is that one can't deploy this extension without restbase, which is adding some complexity to the requirements; for the WMF this is a non-issue anyways so this is not a blocker.

The only thing that should still be checked is, given all the traffic will go through restbase, is the caching strategy at the varnish level (see T143132).

Joe changed the task status from Stalled to Open.Nov 29 2016, 1:39 PM
fgiunchedi triaged this task as Normal priority.Nov 30 2016, 2:15 AM
Addshore updated the task description. (Show Details)Jan 18 2017, 10:57 PM

We are having problems with the electron service in production (cf. T159922: pdfrender fails to serve requests since Mar 8 00:30:32 UTC on scb1003) and we need to do something about it. At the same time, there is an on-going evaluation of various services that might fully replace OCG (and would therefore offer a superset of features needed here) - T166188: Architecture of new rendering backend for Extension:Collection. The question here is what to do in the meantime. Should we:

  1. stop the service and therefore pause this effort until a final decision has been made; or
  2. move the electron service out of production to labs and offer PDFs on a best-effort basis?

IMHO, we should go with option (1) for the time being, since it is quite clear at this point that something will replace OCG. Option (2) requires effort and resources and the uncertainty and imminence of the aforementioned decision do not seem to justify it.

I think it depends on whether we want to have something running while T166188 is under way. If we don't, then it's 1. In both cases btw we should also undeploy the service in order to avoid human mistakes restarting the service, unnecessary confusion and so on.