Page MenuHomePhabricator

Choose a server for the chromium-render service
Closed, ResolvedPublic

Description

The chromium-render service ("Proton") will eventually be replacing the electron-render / pdfrender service in production. This task encompasses the work to identify on what server it will live on. Performance estimates are captured in T178278. Additionally, the service requires a Debian Stretch installation.

Related Objects

StatusSubtypeAssignedTask
OpenNone
ResolvedNone
ResolvedBawolff
Resolvedphuedx
Resolvedmobrovac
Resolvedmobrovac
Resolvedphuedx
ResolvedJdrewniak
Resolvedphuedx
Resolvedphuedx
Resolvedphuedx
Resolvedphuedx
DeclinedNone
Resolvedbmansurov
Resolvedmobrovac
Resolvedovasileva
InvalidNone
ResolvedJdlrobson
Resolvedphuedx
Resolvedphuedx
Resolvedholger.knust
ResolvedTgr
Resolvedjijiki
ResolvedMSantos
Resolvedmobrovac
Resolvedovasileva
Resolvedphuedx
Declinedpmiazga
ResolvedDzahn
Resolvedpmiazga
Duplicateholger.knust
ResolvedMSantos
ResolvedTgr
ResolvedJohan
OpenNone
OpenNone
InvalidNone
Resolvedmobrovac
Resolvedmobrovac

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 20 2018, 6:58 PM

@ovasileva, @phuedx this is in the sprint for tracking because T186748 is blocked on it

Pchelolo moved this task from Backlog to watching on the Services board.Feb 20 2018, 9:49 PM
Pchelolo edited projects, added Services (watching); removed Services.
Restricted Application edited projects, added Readers-Web-Backlog; removed Readers-Web-Backlog (Tracking). · View Herald TranscriptFeb 21 2018, 3:14 PM

Given the requirements, I would be inclined to say Kubernetes, but we don't have any services on it yet. So perhaps Ganeti could temporarily host it? @Joe @akosiaris thoughts?

@ovasileva, this was discussed in today's Services sync. There are a couple options @mobrovac and @Pchelolo are thinking about but they need to get in touch with Ops (next week):

  1. Install Proton to a virtual machine.
  2. Find a spare machine and install Proton.
  3. Some other to-be-determined option from Ops.

The major blocker is that the Proton service requires a Debian Stretch operating system and current thinking is that, despite low traffic, it's too complicated and not a good candidate for a "first service" on Kubernetes. @mobrovac will update this ticket or ping us when he meets with Ops. Looking further ahead once this service is deployed, we’ll replay Electron renderer traffic to verify the load can be handled by Proton.

Legoktm renamed this task from Choose a deploy server for the chromium-render service to Choose a server for the chromium-render service.Feb 22 2018, 10:09 PM

Given the requirements, I would be inclined to say Kubernetes, but we don't have any services on it yet. So perhaps Ganeti could temporarily host it? @Joe @akosiaris thoughts?

Yes, that was the plan all along per T178189#3741790 (granted not the best task to craft that plan).

  1. Install Proton to a virtual machine.

This isn't out of the question, as @pmiazga did all of his performance testing against an instance of the service running on a VM (see T178278: Performance test the service for details).

ovasileva triaged this task as High priority.Feb 27 2018, 5:27 PM
ovasileva moved this task from Upcoming to Needs Prioritization on the Readers-Web-Backlog board.
phuedx added a comment.Mar 2 2018, 5:49 PM

@mobrovac Did you sync with SRE about this task per last week's Audiences Services Sync meeting?

@mobrovac Did you sync with SRE about this task per last week's Audiences Services Sync meeting?

I have. Getting the service out in production is stalled until next Q, with only one month left to the end of this quarter and a lot of work, time is very limited.

Jdlrobson changed the task status from Open to Stalled.Mar 8 2018, 4:34 PM

Hey @mobrovac. We're making final plans for the current quarter. Any idea when we should expect a place for the Proton production server to be available? I'm happy to continue the patches we worked on with @Pchelolo or clarify any ambiguities. Let me know how I can help you help us! Thank you! /cc @ovasileva @thcipriani

@Niedzielski next week @Joe @akosiaris and I will meet to make a decision on where exactly to put the service, and then we'll take a look at the patches. Roughly the next steps are:

  • decide where to put it in production
  • get the service working in beta
  • work towards getting it in production
  • enable traffic from RESTBase to it for the test phase
mobrovac closed this task as Resolved.Apr 13 2018, 3:33 PM
mobrovac claimed this task.
mobrovac edited projects, added Services (done); removed Services (watching).

We have decided to put it on Ganeti for now, so I'm resolving this task.

@Niedzielski I will be taking over your patches and modify them to speed up the process.

@mobrovac that's a great news.

Thanks so much @mobrovac!! Feel free to do whatever you want to the patches and let us know if help is needed. Thanks again!