Page MenuHomePhabricator

Create an LVS endpoint for jobrunners on videoscalers
Closed, ResolvedPublic

Description

Certain jobs are related to videoscalers and must run on videoscalers hosts exclusively, so we need an LVS endpoint configured for them in order to submit the jobs from ChangeProp

Details

Event Timeline

Pchelolo triaged this task as Normal priority.Mar 5 2018, 8:20 PM
Pchelolo created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 5 2018, 8:20 PM
Pchelolo removed a project: Epic.Mar 5 2018, 8:20 PM
mobrovac renamed this task from Create an LVS endpoint for videoscalers to Create an LVS endpoint for jobrunners on videoscalers.Mar 5 2018, 8:23 PM
mobrovac added a project: User-Joe.
mobrovac added a project: Services (blocked).
mobrovac removed subscribers: Nuria, Aklapper.
fdans moved this task from Incoming to Radar on the Analytics board.Mar 8 2018, 6:22 PM
Joe added a comment.Mar 12 2018, 12:56 PM

I'd take the chance we have to do this to do as follows:

  1. Add mediawiki::multimedia to the jobrunners
  2. Add a second LVS IP, to be served from the same cluster, to use for videoscaling. This will guarantee we evenly distribute videoscaling jobs across all the backends
  3. Fold the old videoscalers in the new cluster, for added capacity

the differences between the two clusters are small enough (basically, some apache config we want to remove / unify anyways) that we really don't want to keep up with the cost of separation between the two.

These jobs are not high-traffic, so consolidating the job runners and spreading the load all over them sounds like a good approach to me, +1.

Add a second LVS IP, to be served from the same cluster, to use for videoscaling. This will guarantee we evenly distribute videoscaling jobs across all the backends

Do we even need the second LVS in case it will be the same exact cluster?

Add a second LVS IP, to be served from the same cluster, to use for videoscaling. This will guarantee we evenly distribute videoscaling jobs across all the backends

Do we even need the second LVS in case it will be the same exact cluster?

Yes, so that scaling and transcoding jobs are balanced across the cluster between themselves rather than being in the same pot as with the other (comparatively insignificantly CPU-intense) jobs.

Change 437210 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/dns@master] Add entries for the videoscaler VIP

https://gerrit.wikimedia.org/r/437210

Change 437223 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/puppet@production] jobrunner_tls: generalize to support videoscalers as well

https://gerrit.wikimedia.org/r/437223

Change 437224 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/puppet@production] role::mediawiki::videoscaler: add TLS termination

https://gerrit.wikimedia.org/r/437224

Change 437225 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/puppet@production] conftool-data: Add missing data for videoscalers

https://gerrit.wikimedia.org/r/437225

Change 437226 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[operations/puppet@production] lvs::configuration: Add configuration for the videoscaler service

https://gerrit.wikimedia.org/r/437226

Change 437223 merged by Giuseppe Lavagetto:
[operations/puppet@production] jobrunner_tls: generalize to support videoscalers as well

https://gerrit.wikimedia.org/r/437223

Change 437224 merged by Giuseppe Lavagetto:
[operations/puppet@production] role::mediawiki::videoscaler: add TLS termination

https://gerrit.wikimedia.org/r/437224

Change 437225 merged by Giuseppe Lavagetto:
[operations/puppet@production] conftool-data: Add missing data for videoscalers

https://gerrit.wikimedia.org/r/437225

Change 437210 merged by Giuseppe Lavagetto:
[operations/dns@master] Add entries for the videoscaler VIP

https://gerrit.wikimedia.org/r/437210

Change 437226 merged by Giuseppe Lavagetto:
[operations/puppet@production] lvs::configuration: Add configuration for the videoscaler service

https://gerrit.wikimedia.org/r/437226

Joe closed this task as Resolved.Jun 4 2018, 3:10 PM

The LVS endpoint is now available at videoscaler.discovery.wmnet