Page MenuHomePhabricator
Create Task
Maniphest T372081

Deploy Chart service in production
Open, HighPublic
Actions

Description

Now that T371278 is done, and seems to be working well, deploy the chart-renderer service in production.

Steps below follow https://wikitech.wikimedia.org/wiki/Kubernetes/Add_a_new_service

Steps are described in more detail either in a linked task, or, at the above link.

  • Begin New Service Request process T376939
  • Create k8s namespace and deployment users (@CDanis to do)
  • Write Helm chart for chart-renderer (Charts to do) T376948
  • Deploy the service to staging and production k8s clusters (Charts to do w/ help)
  • Deploy extension (T378127) (Charts to do w/ help)

Post test.wikipedia.org deploy:

  • make sure we have appropriate logging in place (Chris task)
  • Set up Grafana dashboard for monitoring, starting with making a copy of the template (Charts to do)

Doing it after testwiki is totally fine IMO.

We can get latency and also request success rates from different queries based on the one existing Prom metric.

  • Add alerts around latency ("under 5 seconds" perhaps), and one around 5xx result ratio
  • Configure Ingress (@CDanis to do no taskforce assistance needed)
  • Add chart-renderer to service mesh (@CDanis to do)

Details

SubjectRepoBranchLines +/-
deployment group: add audeoperations/puppetproduction+1 -1
Add chart-renderer namespaceoperations/deployment-chartsmaster+1 -0
Add chart-renderer deployment server profileoperations/puppetproduction+4 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

• Catrope created this task.Aug 8 2024, 4:50 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 8 2024, 4:50 PM
• Catrope mentioned this in T370379: Set up shellbox instance for Chart extension (placeholder, not actionable yet).Aug 8 2024, 4:51 PM
• Catrope mentioned this in T369944: Epic: Deploy Chart extension in production.
CCiufo-WMF triaged this task as Low priority.Sep 9 2024, 6:43 PM
gerritbot added a comment.Thu, Oct 10, 7:41 PM

Change #1079350 had a related patch set uploaded (by CDanis; author: CDanis):

[operations/deployment-charts@master] Add chart-renderer namespace

https://gerrit.wikimedia.org/r/1079350

gerritbot added a project: Patch-For-Review.Thu, Oct 10, 7:41 PM

Change #1079345 had a related patch set uploaded (by CDanis; author: CDanis):

[operations/puppet@production] Add chart-renderer deployment server profile

https://gerrit.wikimedia.org/r/1079345

CDanis renamed this task from Deploy Chart service in production (placeholder, not actionable yet) to Deploy Chart service in production.Thu, Oct 10, 7:53 PM
CDanis raised the priority of this task from Low to Medium.
CDanis updated the task description. (Show Details)
CDanis updated the task description. (Show Details)Thu, Oct 10, 8:07 PM
gerritbot added a comment.Wed, Oct 16, 5:43 PM

Change #1079345 merged by CDanis:

[operations/puppet@production] Add chart-renderer deployment server profile

https://gerrit.wikimedia.org/r/1079345

gerritbot added a comment.Wed, Oct 16, 5:53 PM

Change #1079350 merged by jenkins-bot:

[operations/deployment-charts@master] Add chart-renderer namespace

https://gerrit.wikimedia.org/r/1079350

Maintenance_bot removed a project: Patch-For-Review.Wed, Oct 16, 6:31 PM
JMeybohm subscribed.Thu, Oct 17, 4:48 AM
  • Configure LVS service (@CDanis to do)

What are the expectations regarding traffic here? Could this use Ingress instead of LVS?

JMeybohm added a project: serviceops-radar.Thu, Oct 17, 4:49 AM
CDanis added a comment.Thu, Oct 17, 11:01 AM
In T372081#10236490, @JMeybohm wrote:

What are the expectations regarding traffic here? Could this use Ingress instead of LVS?

Discussed with Janis on serviceops IRC and our best guess is that Ingress is totally viable for this service. This removes/simplifies a couple steps which is nice.

CDanis updated the task description. (Show Details)Thu, Oct 17, 7:41 PM
CDanis updated the task description. (Show Details)
CCiufo-WMF raised the priority of this task from Medium to High.Mon, Oct 21, 5:43 PM
CCiufo-WMF assigned this task to Jdlrobson.Mon, Oct 21, 6:29 PM
CCiufo-WMF moved this task from Needs Triage to Sprint 9 on the Charts board.
CCiufo-WMF edited projects, added Charts (Sprint 9); removed Charts.
CDanis closed subtask T377595: Prometheus export of basic request success & latency metrics as Resolved.Mon, Oct 21, 7:40 PM
CDanis updated the task description. (Show Details)Mon, Oct 21, 8:05 PM
Jdlrobson updated the task description. (Show Details)Thu, Oct 24, 1:26 AM
Jdlrobson updated the task description. (Show Details)Mon, Oct 28, 6:49 PM
gerritbot added a comment.Fri, Nov 1, 3:51 PM

Change #1085607 had a related patch set uploaded (by CDanis; author: CDanis):

[operations/puppet@production] deployment group: add aude

https://gerrit.wikimedia.org/r/1085607

gerritbot added a project: Patch-For-Review.Fri, Nov 1, 3:51 PM
gerritbot added a comment.Fri, Nov 1, 6:32 PM

Change #1085607 merged by CDanis:

[operations/puppet@production] deployment group: add aude

https://gerrit.wikimedia.org/r/1085607

Maintenance_bot removed a project: Patch-For-Review.Fri, Nov 1, 7:30 PM
CCiufo-WMF moved this task from Sprint 9 to Needs Triage on the Charts board.Mon, Nov 4, 7:10 PM
CCiufo-WMF edited projects, added Charts; removed Charts (Sprint 9).
CCiufo-WMF moved this task from Needs Triage to Sprint 10 on the Charts board.Mon, Nov 4, 7:20 PM
CCiufo-WMF edited projects, added Charts (Sprint 10); removed Charts.
CCiufo-WMF reassigned this task from Jdlrobson to aude.Mon, Nov 4, 7:29 PM
CCiufo-WMF added a subscriber: Jdlrobson.
Jdlrobson moved this task from Incoming to Ready for Dev on the Charts (Sprint 10) board.Mon, Nov 4, 9:49 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits