Page MenuHomePhabricator

Move Dan's CI metrics cron script / HTML output from people.wikimedia.org to doc.wikimedia.org
Open, MediumPublic

Description

https://people.wikimedia.org/~dduvall/jenkins/

Code was being run in a personal crontab, which didn't stay when the machine was re-imaged (from people1001 to people1002).

Let's move it into doc1001 so it's more stable.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 17 2020, 6:16 PM

Possible good first task; needs the current output to be scped from people to doc, the code to be moved to doc and installed in a crontab, and then properly puppetised so it stays around the next time the host gets re-imaged (but that could be a separate, spun-out task).

Where is the source code for this?

@Jdforrester-WMF: good first task is for tasks which do not require any special permissions in order to test the contribution or to fix the task. Anyone should be able to reproduce. As scp was mentioned I assume that's not the case but please correct me if I misunderstand!

We're intentionally using the tag for tasks for our new starter. I hope you've not removed it from all those carefully curated tasks? Eurgh.

Where is the source code for this?

On people1002:/home/dduvall.

hashar added a subscriber: hashar.Jun 29 2020, 2:22 PM

Alternative is to have a regular Jenkins job to generate the reports then publish to doc1001. That has a few benefits:

  • keeps doc1001 pristine clear and just hosting static assets
  • we can retrigger / tweak the batch however we want via JJB/Jenkins config

We should also get the scripts published in some git repo (maybe integration/config is a good enough fit for that).

(else yeah +1 on the principle of having it documented / shared ;D )

[OT] @Jdforrester-WMF: I hope you (plural; team) will remove the tag from all those tasks, otherwise I will have to do that soon. Thanks :)

You should take that up with the owners of Phabricator. ;-)

Where is the source code for this?

On people1002:/home/dduvall.

I pushed it to gerrit a while back, and there's a patch that needs merging. After that, it should be easy to restore the cron job. Long term, this should be moved to WMCS. I can probably hack on that in the near future.

dduvall claimed this task.Sep 10 2020, 3:56 PM
dduvall triaged this task as Medium priority.

I somehow never noticed https://gerrit.wikimedia.org/r/c/integration/reporting/+/589083 , I gave it a quick review and just submitted it :]

Would be nice to have it run on one of the production machine, as to where and how I don't really now:

  • a Jenkins job on CI, though I would like to eventually remove the timed jobs
  • a Jenkins job on https://releases-jenkins.wikimedia.org/ , but it would not be able to publish outside of that
  • a cronjob on the primary contint that would potentially published somewhere under https://integration.wikimedia.org/ (I have some work going on to have files publishable under that host) potentially using a ruby docker container.
  • or yeah wmcs.