Page MenuHomePhabricator
Create Task
Maniphest T200109

Maintain and deploy Arc Lamp code from its own repository (outside Puppet)
Closed, ResolvedPublic
Actions

Description

  • Create git repository for performance/arc-lamp.
  • Import history from https://github.com/wikimedia/arc-lamp.
  • Delete old wikimedia/arc-lamp GitHub repo and the Gerrit mirror at wikimedia/performance-arc-lamp to wikimedia/arc-lamp (thus keeping the old place to keep URLs working).
  • Import current arclamp source code from operations/puppet.

Using Scap (cancelled):

  • Add scap bootstrap to arc-lamp repository.
  • Add arc-lamp to scap::sources in Beta and in production DCs.
  • Ensure webperf-2 is a scap::target in Beta and production DCs.
  • Update profile::webperf::arclamp in Puppet to provision the tools from this repository.

Details

SubjectRepoBranchLines +/-
arclamp: Deploy from scapoperations/puppetproduction+34 -1 K
Prepare for scap deploymentperformance/arc-lampmaster+1 K -4
Switch from HHVM Xenon to php7-excimer and improve documentationperformance/arc-lampmaster+381 -254
Add composer-test job for performance/arc-lampintegration/configmaster+4 -0
WIP: Turn into a real python package, with testsperformance/arc-lampmaster+164 -0
Add .gitreview fileperformance/arc-lampmaster+6 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

Krinkle created this task.Jul 20 2018, 9:47 PM
• Imarlier subscribed.Jul 20 2018, 9:56 PM

At any stage in this process where it's possible to NOT use Puppet, we should do that. Overlaps between Puppet and applications are bad, and should be avoid. (Frankly, this includes the scap::sources puppet setup, which is a terrible hack that we should eliminate ASAP.)

gerritbot subscribed.Jul 22 2018, 10:52 PM

Change 447342 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[performance/arc-lamp@master] Add .gitreview file

https://gerrit.wikimedia.org/r/447342

gerritbot added a project: Patch-For-Review.Jul 22 2018, 10:52 PM

Change 447342 merged by Krinkle:
[performance/arc-lamp@master] Add .gitreview file

https://gerrit.wikimedia.org/r/447342

• Imarlier moved this task from Inbox, needs triage to Backlog: Maintenance, non-prioritized on the Performance-Team board.Jul 23 2018, 8:07 PM
Krinkle removed a project: Patch-For-Review.Jul 23 2018, 8:10 PM
gerritbot added a comment.Jul 23 2018, 9:49 PM

Change 447555 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[performance/arc-lamp@master] WIP: Turn into a real python package, with tests

https://gerrit.wikimedia.org/r/447555

gerritbot added a project: Patch-For-Review.Jul 23 2018, 9:49 PM
Krinkle removed a project: Patch-For-Review.Jul 28 2018, 2:09 AM
gerritbot added a comment.Mar 8 2019, 6:35 PM

Change 447555 abandoned by Krinkle:
WIP: Turn into a real python package, with tests

https://gerrit.wikimedia.org/r/447555

aaron triaged this task as Low priority.Jun 6 2019, 10:07 AM
Krinkle mentioned this in T227026: Deploy ArcLamp process as stateless/scalable service (Kubernetes).Jul 1 2019, 7:31 PM
Krinkle added a project: Arc-Lamp.Aug 5 2019, 5:52 PM
Krinkle mentioned this in T235456: Let Arc-Lamp store its trace "log" files in compressed format.Oct 14 2019, 10:32 PM
gerritbot added a comment.Dec 21 2019, 9:56 PM

Change 560128 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[integration/config@master] Add composer-test job for performance/arc-lamp

https://gerrit.wikimedia.org/r/560128

gerritbot added a project: Patch-For-Review.Dec 21 2019, 9:56 PM

Change 560125 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[performance/arc-lamp@master] Switch from HHVM Xenon to php7-excimer and improve documentation

https://gerrit.wikimedia.org/r/560125

Krinkle updated the task description. (Show Details)Dec 21 2019, 9:57 PM
gerritbot added a comment.Dec 21 2019, 9:59 PM

Change 560128 merged by jenkins-bot:
[integration/config@master] Add composer-test job for performance/arc-lamp

https://gerrit.wikimedia.org/r/560128

gerritbot added a comment.Dec 21 2019, 10:03 PM

Change 560125 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[performance/arc-lamp@master] Switch from HHVM Xenon to php7-excimer and improve documentation

https://gerrit.wikimedia.org/r/560125

gerritbot added a comment.Dec 21 2019, 10:04 PM

Change 560125 merged by jenkins-bot:
[performance/arc-lamp@master] Switch from HHVM Xenon to php7-excimer and improve documentation

https://gerrit.wikimedia.org/r/560125

Krinkle mentioned this in T238080: Phrasing of arclamp page on perf site needs to be fixed.Dec 21 2019, 11:14 PM
• Gilles assigned this task to • dpifke.Jan 7 2020, 11:35 AM
Krinkle renamed this task from Move Arc Lamp code to a Scap-deployed repository to Main and deploy Arc Lamp code from its own repository (outside Puppet).Mar 15 2020, 11:37 PM
Krinkle added a parent task: T227026: Deploy ArcLamp process as stateless/scalable service (Kubernetes).
Krinkle updated the task description. (Show Details)Mar 15 2020, 11:39 PM
Krinkle added a comment.Mar 15 2020, 11:42 PM

@dpifke I've crossed out the Scap-deploy plan in light of T227026. Do you want to go direcrly from the current files ensured by Puppet and the K8s service from the standalone repo, or have an intermediate step where e.g. puppet ensures the files from the standalone repo?

I suppose this might depend on how long we expect the migration to take, and whether it would reduce complexity that way (not having two versions to maintain short-term), or that it would add complexity (e.g. one source repo needing to work both ways). I'm thinking it might be good to have a clean break and deal with the two places issue as needed. What do you think?

Krinkle moved this task from Backlog: Maintenance, non-prioritized to Inbox, needs triage on the Performance-Team board.Mar 15 2020, 11:42 PM
Krinkle removed a project: Patch-For-Review.Mar 16 2020, 3:01 AM
aaron renamed this task from Main and deploy Arc Lamp code from its own repository (outside Puppet) to Maintain and deploy Arc Lamp code from its own repository (outside Puppet).Mar 16 2020, 9:32 PM
Krinkle closed this task as Resolved.Mar 16 2020, 9:34 PM

No need for a separate stage, per Dave doing that within Puppet as-is is fine. Continuing at T227026. .

• dpifke reopened this task as Open.Jun 23 2020, 9:51 PM

Re-opening this, as there's value in moving these files out of Puppet in the short-term. Patches forthcoming.

gerritbot added a comment.Jun 23 2020, 10:00 PM

Change 607369 had a related patch set uploaded (by Dave Pifke; owner: Dave Pifke):
[performance/arc-lamp@master] Sync with Puppet and prepare for scap deployment

https://gerrit.wikimedia.org/r/607369

gerritbot added a project: Patch-For-Review.Jun 23 2020, 10:00 PM
gerritbot added a comment.Jun 23 2020, 10:04 PM

Change 607370 had a related patch set uploaded (by Dave Pifke; owner: Dave Pifke):
[operations/puppet@production] [WIP] arclamp: Deploy from scap

https://gerrit.wikimedia.org/r/607370

gerritbot added a comment.Jun 25 2020, 11:35 PM

Change 607369 merged by jenkins-bot:
[performance/arc-lamp@master] Prepare for scap deployment

https://gerrit.wikimedia.org/r/607369

• dpifke moved this task from Inbox, needs triage to Doing (old) on the Performance-Team board.Jun 29 2020, 7:30 PM
gerritbot added a comment.Jul 1 2020, 8:50 PM

Change 607370 merged by Dzahn:
[operations/puppet@production] arclamp: Deploy from scap

https://gerrit.wikimedia.org/r/c/operations/puppet/ /607370

• dpifke closed this task as Resolved.Jul 1 2020, 10:11 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL