puppet/scap: failing on aphlict2001
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	jbond
	Feb 23 2023, 12:34 PM

Description

Puppet is currently failing on aphlict2001. Ultimately this is due to the following scap error

 sudo -u phab-deploy /usr/bin/scap deploy-local --repo phabricator/deployment -D log_json:False
12:31:50 Fetch from: http://deploy1002.eqiad.wmnet/phabricator/deployment/.git
12:31:50 Update submodules
12:31:50 Updating .gitmodule: /srv/deployment/phabricator/deployment-cache/cache
12:31:50 Revision directory already exists (use --force to override)
12:31:50 Rendering config_file: /srv/deployment/phabricator/deployment-cache/revs/f68dc24350bc897955d57eeac681db872b0b9e61/phabricator/conf/local/local.json using /etc/phabricator/config.yaml
12:31:50 Unhandled error:
Traceback (most recent call last):
  File "/var/lib/scap/scap/lib/python3.9/site-packages/scap/cli.py", line 532, in run
    exit_status = app.main(app.extra_arguments)
  File "/var/lib/scap/scap/lib/python3.9/site-packages/scap/deploy.py", line 161, in main
    getattr(self, stage)()
  File "/var/lib/scap/scap/lib/python3.9/site-packages/scap/deploy.py", line 237, in config_deploy
    f.write(tmpl.render())
  File "/var/lib/scap/scap/lib/python3.9/site-packages/scap/template.py", line 135, in render
    template_vars = self._get_file_vars()
  File "/var/lib/scap/scap/lib/python3.9/site-packages/scap/template.py", line 125, in _get_file_vars
    with open(self.var_file, "r") as variables:
FileNotFoundError: [Errno 2] No such file or directory: '/etc/phabricator/config.yaml'
12:31:50 deploy-local failed: <FileNotFoundError> [Errno 2] No such file or directory: '/etc/phabricator/config.yaml'

Ultimately it seems like something needs to create the /etc/phabricator directory on this system

Details

	Title	Reference	Author	Source Branch	Dest Branch
	scap: add aphlict2001 to phabricator-targets	repos/phabricator/deployment!3	brennen	work/aphlict2001-target	wmf/stable

Customize query in GitLab

Related Objects
Search...

Status	Assigned	Task
Resolved	dancy	T292646 Reduce latency of new Scap releases
Resolved	jnuche	T303559 Use scap to deploy itself to scap targets
Resolved	jnuche	T326668 Scap fails on debian bullseye targets
Resolved	eoghan	T322369 create aphlict2001 (Phabricator realtime notifications codfw)
Resolved	eoghan	T329908 Deploy phabricator to aphlict2001.codfw.wmnet
Resolved	eoghan	T330393 puppet/scap: failing on aphlict2001

Event Timeline

jbond created this task.Feb 23 2023, 12:34 PM

jbond mentioned this in T326668: Scap fails on debian bullseye targets.Feb 23 2023, 12:37 PM

@Dzahn /etc/phabricator and /etc/phabricator/config.yaml are only defined in the main class phabricator which is not present for the aphlict hosts.

@Dzahn /etc/phabricator and /etc/phabricator/config.yaml are only defined in the main class phabricator which is not present for the aphlict hosts.

Yeah, was just looking at T329908: Deploy phabricator to aphlict2001.codfw.wmnet. I'll push a commit to add it to the scap targets after I post this comment, but confirming that a scap deploy will still require the config.yaml.

@MoritzMuehlenhoff Thanks for reporting! We are aware of this and it's because aphlict2001 is still being setup (T322369). We just recently created it and it's not yet in production. The puppet role was applied and then the next step is that we need a Phabricator scap deploy to this host which should fix this. The production host is aphlict1001 so no impact on prod.

Dzahn added parent tasks: T322369: create aphlict2001 (Phabricator realtime notifications codfw), T329908: Deploy phabricator to aphlict2001.codfw.wmnet.Feb 23 2023, 7:09 PM

brennen opened https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/3

scap: add aphlict2001 to phabricator-targets

gerritbot added a project: Patch-For-Review.Feb 23 2023, 7:10 PM

Dzahn triaged this task as Medium priority.Feb 23 2023, 7:10 PM

Dzahn mentioned this in T322369: create aphlict2001 (Phabricator realtime notifications codfw).

Dzahn mentioned this in T329908: Deploy phabricator to aphlict2001.codfw.wmnet.

brennen merged https://gitlab.wikimedia.org/repos/phabricator/deployment/-/merge_requests/3

scap: add aphlict2001 to phabricator-targets

Maintenance_bot removed a project: Patch-For-Review.Feb 23 2023, 7:30 PM

Mentioned in SAL (#wikimedia-operations) [2023-02-23T19:50:19Z] <mutante> aphlict2001 - manually created /etc/phabricator/config.yaml - empty file owned by root:phab-deploy to debug for T330393 T322369

19:41 < mutante> scap has defaults to use gerrit as origin: modules/scap/lib/puppet/provider/scap_source/default.rb: "https://gerrit.wikimedia.org/r/#{repo_name}.git"
19:42 < mutante> so you cant use gitlab repos as origin for something deployed with scap
19:42 < mutante> this currently is a problem for phabricator repos which got moved from phabricator itself to gitlab

In T330393#8641405, @MoritzMuehlenhoff wrote:

@Dzahn /etc/phabricator and /etc/phabricator/config.yaml are only defined in the main class phabricator which is not present for the aphlict hosts.

ACK, confirmed. Was just talking to @eoghan about this. This is because aphlict and phabricator used to be on the same host and were split later. We will fix it shortly!

hashar updated the task description. (Show Details)Feb 24 2023, 2:05 PM

also see T326668 (scap failed on bullseye targets, that is fixed now. aphlict is on bullseye. most things that are scap targets are not bullseye, I think.