Page MenuHomePhabricator
Create Task
Maniphest T313953

Scap3-ify Phabricator
Closed, ResolvedPublic5 Estimated Story Points
Actions

Description

  • Ensure that scap deploy works from deploy1002:/srv/deployment/phabricator/deployment
  • systemctl stop phd prior to update, systemctl start phd afterwards. (Done in check script.)
  • There are currently a /srv/phab/libphutil and /srv/phab/libext/security on phab1001. These are untracked and according to READMEs in both, I don't think they're doing anything, but we should make sure.
    • Going to take READMEs and repo contents at face value here.
    • Both have remote repos.
  • Symlink /srv/phab/repos to /srv/repos
    • Does anything actually use this? Config has "repository.default-local-path": "/srv/repos"... Seems like no.
  • Run storage migrations after code updates with /usr/local/sbin/phab_deploy_finalize - this is puppetized and should already be available on phab2001.
    • Check that migrations are actually being run

Details

Show related patches Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedDzahnT280597 move phabricator to new hardware generation
 ResolvedbrennenT313259 Deploy Phabricator with scap
 ResolveddduvallT313953 Scap3-ify Phabricator

Event Timeline

dduvall created this task.Jul 27 2022, 4:53 PM
dduvall mentioned this in T313259: Deploy Phabricator with scap.Jul 27 2022, 4:55 PM
brennen set the point value for this task to 5.Jul 27 2022, 5:15 PM
brennen added a comment.Jul 27 2022, 6:46 PM

We were experimenting with something like this for managing services and migrations yesterday, in scap/checks.yaml:

checks:
  stop_services:
    type: command
    stage: fetch
    command: "systemctl stop phd && systemctl stop apache2 && systemctl stop php7.3-fpm"
    timeout: 120

  finalize_deploy:
    type: script
    stage: promote
    command: "/usr/local/sbin/phab_deploy_finalize"
    timeout: 3600

Notes:

  • checks are run after their specified stage
  • My model of this is that using scap's builtin service_name to specify a service to restart after the promote stage won't work here, because what we want is to:
    • stop services
    • swap in new code
    • make sure config is in place
    • run any storage migrations
    • restart services
  • checks.yaml has to be committed to the repo to be run on deploy, since it's run from the target
  • I'm not sure if the systemctl stop phd && systemctl stop apache2 && systemctl stop php7.3-fpm pattern will work since I don't know if the command actually hits a shell or what. If not, maybe that needs to be changed to a type: script check.
  • /usr/local/sbin/phab_deploy_finalize is managed by Puppet, since it needs access to the MySQL password secret.
brennen updated the task description. (Show Details)Jul 28 2022, 8:58 PM
brennen updated the task description. (Show Details)Jul 28 2022, 9:01 PM
brennen added a comment.Jul 28 2022, 9:25 PM

Further experimented with checks.yaml and got PermissionErrors because there's a root-owned symlink in each checkout, put in place by puppet. We'll need to clear out everything puppet manages this way in order to make scap deployments viable.

gerritbot added a comment.Jul 28 2022, 9:30 PM

Change 818231 had a related patch set uploaded (by Brennen Bearnes; author: Brennen Bearnes):

[phabricator/deployment@wmf/stable] scap: stub out a checks.yaml

https://gerrit.wikimedia.org/r/818231

gerritbot added a project: Patch-For-Review.Jul 28 2022, 9:30 PM
gerritbot added a comment.Aug 3 2022, 8:08 PM

Change 820213 had a related patch set uploaded (by Dduvall; author: Dduvall):

[operations/puppet@production] phabricator: Stop managing /srv/phab/repos

https://gerrit.wikimedia.org/r/820213

gerritbot added a comment.Aug 4 2022, 1:36 AM

Change 820213 merged by Dzahn:

[operations/puppet@production] phabricator: Stop managing /srv/phab/repos

https://gerrit.wikimedia.org/r/820213

gerritbot added a comment.Aug 4 2022, 5:25 PM

Change 818231 merged by Brennen Bearnes:

[phabricator/deployment@wmf/stable] scap: stub out a checks.yaml

https://gerrit.wikimedia.org/r/818231

dduvall mentioned this in rPHDEPfe89eda96dc7: scap: stub out a checks.yaml.Aug 4 2022, 5:27 PM
Maintenance_bot removed a project: Patch-For-Review.Aug 4 2022, 5:30 PM
gerritbot added a comment.Aug 12 2022, 9:28 PM

Change 822688 had a related patch set uploaded (by Brennen Bearnes; author: Brennen Bearnes):

[phabricator/deployment@wmf/stable] scap: add permission mangling, reorder checks

https://gerrit.wikimedia.org/r/822688

gerritbot added a project: Patch-For-Review.Aug 12 2022, 9:28 PM
thcipriani edited projects, added Release-Engineering-Team (Bonus Level 🕹ī¸); removed Release-Engineering-Team (The Decommission Mission 💀).Aug 16 2022, 7:59 PM
gerritbot added a comment.Aug 18 2022, 6:15 PM

Change 824547 had a related patch set uploaded (by Brennen Bearnes; author: Brennen Bearnes):

[operations/puppet@production] phabricator: remove phab_deploy_ensure_config_ownership.sh

https://gerrit.wikimedia.org/r/824547

gerritbot added a comment.Aug 18 2022, 10:49 PM

Change 824547 merged by Dzahn:

[operations/puppet@production] phabricator: remove phab_deploy_ensure_config_ownership.sh

https://gerrit.wikimedia.org/r/824547

Stashbot added a comment.Aug 18 2022, 10:53 PM

Mentioned in SAL (#wikimedia-operations) [2022-08-18T22:53:23Z] <mutante> phab1001, phab2001: sudo rm /usr/local/sbin/phab_deploy_ensure_config_ownership (follow-up gerrit:824547 T313953)

brennen moved this task from Backlog to In progress on the Release-Engineering-Team (Bonus Level 🕹ī¸) board.Aug 22 2022, 4:06 PM
brennen updated the task description. (Show Details)Aug 22 2022, 4:13 PM
gerritbot added a comment.Aug 22 2022, 4:51 PM

Change 822688 merged by Brennen Bearnes:

[phabricator/deployment@wmf/stable] scap: add permission mangling, reorder checks

https://gerrit.wikimedia.org/r/822688

Maintenance_bot removed a project: Patch-For-Review.Aug 22 2022, 5:30 PM
brennen mentioned this in rPHDEP53e8c5d7870d: scap: add permission mangling, reorder checks.Aug 22 2022, 5:53 PM
gerritbot added a comment.Aug 23 2022, 9:26 PM

Change 825911 had a related patch set uploaded (by Brennen Bearnes; author: Brennen Bearnes):

[phabricator/deployment@wmf/stable] scap: separate new rev perms from old rev perm cleanup

https://gerrit.wikimedia.org/r/825911

gerritbot added a project: Patch-For-Review.Aug 23 2022, 9:26 PM
gerritbot added a comment.Aug 31 2022, 11:12 PM

Change 828654 had a related patch set uploaded (by Dduvall; author: Dduvall):

[operations/puppet@production] phabricator: Reintroduce script to ensure correct config ownership/perms

https://gerrit.wikimedia.org/r/828654

gerritbot added a comment.Aug 31 2022, 11:25 PM

Change 828655 had a related patch set uploaded (by Dduvall; author: Dduvall):

[phabricator/deployment@wmf/stable] Run all puppetized deploy scripts as checks

https://gerrit.wikimedia.org/r/828655

gerritbot added a comment.Sep 1 2022, 12:17 AM

Change 828654 merged by Dzahn:

[operations/puppet@production] phabricator: Reintroduce script to ensure correct config ownership/perms

https://gerrit.wikimedia.org/r/828654

gerritbot added a comment.Sep 1 2022, 8:22 PM

Change 829063 had a related patch set uploaded (by Dduvall; author: Dduvall):

[operations/puppet@production] phabricator: Deploy user should own everything under old rev directories

https://gerrit.wikimedia.org/r/829063

gerritbot added a comment.Sep 2 2022, 10:19 PM

Change 829063 merged by Dzahn:

[operations/puppet@production] phabricator: Deploy user should own everything under old rev directories

https://gerrit.wikimedia.org/r/829063

gerritbot added a comment.Sep 6 2022, 6:31 PM

Change 830234 had a related patch set uploaded (by Dduvall; author: Dduvall):

[operations/puppet@production] phabricator: Add missing line continuation to phab_deploy_promote

https://gerrit.wikimedia.org/r/830234

gerritbot added a comment.Sep 6 2022, 6:36 PM

Change 830234 merged by Dzahn:

[operations/puppet@production] phabricator: Add missing line continuation to phab_deploy_promote

https://gerrit.wikimedia.org/r/830234

gerritbot added a comment.Sep 6 2022, 7:05 PM

Change 830241 had a related patch set uploaded (by Dduvall; author: Dduvall):

[operations/puppet@production] phabricator: Include scap "done" rev in pre-finalize permissions reset

https://gerrit.wikimedia.org/r/830241

gerritbot added a comment.Sep 6 2022, 7:13 PM

Change 830241 merged by Dzahn:

[operations/puppet@production] phabricator: Include scap "done" rev in pre-finalize permissions reset

https://gerrit.wikimedia.org/r/830241

gerritbot added a comment.Sep 7 2022, 4:34 PM

Change 828655 merged by Dduvall:

[phabricator/deployment@wmf/stable] Run all puppetized deploy scripts as checks

https://gerrit.wikimedia.org/r/828655

dduvall mentioned this in rPHDEPa7616e67c795: Run all puppetized deploy scripts as checks.Sep 7 2022, 5:56 PM
gerritbot added a comment.Sep 13 2022, 6:05 PM

Change 831944 had a related patch set uploaded (by Dduvall; author: Dduvall):

[phabricator/deployment@wmf/stable] scap: Remove use of --preserve-env for sudo'd scripts

https://gerrit.wikimedia.org/r/831944

Dzahn added a comment.Sep 14 2022, 12:04 AM

also see T313259#8234768

gerritbot added a comment.Sep 14 2022, 3:38 PM

Change 831944 merged by Dduvall:

[phabricator/deployment@wmf/stable] scap: Remove use of --preserve-env for sudo'd scripts

https://gerrit.wikimedia.org/r/831944

dduvall closed this task as Resolved.Sep 14 2022, 3:39 PM
dduvall claimed this task.
dduvall moved this task from In progress to Done on the Release-Engineering-Team (Bonus Level 🕹ī¸) board.
dduvall mentioned this in rPHDEP9bcfc9e665cc: scap: Remove use of --preserve-env for sudo'd scripts.Sep 14 2022, 4:05 PM
Dzahn awarded a token.Sep 14 2022, 10:03 PM
gerritbot added a comment.Sep 20 2022, 9:00 PM

Change 825911 abandoned by Brennen Bearnes:

[phabricator/deployment@wmf/stable] scap: separate new rev perms from old rev perm cleanup

Reason:

Superseded.

https://gerrit.wikimedia.org/r/825911

gerritbot added a comment.Dec 8 2022, 10:35 PM

Related commit fe89eda9 pushed by brennen (author: Brennen Bearnes):

[ repos/phabricator/deployment@wmf/stable ] scap: stub out a checks.yaml

gerritbot added a comment.Dec 8 2022, 10:35 PM

Related commit 53e8c5d7 pushed by brennen (author: Brennen Bearnes):

[ repos/phabricator/deployment@wmf/stable ] scap: add permission mangling, reorder checks

gerritbot added a comment.Dec 8 2022, 10:35 PM

Related commit a7616e67 pushed by brennen (author: Dan Duvall):

[ repos/phabricator/deployment@wmf/stable ] Run all puppetized deploy scripts as checks

gerritbot added a comment.Dec 8 2022, 10:36 PM

Related commit 9bcfc9e6 pushed by brennen (author: Dan Duvall):

[ repos/phabricator/deployment@wmf/stable ] scap: Remove use of --preserve-env for sudo'd scripts

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. ¡ Wikimedia Foundation ¡ Privacy Policy ¡ Code of Conduct ¡ Terms of Use ¡ Disclaimer ¡ CC-BY-SA ¡ GPL ¡ Credits