Page MenuHomePhabricator
Create Task
Maniphest T273822

De-symlink our repos
Closed, ResolvedPublic
Actions

Description

Just adding this as another task in the sprint as our symlinks create both daily complexity & docker related complexity - this task spin-off came our of tech talk & it makes sense

Note this is actually important for our civiproxy docker specifically.

The directories involved are

  • modules (this is a convenience symlink & is the low hanging fruit)
  • civicrm
  • sites

in order to fix the last of these we need to convert the drupal directory from a submodule to a normal folder. Note that we don't care much about the history of the existing submodule because we basically alternate 2 commits in it

  1. drupal stock commit
  2. re-apply wmff patches - eg https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/drupal/+/661276

(some of those patches might be unnecessary without the symlinks)

@Cstone I updated ^^

Details

Show related patches Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 OpenNoneT262971 Epic: feature parity with vagrant on docker
 ResolvedEileenmcnaughtonT273822 De-symlink our repos

Event Timeline

Eileenmcnaughton created this task.Feb 3 2021, 9:23 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 3 2021, 9:23 PM
Eileenmcnaughton added a parent task: T262971: Epic: feature parity with vagrant on docker.Feb 3 2021, 9:23 PM
gerritbot added a comment.Feb 3 2021, 9:36 PM

Change 661480 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] One symlink down

https://gerrit.wikimedia.org/r/661480

gerritbot added a project: Patch-For-Review.Feb 3 2021, 9:36 PM
gerritbot added a comment.Feb 3 2021, 10:08 PM

Change 661509 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Remove civicrm symlink

https://gerrit.wikimedia.org/r/661509

Eileenmcnaughton updated the task description. (Show Details)Feb 3 2021, 10:30 PM
Eileenmcnaughton added a subscriber: Cstone.
Eileenmcnaughton mentioned this in T272061: Update civi prod with latest search kit.Feb 3 2021, 11:26 PM
gerritbot added a comment.Feb 4 2021, 12:36 AM

Change 661538 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Convert drupal submodule to being 'just a folder'

https://gerrit.wikimedia.org/r/661538

gerritbot added a comment.Feb 4 2021, 12:59 AM

Change 661541 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Convert drupal submodule to being 'just a folder'

https://gerrit.wikimedia.org/r/661541

gerritbot added a comment.Feb 5 2021, 12:30 AM

Change 661538 merged by Eileen:
[wikimedia/fundraising/crm@master] Convert drupal submodule to being 'just a folder'

https://gerrit.wikimedia.org/r/661538

gerritbot added a comment.Feb 5 2021, 12:31 AM

Change 661541 abandoned by Eileen:
[wikimedia/fundraising/crm@master] Convert drupal submodule to being 'just a folder'

Reason:
otherwise merged

https://gerrit.wikimedia.org/r/661541

gerritbot added a comment.Feb 5 2021, 12:58 AM

Change 661826 had a related patch set uploaded (by Eileen; owner: Eileen):
[integration/config@master] Remove drupal submodule from docker zuul coner

https://gerrit.wikimedia.org/r/661826

gerritbot added a comment.Feb 5 2021, 1:06 AM

Change 661828 had a related patch set uploaded (by Eileen; owner: Eileen):
[integration/config@master] Attempt to switch to recurse_submodules

https://gerrit.wikimedia.org/r/661828

gerritbot added a comment.Feb 5 2021, 1:27 AM

Change 661829 had a related patch set uploaded (by Eileen; owner: Eileen):
[integration/config@master] Switch fundraising tech setup to use recurse_submodules

https://gerrit.wikimedia.org/r/661829

Eileenmcnaughton added a comment.Feb 5 2021, 1:45 AM

@Cstone & I forced the change to the drupal repo through just now in the hope the jenkins problem was just trying to find it's way around applying the changes (it appeared to have issues with the remove & re-add).

However, that was not joyous and I believe that we need this merged https://gerrit.wikimedia.org/r/c/integration/config/+/661829 before it will work. I'm going to revert the change in the meantime & try to get someone in releng to +2

gerritbot added a comment.Feb 5 2021, 1:48 AM

Change 661829 abandoned by Eileen:
[integration/config@master] Switch fundraising tech setup to use recurse_submodules

Reason:
https://gerrit.wikimedia.org/r/c/integration/config/ /661829 instead

https://gerrit.wikimedia.org/r/661829

gerritbot added a comment.Feb 5 2021, 1:48 AM

Change 661826 abandoned by Eileen:
[integration/config@master] Remove drupal submodule from docker zuul coner

Reason:
https://gerrit.wikimedia.org/r/c/integration/config/ /661829 instead

https://gerrit.wikimedia.org/r/661826

gerritbot added a comment.Feb 5 2021, 1:49 AM

Change 661829 restored by Eileen:
[integration/config@master] Switch fundraising tech setup to use recurse_submodules

https://gerrit.wikimedia.org/r/661829

gerritbot added a comment.Feb 5 2021, 1:50 AM

Change 661828 abandoned by Eileen:
[integration/config@master] Attempt to switch to recurse_submodules

Reason:
https://gerrit.wikimedia.org/r/c/integration/config/ /661829/2/jjb/wm-fundraising.yaml

https://gerrit.wikimedia.org/r/661828

gerritbot added a comment.Feb 5 2021, 9:32 PM

Change 661480 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] One symlink down

https://gerrit.wikimedia.org/r/661480

gerritbot added a comment.Feb 9 2021, 4:11 AM

Change 661509 abandoned by Eileen:
[wikimedia/fundraising/crm@master] Remove civicrm symlink

Reason:
Will redo once preliminary steps are done (reduce review noise)

https://gerrit.wikimedia.org/r/661509

• DStrine moved this task from Triage to Current Sprint on the Fundraising-Backlog board.Feb 9 2021, 5:21 PM
gerritbot added a comment.Feb 15 2021, 9:49 PM

Change 664359 had a related patch set uploaded (by Eileen; owner: Eileen):
[integration/config@master] Remove reference to submodule

https://gerrit.wikimedia.org/r/664359

gerritbot added a comment.Feb 15 2021, 9:51 PM

Change 664360 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Convert drupal submodule to being 'just a folder'

https://gerrit.wikimedia.org/r/664360

Eileenmcnaughton moved this task from Backlog to Doing on the Fundraising Sprint Corrugated super slide board.Feb 16 2021, 4:28 AM
Eileenmcnaughton added a subscriber: hashar.EditedFeb 16 2021, 5:50 AM

I put up https://gerrit.wikimedia.org/r/c/integration/config/+/664359 to remove reference to the drupal modules - that needs releng (@hashar ) to merge it - once merged we need to merge https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/664360 and then we will no longer need the drupal repo (which can then be fully removed from the ci config & 'closed' whatever that means)

I'm going to dig into the other parts of this once that is merged - it's a trivial change but it requires a change outside our control to be co-ordinated with one in our control.

Note this change supercedes my efforts to switch to using submodules. The main reason for that approach was that it would make the co-ordination simpler but it turned out to be fatally flawed

Stashbot added a comment.Feb 16 2021, 7:58 PM

Mentioned in SAL (#wikimedia-releng) [2021-02-16T19:58:22Z] <hashar> Updating Jenkins job wikimedia-fundraising-civicrm-docker to stop cloning the drupal repository # T273822

gerritbot added a comment.Feb 16 2021, 7:58 PM

Change 664359 merged by jenkins-bot:
[integration/config@master] Remove reference to submodule

https://gerrit.wikimedia.org/r/664359

hashar mentioned this in T274941: Archive the wikimedia/fundraising/crm/drupal repository.Feb 16 2021, 8:10 PM
• DStrine moved this task from Current Sprint to Q3 2021-2022 on the Fundraising-Backlog board.Feb 16 2021, 8:25 PM
RhinosF1 subscribed.Feb 16 2021, 8:42 PM
gerritbot added a comment.Feb 16 2021, 9:53 PM

Change 664360 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Convert drupal submodule to being 'just a folder'

https://gerrit.wikimedia.org/r/664360

gerritbot added a comment.Feb 17 2021, 9:34 PM

Change 664929 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] De-symlink sites folder

https://gerrit.wikimedia.org/r/664929

gerritbot added a comment.Feb 17 2021, 10:50 PM

Change 664946 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Commit harvey-dent with correct executable file mode

https://gerrit.wikimedia.org/r/664946

gerritbot added a comment.Feb 17 2021, 11:02 PM

Change 664948 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm/civicrm-buildkit@master] Remove hardver dent chmod in wmff specific

https://gerrit.wikimedia.org/r/664948

gerritbot added a comment.Feb 17 2021, 11:10 PM

Change 664946 merged by Eileen:
[wikimedia/fundraising/crm@master] Commit harvey-dent with correct executable file mode

https://gerrit.wikimedia.org/r/664946

gerritbot added a comment.Feb 17 2021, 11:13 PM

Change 664948 merged by Eileen:
[wikimedia/fundraising/crm/civicrm-buildkit@master] Remove hardver dent chmod in wmff specific

https://gerrit.wikimedia.org/r/664948

gerritbot added a comment.Feb 18 2021, 1:01 AM

Change 664960 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Use full path to drupal in preparation for desymlinking

https://gerrit.wikimedia.org/r/664960

gerritbot added a comment.Feb 18 2021, 1:22 AM

Change 661829 abandoned by Eileen:
[integration/config@master] Switch fundraising tech setup to use recurse_submodules

Reason:
alternate approach used

https://gerrit.wikimedia.org/r/661829

gerritbot added a comment.Feb 24 2021, 4:40 AM

Change 664960 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Use full path to drupal in preparation for desymlinking

https://gerrit.wikimedia.org/r/664960

gerritbot added a comment.Mar 1 2021, 10:59 PM

Change 667727 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] De-symlink sites folder

https://gerrit.wikimedia.org/r/667727

gerritbot added a comment.Mar 1 2021, 11:01 PM

Change 667728 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Update gitignore for sites move

https://gerrit.wikimedia.org/r/667728

gerritbot added a comment.Mar 2 2021, 11:54 PM

Change 667966 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Make path relative

https://gerrit.wikimedia.org/r/667966

gerritbot added a comment.Mar 3 2021, 12:12 AM

Change 667728 abandoned by Eileen:
[wikimedia/fundraising/crm@master] Update gitignore for sites move

Reason:

https://gerrit.wikimedia.org/r/667728

gerritbot added a comment.Mar 3 2021, 12:12 AM

Change 667966 abandoned by Eileen:
[wikimedia/fundraising/crm@master] Make path relative

Reason:

https://gerrit.wikimedia.org/r/667966

gerritbot added a comment.Mar 3 2021, 12:52 AM

Change 667727 merged by Eileen:
[wikimedia/fundraising/crm@master] De-symlink sites folder

https://gerrit.wikimedia.org/r/667727

Eileenmcnaughton added a project: Fundraising Sprint Esperantoland.Mar 4 2021, 4:08 AM
Eileenmcnaughton added a comment.EditedMar 4 2021, 4:12 AM

I just added this into the sprint as we had un-reviewed work from when it was in the sprint that we got merged - some challenges around deployment & we can't deploy anything else unit resolved

Notes of things to do when we push to live

  • (can be done in preparation ) usedrush vget & drush vset to update exchange_rates_google_account_file_path
  • move/copy civicrm.settings.php & settings.php to the new location
  • recreate files symlink
  • drush cc all
  • drush cvapi System.flush
Eileenmcnaughton added a comment.EditedMar 4 2021, 4:36 AM

I wound up getting it going again on staging by

  1. fully removing sites folder
  2. it then worked but later stopped -so as a stop gap I put back a symlink but going the reverse way - this seems OK as a transitional measure to me but I'm not sure how we manage in terms of deployment
  3. another cache clear in the UI
  4. removed a dashlet that seemed broken - I think that might be unrelated

@Ejegg @Dwisehaupt

Eileenmcnaughton added subscribers: Dwisehaupt, Ejegg.Mar 4 2021, 4:36 AM
Eileenmcnaughton claimed this task.Mar 8 2021, 8:35 PM
Eileenmcnaughton moved this task from Backlog to Doing on the Fundraising Sprint Esperantoland board.
gerritbot added a comment.Mar 8 2021, 8:37 PM

Change 669973 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Update gitignore for sites move

https://gerrit.wikimedia.org/r/669973

gerritbot added a comment.Mar 9 2021, 1:14 AM

Change 670018 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Update directory path

https://gerrit.wikimedia.org/r/670018

gerritbot added a comment.Mar 10 2021, 1:13 AM

Change 669973 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Update gitignore for sites move

https://gerrit.wikimedia.org/r/669973

gerritbot added a comment.Mar 10 2021, 1:33 AM

Change 670018 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Update directory path and geocoder

https://gerrit.wikimedia.org/r/670018

gerritbot added a comment.Mar 15 2021, 7:08 PM

Change 672513 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Move CiviCRM folder to the 'real' location

https://gerrit.wikimedia.org/r/672513

Eileenmcnaughton moved this task from Doing to Review on the Fundraising Sprint Esperantoland board.Mar 15 2021, 7:57 PM
gerritbot added a comment.Mar 17 2021, 1:09 AM

Change 664929 abandoned by Eileen:
[wikimedia/fundraising/crm@master] De-symlink sites folder

Reason:

https://gerrit.wikimedia.org/r/664929

gerritbot added a comment.Mar 18 2021, 3:59 AM

Change 673177 had a related patch set uploaded (by Cstone; owner: Cstone):
[mediawiki/vagrant@master] Update paths for symlink removal.

https://gerrit.wikimedia.org/r/673177

gerritbot added a comment.Mar 18 2021, 10:37 PM

Change 673177 merged by jenkins-bot:
[mediawiki/vagrant@master] Update paths for symlink removal.

https://gerrit.wikimedia.org/r/673177

gerritbot added a comment.May 18 2021, 1:40 AM

Change 672513 abandoned by Eileen:

[wikimedia/fundraising/crm@master] Move CiviCRM folder to the 'real' location

Reason:

Need to re-do 'at the time'

https://gerrit.wikimedia.org/r/672513

Eileenmcnaughton moved this task from Q3 2021-2022 to Triage on the Fundraising-Backlog board.Mar 28 2022, 8:47 PM
Eileenmcnaughton added a comment.Mar 28 2022, 8:50 PM

I've brought this back to triage as getting this sorted will remove a blocker on drupal work (& save us work now)

We pushed it out because we got too close to big English

• DStrine moved this task from Triage to Sprint +3 on the Fundraising-Backlog board.Mar 29 2022, 8:08 PM
XenoRyet moved this task from Sprint +3 to Sprint +1 on the Fundraising-Backlog board.Jun 7 2022, 8:07 PM
XenoRyet added a project: Fundraising Sprint Localhost Hospitality Studies.Jun 21 2022, 8:25 PM
XenoRyet removed a project: Fundraising Sprint Localhost Hospitality Studies.Jun 21 2022, 8:30 PM
Ejegg closed this task as Resolved.Aug 24 2022, 10:52 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL