Page MenuHomePhabricator

Point version upgrade on civi -do early April in prep for security release mid April -
Closed, ResolvedPublic

Description

Ideally we will upgrade Civi shortly after 1 April & then apply the patch release when it comes out on the first Wed of the month.

Note we will need to disable the core extension sequential credit notes but at this stage I don't think there are any patches we would need to apply on top of the 5.24 beta that comes out of 1 April

Event Timeline

@Eileenmcnaughton On https://download.civicrm.org/latest/, I see the 5.24.1 beta but I also see 5.25 and 5.26. I assume I should just be looking at 5.24.

@Eileen also I see two commits from March 26 and March 31 that I'm not sure are upstreamed:

@mepps so both of those commits can go in favour of getting to a clean install - the first we may need again in another year - but we'll cross that when it comes. The second has an alternate fix in core.

I normally go with the latest beta - the trade off is having our code more closely aligned to master vs the possibility a bug has been introduced to rc that would be picked up by the time it's stable. The latter tends to be pretty rare & is usually not that tough for us to address so I feel like the value lies with being closer to master code - esp as we don't update every month (although we have a 'soft' intention to).

@Eileenmcnaughton thanks! So in this case, that'd be 5.25, right? It's listed as rc.

Yep - the one extra thing we need to do this time is disable the sequentialcreditnotes extension

Yup, I saw that in your beautiful documentation on the chores list :). Thank you!

Change 587340 had a related patch set uploaded (by Mepps; owner: Mepps):
[wikimedia/fundraising/crm/civicrm@master] Stock CiviCRM 5.25 rc

https://gerrit.wikimedia.org/r/587340

@Eileenmcnaughton so I have the patch up, but I assume my next step is testing on staging? I ran the upgrade locally but I couldn't get ui to work so I did it with drush.

@mepps yep - upgrade staging sounds good!

We just got 21:18:10 civicrm is incompatible with the PHP version. on testing - I think we can hack that version - although officially 7.0 is no longer supported I don't think changes have been merged to core yet that won't work on it

@Eileenmcnaughton Notice the patch is also failing CI due to missing extensions. Is that something wrong with how I got the code? What's 21:18:10?

21.18.10 is the timestamp I copied :-)

I'm seeing

21:18:10 /src/wikimedia/fundraising/crm/drupal /src/wikimedia/fundraising/crm/civicrm-buildkit/build/wmff /
21:18:10 ++++ cat sites/default/enabled_modules
21:18:10 +++ drush -y en admin_menu advanced_help adyen_audit amazon_audit astropay_audit banner_history civicrm contribution_tracking ctools environment_indicator exchange_rates ingenico_audit large_donation libraries metrics_reporting oauth_common oauth_common_providerui offline2civicrm orphan_slayer queue2civicrm recurring recurring_globalcollect redis rest_server services services_oauth thank_you wmf_audit wmf_campaigns wmf_civicrm wmf_common wmf_communication wmf_fredge_qc wmf_refund_qc wmf_ct_qc wmf_reports wmf_unsubscribe_qc wmf_optin_qc wmf_eoy_receipt
21:18:10 civicrm is incompatible with the PHP version.

So not sure about the missing extensions?

@Eileenmcnaughton I see this:

17:17:36 WARNING: Failed to find recommended PHP extension "imap".
17:17:36 WARNING: Failed to locate the PHP "mcrypt" extension. Please install this manually to avoid issues with SMTP passwords or older versions of CiviCRM.
17:17:36 WARNING: Failed to find recommended PHP extension "soap".
17:17:37 WARNING: Failed to locate command "node". NodeJS (http://nodejs.org/) is required for development of CiviCRM v4.6+.
17:17:37 WARNING: Failed to locate command "npm". NodeJS (http://nodejs.org/) is required for development of CiviCRM v4.6+.

Hmm but it sould find node and npm at least and would have for earlier versions. It's likely just be related to the php version.

@mepps it should be fine to ignore those - I think I removed mcrypt from recommended in a later version of buildkit. imap & soap we don't need in our dev env (or production I guess). Nodejs might be needed for valid js in a dev env (not needed in production) but it doesn't matter on jenkins

They are required for compiling civi with build kit - but I think they only compile js portions of it

@Eileenmcnaughton I got past the version issue and got a new error!

22:42:04 PHP Warning: require_once(api/v3/utils.php): failed to open stream: No such file or directory in /src/wikimedia/fundraising/crm/civicrm/Civi/Test/Api3TestTrait.php on line 7
22:42:04 PHP Fatal error: require_once(): Failed opening required 'api/v3/utils.php' (include_path='.:/usr/share/php') in /src/wikimedia/fundraising/crm/civicrm/Civi/Test/Api3TestTrait.php on line 7

And on that, I'm going to bed now :).

Change 587426 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Add set_include_path to our base test file.

https://gerrit.wikimedia.org/r/587426

@mepps I think this will address that https://gerrit.wikimedia.org/r/#/c/wikimedia/fundraising/crm/+/587426

As an aside this got merged to 5.25 today (& is ready for 5.24) - it might be worth grabbing the updated tarball in case it saves us some pain - I feel like I've hit the bug it mentions in previous upgrades

I think you meant to include a PR link? But I can pull the latest 5.25.

Change 587426 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Add set_include_path to our base test file.

https://gerrit.wikimedia.org/r/587426

There are now new errors in CI (yay?). It looks like some CRM_Smashpig tests are failing.

We had to add a hack in /api/v3/PaymentProcessor.php _civicrm_payment_processor_pay_spec to unrequire contribution_id.

Down to only two errors/failures! There's one risky test too but I assume that's okay. Now it has to do with trying to delete a contact with a live financial txn.

If I'm reading it right, I need to change the tearDown for that test to delete the contribution then the contact.

I got a little stuck trying to run local tests because the upgrade broke on my new vagrant install due to the config_backend field being dropped.

Hmm - the check looks like it should handle NULL OK

mepps - I think https://gerrit.wikimedia.org/r/588060 will address the test issue - not sure why it didn't fail before this as it was wrong

@Eileenmcnaughton I got the upgrade to work locally by dropping the config_backup column manually before running it.

Also nice! re: test fix but yeah it's weird it worked before..

@mepps I upgraded on staging & it was quick to run the upgrade. Staging seems to be working fine from my click around. Things I can think of

  1. provide some warning to users so they are aware to report anything that changes - we won't need an outage
  2. disable the hidden sequentialcreditnotes extension - could be done on the command line later, disabling it will prevent slowness in any refund jobs we run
  3. check up on recurrings after the update. The history here is that it's not possible to edit Completed recurrings in core via the UI. We used to hack that out but now we have fixed our implementation such that they shouldn't incorrectly show as completed but rather should show as 'In Progress'. I think up til this update our hack has still been present as a backstop on this though. Affects Donor Services

Change 587340 merged by jenkins-bot:
[wikimedia/fundraising/crm/civicrm@master] Stock CiviCRM 5.25 rc with hacks for php version

https://gerrit.wikimedia.org/r/587340