I'm not sure about #1 yet. The configured credentials work with mysql client, so my first thought is that it could be that civi/drupal isn't using SSL, so I need to figure out how we enabled SSL for the production civi instance.

Hopefully #2 is fixed, I changed the git config to by setting "sharedRepository = group" and adjusting permissions for the whole directory tree.

#1 does appear to be the mariadb connection SSL requirement. I suspect it's cropping up here because we didn't bother to require SSL for the db connections on frdev1001, and we've switched to frdb1006. For production we fixed this with T280080. Given the complexity of that task it will take some effort to figure out what's up.

#3 This seems like a filesystem permissions issue, but I am failing to determine which directory it is trying to use. I would think it's /srv/org.wikimedia.civicrm-files but group, owner, and permissions look correct for that directory and its contents. Nothing useful in the logs that I can see.

@Jgreen I was able to make the filesystem issue go away by flushing caches (drupal caches)

@Jgreen one last thing - I realised the code base was on a very old commit (possibly related to investigations on the ssl stuff).

I brougt it up to the latest but hit some lingering permissions issues

git checkout .
error: unable to unlink old 'drupal/sites/default/default.settings.php': Permission denied
error: unable to unlink old 'drupal/sites/default/enabled_modules': Permission denied
error: unable to unlink old 'drupal/sites/default/wmf_settings.json': Permission denied
Updated 1 path from the index
$ git clean -fd
warning: failed to remove drupal/sites/default/civicrm.settings.php-20230710: Permission denied
warning: failed to remove drupal/sites/default/settings.php-20230710: Permission denied

In T341029#9003587, @Eileenmcnaughton wrote:

@Jgreen one last thing - I realised the code base was on a very old commit (possibly related to investigations on the ssl stuff).

Yeah, sorry about that, I left the config files with incorrect permissions when I was editing them. Fixed!

@Jgreen so the database on staging seemed more out of date than it used to be. I've run the upgrades but then I noticed that there are bunch of tables in the staging drupal that are not in live drupal - eg.

Possibly the easiest would be to refresh the database to live. Just drupal won't take long but perhaps we should do both over the weekend

Database restore is in progress. I started it yesterday AM, hopefully it will be done by Monday.