### Product Version
MediaWiki 1.34.4 (e34e7f2)
PHP 7.2.33-1+0~20201008.48+debian9~1.gbpcb3068 (apache2handler)
MariaDB 10.1.47-MariaDB-0+deb9u1
ICU 65.1
Elasticsearch 6.8.12
### Entry point URLs
Entry point URL
Article path /wiki/en/$1
Script path /wiki/en
index.php /wiki/en/index.php
api.php /wiki/en/api.php
load.php /wiki/en/load.php
---
### Brief
I upgraded 11 (public) wikis in a farm from REL1_32 to REL1_34 and encountered multiple failures. In the end, after seeming to succeed, we found that access to over 1,000 images was 'lost'. Only after manipulating the database records for users, revisions, and images were we able to restore access to the images (but there is a new problem with the `maintenance/purgeList.php` script not working.) See below for details and attempted solutions (with code).
### Steps to Reproduce:
I tried to run the `maintenance/migrateActors.php` script beforehand (using **SCHEMA_COMPAT_NEW** in LocalSettings.php). For preparation, I ran `maintenance/cleanupUsersWithNoId.php --force --prefix=mw`
**migrateActors** would **fail** with
```
Query: INSERT INTO `actor` (actor_name) VALUES ('')
Function: MigrateActors::addActorsForRows
Error: 1062 Duplicate entry '' for key 'actor_name' (localhost)
```
I solved this (the wrong way) by removing the record in `actor`:
```
mysql wiki_fr -e 'DELETE FROM wiki_fr.actor WHERE CAST(actor_name AS CHAR) = "";'
```
But it did allow **actorMigration** to complete. For example:
```
Completed migration, updated 21262 row(s) with 1 new actor(s), 0 error(s)
Beginning migration of log_search
Completed migration, updated 0 row(s) with 0 new actor(s), 0 error(s)
real 2m48.582s
user 0m4.468s
sys 0m0.808s
```
But we also ran into the problem of **Duplicate entry for actor_name** for 'Redirect fixer' (as compared to '')
```
Wikimedia\Rdbms\DBQueryError from line 1603 of /opt/htdocs/mediawiki/includes/libs/rdbms/database/Database.php: A database query error has occurred. Did you forget to run your application's database schema updater after upgrading?
Query: INSERT INTO `actor` (actor_name) VALUES ('Redirect fixer')
Function: MigrateActors::addActorsForRows
Error: 1062 Duplicate entry 'Redirect fixer' for key 'actor_name' (localhost)
```
I tried fixing this (incorrectly) by changing rev_user to 0 in `revision` where rev_user was 7425 ('Redirect fixer); then running `maintenance/cleanupUserWithNoId.php` and `maintenance/migrateActors.php`. Not only is this approach incorrect (because we don't have a user_id 0; and rev_user 7425 was not the problem), but it still failed on similar records in the logging table.
```
Completed migration, updated 81604 row(s) with 0 new actor(s), 0 error(s)
Beginning migration of logging.log_user and logging.log_user_text to logging.log_actor
... log_id=100
... log_id=200
... log_id=300
Wikimedia\Rdbms\DBQueryError from line 1603 of /opt/htdocs/mediawiki/includes/libs/rdbms/database/Database.php: A database query error has occurred. Did you forget to run your application's database schema updater after upgrading?
Query: INSERT INTO `actor` (actor_name) VALUES ('')
Function: MigrateActors::addActorsForRows
Error: 1062 Duplicate entry '' for key 'actor_name' (localhost)
```
We looked at `archive`, `logging`, `image`, `oldimage`, `filearchive`, and `ipblocks`
for records with empty user_text. The only ones found were in `logging` (and they were old records.) These records had various 'logging_user' values which I believe were non-existent in the `user` table. Not knowing how to 'fix' these records, we deleted them (-- affected rows).
```
delete from `wiki_de`.`logging` where log_user_text = ""; -- 349
delete from `wiki_en`.`logging` where log_user_text = ""; -- 21733
delete from `wiki_es`.`logging` where log_user_text = ""; -- 22853
delete from `wiki_fr`.`logging` where log_user_text = ""; -- 332
delete from `wiki_it`.`logging` where log_user_text = ""; -- 0
delete from `wiki_ja`.`logging` where log_user_text = ""; -- 257
delete from `wiki_ko`.`logging` where log_user_text = ""; -- 24
delete from `wiki_pt`.`logging` where log_user_text = ""; -- 535
delete from `wiki_ru`.`logging` where log_user_text = ""; -- 0
delete from `wiki_sv`.`logging` where log_user_text = ""; -- 2979
delete from `wiki_zh`.`logging` where log_user_text = ""; -- 154
```
After all this, we could only get through the `maintenance/update.php` process by deleting the `actor` record for 'Redirect fixer' a few times. (Redirect fixer has a `actor_user` value of NULL)
Not exactly a smooth upgrade experience, but at least now we are running 34.x and everything "looked good".
### But the images
When it was reported that many previous images were failing to appear on the wiki, we determined that it was because the system declines to show them when the proper actor relation does not exist in the image table. Here's an example query that must be satisfied for an image to show on it's `File:` page:
```
SELECT
img_name,
img_size,
img_width,
img_height,
img_metadata,
img_bits,
img_media_type,
img_major_mime,
img_minor_mime,
img_timestamp,
img_sha1,
comment_img_description.comment_text AS `img_description_text`,
comment_img_description.comment_data AS `img_description_data`,
comment_img_description.comment_id AS `img_description_cid`,
actor_img_user.actor_user AS `img_user`,
actor_img_user.actor_name AS `img_user_text`,
img_actor,
img_metadata
FROM
`image`
JOIN
`comment` `comment_img_description` ON ((comment_img_description.comment_id = img_description_id))
JOIN
`actor` `actor_img_user` ON ((actor_img_user.actor_id = img_actor))
WHERE
img_name = 'Flag_of_Norway.png'
LIMIT 1;
```
The problem images had `img_actor` value of 0 in the `image` table. (Zero is a non-existent record ID in the actor table).
We tried to fix all images with this new `[fixImages.php](https://paste.debian.net/1168947/)` maintenance script.
But, that didn't resolve all cases.
```
Fixed 415 out of 1185 images
570 user records missing
770 actor records missing
0 duplicate user records
```
Osnard encountered the same problem with Images and offered a [prepareActorMigration.php](https://github.com/wikimedia/mediawiki-extensions-BlueSpiceFoundation/blob/c7af85ef049e917c0e5bad5f4e7946648cb94486/maintenance/PrepareActorMigration.php) script to fix up "orphan" records (to be used prior to migration). Using his script ([lightly modified]((https://paste.debian.net/1168950/))), even after migration to 1.34, I was able to re-run my `fixImages.php` script to complete fixing up the image table.
```
Fixed 766 out of 766 images
0 user records missing
0 actor records missing
0 duplicate user records
```
At this point it looks like everything is as good as it can be -- although we certainly lost data in the process. I'm reporting here to hopefully help others not make the same mistakes I did; and to possibly get improvements into the migration process or documentation so that upgrades go more smoothly.
### Separate Issue with purgeList.php
(I'll open a separate issue for this -- but it should be noted because otherwise you might think your images are still 'broken'.)
One final sticking point: The images still would not appear on the wiki without a 'purge' of the page cache for each one. Not wanting to purge the entire wiki cache (for performance reasons); nor wanting to click ~2000 URLs, I used the [purgeList.php maintenance script](https://www.mediawiki.org/wiki/Manual:PurgeList.php)
```
WIKI=en php /opt/htdocs/wiki/mediawiki/maintenance/purgeList.php --purge < /opt/htdocs/wiki/mediawiki/maintenance/imagelist.txt
```
but it doesn't actually purge the cache for those files. The output seems to indicate that it was requesting the `?action=history` page instead of `?action=purge`.
```
[snip]
https://www.familysearch.org/wiki/en/File:Ygelli.png
https://www.familysearch.org/wiki/en/index.php?title=File:Ygelli.png&action=history
https://www.familysearch.org/wiki/en/File:Young_gives_old_papers-.png
https://www.familysearch.org/wiki/en/index.php?title=File:Young_gives_old_papers-.png&action=history
Purging 2134 urls
Done!
```