Page MenuHomePhabricator

Upgrade from 1.16 failed
Closed, InvalidPublic

Description

I had an existing 1.16 installation.

I read the Upgrade guides, checked all the requirements, followed all the instructions.

I went to mydomain.com/mw-config/ and followed all the steps. There were some warning but nothing relevant. It said it was ready to install, all the requirements were met. At the step where it was supposed to perform the database upgrade, I got this error (see screenshot and output below).

This is f***ed up at so many levels.

  1. No error should have occurred in the first place, as all requirement were made and all checks had been succesful. If some requirement were missing, then it should have been detected in the checks.
  1. Additionally, the spinning icon below the "Upgrade existing installation" title remained stuck spinning forever, which is an additional bug. Looks like some client-side script doesn't handle the error correctly. Even though the error was not supposed to happen in the first place, you have to handle it correctly. A spinning icon that keeps spinning forever upon an error is unacceptable.
  1. I should get an understandable error message telling me exactly what wrong and what I need to do to fix it, in a language that I, as the ADMINISTRATOR of the wiki, can understand.
  1. I see that it has something to do with database permissions: access being denied to the mysql user to perform some query.

4.1) At no point did the instructions tell me that I had to set up any particular permissions for the configured database user, or that I needed to set up a different user for the upgrade.

4.2) Why the f*** didn't the script check whether it had the needed permissions before performing the upgrade?? That is a requirement that should have been checked when checking requirements.

Turning off Content Handler DB fields for this part of upgrade.
...have ipb_id field in ipblocks table.
...have ipb_expiry field in ipblocks table.
...already have interwiki table
...indexes seem up to 20031107 standards.
...have rc_type field in recentchanges table.
...index new_name_timestamp already set on recentchanges table.
...have user_real_name field in user table.
...querycache table already exists.
...objectcache table already exists.
...categorylinks table already exists.
...have pagelinks; skipping old links table updates
...il_from OK
...have rc_ip field in recentchanges table.
...index PRIMARY already set on image table.
...have rc_id field in recentchanges table.
...have rc_patrolled field in recentchanges table.
...logging table already exists.
...have user_token field in user table.
...have wl_notificationtimestamp field in watchlist table.
...watchlist talk page rows already present.
...user table does not contain user_emailauthenticationtimestamp field.
...page table already exists.
...have log_params field in logging table.
...logging table has correct log_title encoding.
...have ar_rev_id field in archive table.
...have page_len field in page table.
...revision table does not contain inverse_timestamp field.
...have rev_text_id field in revision table.
...have rev_deleted field in revision table.
...have img_width field in image table.
...have img_metadata field in image table.
...have user_email_token field in user table.
...have ar_text_id field in archive table.
...page_namespace is already a full int (int(11)).
...ar_namespace is already a full int (int(11)).
...rc_namespace is already a full int (int(11)).
...wl_namespace is already a full int (int(11)).
...qc_namespace is already a full int (int(11)).
...log_namespace is already a full int (int(11)).
...have img_media_type field in image table.
...already have pagelinks table.
...image table does not contain img_type field.
...already have unique user_name index.
...user_groups table exists and is in current format.
...have ss_total_pages field in site_stats table.
...user_newtalk table already exists.
...transcache table already exists.
...have iw_trans field in interwiki table.
...wl_notificationtimestamp is already nullable.
...index times already set on logging table.
...have ipb_range_start field in ipblocks table.
...no page_random rows needed to be set
...have user_registration field in user table.
...templatelinks table already exists
...externallinks table already exists.
...job table already exists.
...have ss_images field in site_stats table.
...langlinks table already exists.
...querycache_info table already exists.
...filearchive table already exists.
...have ipb_anon_only field in ipblocks table.
...index rc_ns_usertext already set on recentchanges table.
...index rc_user_text already set on recentchanges table.
...have user_newpass_time field in user table.
...redirect table already exists.
...querycachetwo table already exists.
...have ipb_enable_autoblock field in ipblocks table.
...index pl_namespace on table pagelinks includes field pl_from.
...index tl_namespace on table templatelinks includes field tl_from.
...index il_to on table imagelinks includes field il_from.
...have rc_old_len field in recentchanges table.
...have user_editcount field in user table.
...page_restrictions table already exists.
...have log_id field in logging table.
...have rev_parent_id field in revision table.
...have pr_id field in page_restrictions table.
...have rev_len field in revision table.
...have rc_deleted field in recentchanges table.
...have log_deleted field in logging table.
...have ar_deleted field in archive table.
...have ipb_deleted field in ipblocks table.
...have fa_deleted field in filearchive table.
...have ar_len field in archive table.
...have ipb_block_email field in ipblocks table.
...index cl_sortkey on table categorylinks includes field cl_from.
...have oi_metadata field in oldimage table.
...index usertext_timestamp already set on archive table.
...index img_usertext_timestamp already set on image table.
...index oi_usertext_timestamp already set on oldimage table.
...have ar_page_id field in archive table.
...have img_sha1 field in image table.
...protected_titles table already exists.
...have ipb_by_text field in ipblocks table.
...page_props table already exists.
...updatelog table already exists.
...category table already exists.
Populating category table, printing progress markers. For large databases, you
may want to hit Ctrl-C and do this manually with maintenance/
populateCategory.php.
Category population complete.
Done populating category table.
...have ar_parent_id field in archive table.
...have user_last_timestamp field in user_newtalk table.
Populating rev_parent_id fields, printing progress markers. For large
databases, you may want to hit Ctrl-C and do this manually with
maintenance/populateParentId.php.
Populating rev_parent_id column
...doing rev_id from 1 to 200
...doing rev_id from 201 to 400
...doing rev_id from 401 to 600
...doing rev_id from 601 to 800
...doing rev_id from 801 to 1000
...doing rev_id from 1001 to 1200
...doing rev_id from 1201 to 1400
...doing rev_id from 1401 to 1600
...doing rev_id from 1601 to 1800
...doing rev_id from 1801 to 2000
...doing rev_id from 2001 to 2200
...doing rev_id from 2201 to 2400
...doing rev_id from 2401 to 2600
...doing rev_id from 2601 to 2800
...doing rev_id from 2801 to 3000
...doing rev_id from 3001 to 3200
...doing rev_id from 3201 to 3400
...doing rev_id from 3401 to 3600
...doing rev_id from 3601 to 3800
...doing rev_id from 3801 to 4000
...doing rev_id from 4001 to 4200
...doing rev_id from 4201 to 4400
...doing rev_id from 4401 to 4600
...doing rev_id from 4601 to 4800
...doing rev_id from 4801 to 5000
rev_parent_id population complete ... 0 rows [0 changed]
...protected_titles table has correct pt_title encoding.
...have ss_active_users field in site_stats table.
...ss_active_users user count set...
...have ipb_allow_usertalk field in ipblocks table.
...change_tag table already exists.
...tag_summary table already exists.
...valid_tag table already exists.
...user_properties table already exists.
...log_search table already exists.
...have log_user_text field in logging table.
Populating log_user_text field, printing progress markers. For large
databases, you may want to hit Ctrl-C and do this manually with
maintenance/populateLogUsertext.php.
...doing log_id from 1 to 100
...doing log_id from 101 to 200
...doing log_id from 201 to 300
...doing log_id from 301 to 400
...doing log_id from 401 to 500
...doing log_id from 501 to 600
...doing log_id from 601 to 700
...doing log_id from 701 to 800
...doing log_id from 801 to 900
...doing log_id from 901 to 1000
...doing log_id from 1001 to 1100
...doing log_id from 1101 to 1200
...doing log_id from 1201 to 1300
...doing log_id from 1301 to 1400
...doing log_id from 1401 to 1500
...doing log_id from 1501 to 1600
...doing log_id from 1601 to 1700
...doing log_id from 1701 to 1800
...doing log_id from 1801 to 1900
...doing log_id from 1901 to 2000
...doing log_id from 2001 to 2100
...doing log_id from 2101 to 2200
...doing log_id from 2201 to 2300
...doing log_id from 2301 to 2400
...doing log_id from 2401 to 2500
...doing log_id from 2501 to 2600
...doing log_id from 2601 to 2700
...doing log_id from 2701 to 2800
...doing log_id from 2801 to 2900
...doing log_id from 2901 to 3000
...doing log_id from 3001 to 3100
...doing log_id from 3101 to 3200
...doing log_id from 3201 to 3300
...doing log_id from 3301 to 3400
...doing log_id from 3401 to 3500
...doing log_id from 3501 to 3600
...doing log_id from 3601 to 3700
...doing log_id from 3701 to 3800
...doing log_id from 3801 to 3900
...doing log_id from 3901 to 4000
...doing log_id from 4001 to 4100
...doing log_id from 4101 to 4200
...doing log_id from 4201 to 4300
...doing log_id from 4301 to 4400
...doing log_id from 4401 to 4500
...doing log_id from 4501 to 4600
...doing log_id from 4601 to 4700
...doing log_id from 4701 to 4800
Done populating log_user_text field.
done.
Populating log_search table, printing progress markers. For large
databases, you may want to hit Ctrl-C and do this manually with
maintenance/populateLogSearch.php.
...doing log_id from 1 to 100
...doing log_id from 101 to 200
...doing log_id from 201 to 300
...doing log_id from 301 to 400
...doing log_id from 401 to 500
...doing log_id from 501 to 600
...doing log_id from 601 to 700
...doing log_id from 701 to 800
...doing log_id from 801 to 900
...doing log_id from 901 to 1000
...doing log_id from 1001 to 1100
...doing log_id from 1101 to 1200
...doing log_id from 1201 to 1300
...doing log_id from 1301 to 1400
...doing log_id from 1401 to 1500
...doing log_id from 1501 to 1600
...doing log_id from 1601 to 1700
...doing log_id from 1701 to 1800
...doing log_id from 1801 to 1900
...doing log_id from 1901 to 2000
...doing log_id from 2001 to 2100
...doing log_id from 2101 to 2200
...doing log_id from 2201 to 2300
...doing log_id from 2301 to 2400
...doing log_id from 2401 to 2500
...doing log_id from 2501 to 2600
...doing log_id from 2601 to 2700
...doing log_id from 2701 to 2800
...doing log_id from 2801 to 2900
...doing log_id from 2901 to 3000
...doing log_id from 3001 to 3100
...doing log_id from 3101 to 3200
...doing log_id from 3201 to 3300
...doing log_id from 3301 to 3400
...doing log_id from 3401 to 3500
...doing log_id from 3501 to 3600
...doing log_id from 3601 to 3700
...doing log_id from 3701 to 3800
...doing log_id from 3801 to 3900
...doing log_id from 3901 to 4000
...doing log_id from 4001 to 4100
...doing log_id from 4101 to 4200
...doing log_id from 4201 to 4300
...doing log_id from 4301 to 4400
...doing log_id from 4401 to 4500
...doing log_id from 4501 to 4600
...doing log_id from 4601 to 4700
...doing log_id from 4701 to 4800
Done populating log_search table.
done.
...l10n_cache table already exists.
...index change_tag_rc_tag already set on change_tag table.
...have rd_interwiki field in redirect table.
Converting tc_time from UNIX epoch to MediaWiki timestamp ...
An error occurred:
A database query error has occurred. Did you forget to run your application's database schema updater after upgrading? 
Query: ALTER TABLE `transcache` MODIFY tc_time binary(14)

Function: Wikimedia\Rdbms\Database::sourceFile( /srv/www/xxxxxxxx.com/public_html/xxx-xxxxx/maintenance/archives/patch-tc-timestamp.sql )
Error: 1142 ALTER command denied to user 'xxxx4_xxxrede'@'localhost' for table 'transcache' (localhost)
Purging caches...<!DOCTYPE html>
<html><head><title>Internal error - MediaWiki</title><style>body { font-family: sans-serif; margin: 0; padding: 0.5em 2em; }</style></head><body>
<div class="errorbox mw-content-ltr"><p>[3d217e9d4ecd66fe2e6bcddb] /xxx-xxxxx/mw-config/?page=Upgrade   Wikimedia\Rdbms\DBQueryError from line 1457 of /srv/www/xxxxxxxx.com/public_html/xxx-xxxxx/includes/libs/rdbms/database/Database.php: A database query error has occurred. Did you forget to run your application's database schema updater after upgrading? <br />
Query: DELETE FROM `module_deps`<br />
Function: DatabaseUpdater::purgeCache<br />
Error: 1146 Table 'xxxx4_xxxxxxxxdb.module_deps' doesn't exist (localhost)<br />
</p><p>Backtrace:</p><p>#0 /srv/www/xxxxxxxx.com/public_html/xxx-xxxxx/includes/libs/rdbms/database/Database.php(1427): Wikimedia\Rdbms\Database-&gt;makeQueryException(string, integer, string, string)<br />
#1 /srv/www/xxxxxxxx.com/public_html/xxx-xxxxx/includes/libs/rdbms/database/Database.php(1200): Wikimedia\Rdbms\Database-&gt;reportQueryError(string, integer, string, string, boolean)<br />
#2 /srv/www/xxxxxxxx.com/public_html/xxx-xxxxx/includes/libs/rdbms/database/Database.php(2845): Wikimedia\Rdbms\Database-&gt;query(string, string)<br />
#3 /srv/www/xxxxxxxx.com/public_html/xxx-xxxxx/includes/installer/DatabaseUpdater.php(1022): Wikimedia\Rdbms\Database-&gt;delete(string, string, string)<br />
#4 /srv/www/xxxxxxxx.com/public_html/xxx-xxxxx/includes/installer/DatabaseInstaller.php(394): DatabaseUpdater-&gt;purgeCache()<br />
#5 /srv/www/xxxxxxxx.com/public_html/xxx-xxxxx/includes/installer/WebInstallerUpgrade.php(65): DatabaseInstaller-&gt;doUpgrade()<br />
#6 /srv/www/xxxxxxxx.com/public_html/xxx-xxxxx/includes/installer/WebInstaller.php(281): WebInstallerUpgrade-&gt;execute()<br />
#7 /srv/www/xxxxxxxx.com/public_html/xxx-xxxxx/mw-config/index.php(79): WebInstaller-&gt;execute(array)<br />
#8 /srv/www/xxxxxxxx.com/public_html/xxx-xxxxx/mw-config/index.php(38): wfInstallerMain()<br />
#9 {main}</p></div>
</body></html>

{F27200575}

Event Timeline

Aklapper added a project: MediaWiki-Installer.

php4fan, welcome to Wikimedia Phabricator.
For future reference, see https://www.mediawiki.org/wiki/How_to_report_a_bug after calming down so there is no need for swearing.

I imagine that Converting tc_time from UNIX epoch to MediaWiki timestamp ... is performed instead of showing ...transcache tc_time already converted. as https://www.mediawiki.org/wiki/Manual:Transcache_table#tc_time implies that this it's need from 1.16 on.

And Error: 1146 Table 'xxxx4_xxxxxxxxdb.module_deps' doesn't exist (localhost)<br /> instead of ...module_deps table already exists. because https://www.mediawiki.org/wiki/Manual:Module_deps_table only exists since 1.16.

(And whatever F27200575 is supposed to show, that file has permissions which don't allow others to see it.)

[Technically this is not a bug but a support request].

Just give your db user privs to CREATE and ALTER tables in your wiki database (If you are paranoid its even possible to setup a different db user only for the upgrade script), and rerun the update script. Everything should be fine (its totally ok to run the script multiple times)

Thanks! Closing as per last comment --- see also https://www.mediawiki.org/wiki/Project:Support_desk for support requests.

[If i actually treat this more as a bug report asking for better error handling than a support request]:

  • the default installer would have created a user with the necessary rights. I am of the opinion it is not unreasonable to assume that the amount of rights the db user has stays constant between upgrade (unlike other requirements like php version which changes between releases)

*i agree that the spinning icon failing to handle the exception is subpar. I consider it a very low severity issue, but please file a separate bug for that (one bug report per issue please)

  • there's lots of things in the world that can go wrong. Expecting every possible situation (including rare ones like this) to have an excellent error message is probably unrealistic. I'd further suggest this error message is reasonably understandable as it can only happen if the administrator setup a custom db user or modified the db users privs after the install. That said, better error messages is always better. Please file a separate bug (one issue per ticket) about having update.php more gracefully handle db permission errors.
  • (4.1): instructions cant handle every possible eventuality. Given this doesnt happen in the default install workflow i think this is reasonable.

Thanks! Closing as per last comment --- see also https://www.mediawiki.org/wiki/Project:Support_desk for support requests.

I may have been too quick to judge when i first skimmed it.

I may have been too quick to judge when i first skimmed it.

Yep you have. This was not a support request at all. I found a bunch of things that are seriously wrong with the upgrade script and I reported them. I have already solved my problem (that is, getting my wiki to work again). If I needed help with that, I would have asked in a forum.

(one bug report per issue please)

Feel free to do that yourself, I am too lazy. My report contains all the necessary information, whether you are willing to fix the issues or not is up to you, I felt obliged to let you know of their existence in case you didn't already.

i agree that the spinning icon failing to handle the exception is subpar. I consider it a very low severity issue

Think again.
It probably looks low severity to you because you already know that when the exception happenes, the upgrade process is over, so to you the spinning icon is just an aesthetic thing. You fail to realise that the user doesn't know whether the process is finished or still ongoing.
When you initiate the install/upgrade process, you see a spinning icon that indicates that the upgrade is in progress, and you start seeing output in a textarea. As the process goes on, the icon keeps spinning and more output appears in the text area. Some of the steps may take longer than others, so at any moment you may stop seeing new output added to the textarea and as long as the spinning icon is still spinning, you assume you need to wait. You know that the process is finished (succesfully or with errors) when you get a message that says so, and the icon stops spinning. Neither happened in this case. I seriously doubted whether the upgrade had stopped with the exception, or if it was still going on. I thought I may have to wait for it to finish before I could try to fix the underlying cause and try again.

I'd further suggest this error message is reasonably understandable

Seriously? Let's analyse it.
If we look at it starting at the bottom, which is how I usually start looking at an error, the first thing I see is a stacktrace with html tags in it.
That starts with a query that fails because a table doesn't exist (which isn't actually related with the root cause of the issue). By "zooming out" a little, we see that this stacktrace and the error that originates it, are actually inside a full HTML page whose html code for some reason is spit out (encoded) into the textarea after "Purging caches...". That makes no sense.
But wait a moment, there are more errors before that. So maybe let's start at the top rather than the bottom. Let's look for the first occurrence of the word "error".
That leads to:

An error occurred:
A database query error has occurred. Did you forget to run your application's database schema updater after upgrading?

No I didn't. Actually I am trying to run the database schema updater.
Now I notice the other error, about the ALTER command being denied.

There are at least 3 errors without a clear separation between one another, nor even between them and the sentences describing the steps being performed, and there's a whole HTML document encoded within the output with a badly formatted stack trace inside it.

And you call that "reasonably understandable"?

the default installer would have created a user with the necessary rights. I am of the opinion it is not unreasonable to assume that the amount of rights the db user has stays constant between upgrade (unlike other requirements like php version which changes between releases)

I beg to disagree.
I have been using my wiki (without upgrading it) for literally years. I don't remember how I installed it, so I can't confirm whether you are right that the installer (which would be the one of the older version) actually had created a user with the necessary rights and if those were the same right that are required by this version. Let's assume it's as you say. I'm pretty sure I migrated this wiki from a server to another at some point several years ago, so I probably exported and imported the database, and recreated the user manually, so maybe I simply gave it select/insert/update/delete privileges. I don't think that's an uncommon situation.
I don't see why you wouldn't just check that you have the privileges you need.

Anyway, as you say, you can't foresee every possible thing that may go wrong, but a database query that fails is one of the trivial ones that you do usually consider, so, graciously handling it, even in a generic way (an error message that clearly tells you that the upgrade process has failed, because of a query failing to be executed, with the query and the database error message clearly standing out, and without the spinning icon) is the least one could expect.

Alright, let's reopen. :)

I've split out the stuff i feel are valid bugs to separate tasks