Page MenuHomePhabricator

Cannot install 1.28 alpha with PostgreSQL database
Closed, ResolvedPublic

Description

As of rMW7197f32c82b468d83da0aed239ba44d5435816c3, you cannot install using PostgreSQL, because it automatically wraps the "CREATE DATABASE" command into a transaction block, which PostgreSQL does not support.

Based on comments in includes/db/DatabaseMysql.php, it seems like doQuery is supposed to be a way to bypass this, but I haven't been able to make it work.

Related Objects

View Standalone Graph
This task is connected to more than 200 other tasks. Only direct parents and subtasks are shown here. Use View Standalone Graph to show more of the graph.

Event Timeline

Jjanes created this task.Oct 6 2016, 9:59 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 6 2016, 9:59 PM
Aklapper updated the task description. (Show Details)
Aklapper added a subscriber: aaron.
Paladox added a subscriber: Paladox.Oct 8 2016, 5:46 PM

Change 314861 had a related patch set uploaded (by Paladox):
Fix postgres support

https://gerrit.wikimedia.org/r/314861

Paladox triaged this task as High priority.Oct 15 2016, 5:17 PM

Changing to high, since we carn't have an mw release with a broken postgres, mw 1.28 is expected to be branched soon, This task is a blocker to it.

Please lower it.

@Aklapper and I'm sorry if I should not have touched this but this is broken, as I have been testing with travis ci.

https://travis-ci.org/paladox/mediawiki/jobs/167910508

Aklapper added a comment.EditedOct 15 2016, 10:10 PM

This task is already tagged for the MW-1.28-release tarball. In terms of priorities for the Wikimedia movement, this might not be high priority. In terms of priorities for external users installing MediaWiki on their servers, it might be higher priority.

In general, please file separate tasks (if not yet existing; if existing please always provide links to the existing tasks) about separate issues, explaining how and why it breaks Postgres (preferably by copying the relevant output / error message). See https://mediawiki.org/wiki/How_to_report_a_bug for more info. Thanks!

What about simply changing the $conn->query("createdb") to $conn->doQuery in PostgresInstaller.php? We lose some random sanity checks and profiling, but this is a one-time create database call so those are not really important.

@Turnstep hi, I carn't seem to find $conn->query("createdb") in PostgresInstaller.php.

Replacing $error = $conn->sourceFile( $conn->getSchemaPath() ); with $error = $conn->sourceFile( "$IP/maintenance/postgres/tables.sql" ); gets us a little further.

But fails at

PHP Warning: Invalid error type specified in /var/www/html/phabricator/includes/libs/rdbms/loadbalancer/LoadBalancer.php on line 192
PHP Notice: Uncommitted DB writes (transaction from DatabasePostgres::determineCoreSchema). in /var/www/html/phabricator/includes/libs/rdbms/database/Database.php on line 3415
PHP Notice: DB transaction writes or callbacks still pending (Database::query). in /var/www/html/phabricator/includes/libs/rdbms/database/Database.php on line 3421
PHP Warning: pg_close(): supplied resource is not a valid PostgreSQL link resource in /var/www/html/phabricator/includes/libs/rdbms/database/DatabasePostgres.php on line 188
PHP Warning: pg_close(): supplied resource is not a valid PostgreSQL link resource in /var/www/html/phabricator/includes/libs/rdbms/database/DatabasePostgres.php on line 188

now.

@Turnstep hi, I carn't seem to find $conn->query("createdb") in PostgresInstaller.php.

It's line 487. Sorry, I was paraphrasing, it's not literally 'createdb' but "CREATE DATABASE $safedb"

Oh, thanks.

Paladox added a comment.EditedOct 16 2016, 3:32 PM

@Turnstep hi, doing that, and it still fails.

I get

Original exception: [a532b4e39d6fc1dabc0c2b24] / DBUnexpectedError from line 2787 of /var/www/html/phabricator/includes/libs/rdbms/database/Database.php: DatabasePostgres::reportQueryError: Expected mass rollback of all peer databases (DBO_TRX set).
Backtrace:
#0 /var/www/html/phabricator/includes/libs/rdbms/database/DatabasePostgres.php(242): Database->rollback(string)
#1 /var/www/html/phabricator/includes/libs/rdbms/database/Database.php(895): DatabasePostgres->reportQueryError(string, string, string, string, boolean)
#2 /var/www/html/phabricator/includes/libs/rdbms/database/Database.php(1237): Database->query(string, string)
#3 /var/www/html/phabricator/includes/cache/MessageCache.php(482): Database->select(string, string, array, string)
#4 /var/www/html/phabricator/includes/cache/MessageCache.php(405): MessageCache->loadFromDB(string, NULL)
#5 /var/www/html/phabricator/includes/cache/MessageCache.php(327): MessageCache->loadFromDBWithLock(string, array, NULL)
#6 /var/www/html/phabricator/includes/cache/MessageCache.php(926): MessageCache->load(string)
#7 /var/www/html/phabricator/includes/cache/MessageCache.php(858): MessageCache->getMsgFromNamespace(string, string)
#8 /var/www/html/phabricator/includes/cache/MessageCache.php(828): MessageCache->getMessageForLang(Language, string, boolean, array)
#9 /var/www/html/phabricator/includes/cache/MessageCache.php(768): MessageCache->getMessageFromFallbackChain(Language, string, boolean)
#10 /var/www/html/phabricator/includes/Message.php(1188): MessageCache->get(string, boolean, Language)
#11 /var/www/html/phabricator/includes/Message.php(802): Message->fetchMessage()
#12 /var/www/html/phabricator/includes/Message.php(902): Message->toString()
#13 /var/www/html/phabricator/includes/Title.php(557): Message->text()
#14 /var/www/html/phabricator/includes/MediaWiki.php(116): Title::newMainPage()
#15 /var/www/html/phabricator/includes/MediaWiki.php(136): MediaWiki->parseTitle()
#16 /var/www/html/phabricator/includes/MediaWiki.php(776): MediaWiki->getTitle()
#17 /var/www/html/phabricator/includes/MediaWiki.php(522): MediaWiki->main()
#18 /var/www/html/phabricator/index.php(43): MediaWiki->run()
#19 {main}

Ok, well it was just a quick idea. If I get some time I will test this out but most of my test boxes cannot handle 1.28 at the moment due to the recent PHP requirement.

Paladox added a comment.EditedOct 16 2016, 4:07 PM

@Turnstep oh, thankyou. What about a vm?

I'm going to see if I can get postgres setup in the ci to prevent postgres from failing in the future.

Change 314861 abandoned by Paladox:
Fix postgres support

https://gerrit.wikimedia.org/r/314861

Paladox added a comment.EditedOct 16 2016, 6:16 PM

@Turnstep hi, doing https://gerrit.wikimedia.org/r/316233 gets a little further but then fails with this error

702PHP Fatal error: Call to a member function defaultValue() on a non-object
in /home/travis/build/paladox/mw/includes/installer/PostgresUpdater.php on line 747

703PHP Stack trace:

704PHP 1. {main}() /home/travis/build/paladox/mw/maintenance/update.php:0

705PHP 2. require_once() /home/travis/build/paladox/mw/maintenance/update.php:216

706PHP 3. UpdateMediaWiki->execute() /home/travis/build/paladox/mw/maintenance/doMaintenance.php:110

707PHP 4. DatabaseUpdater->doUpdates() /home/travis/build/paladox/mw/maintenance/update.php:171

708PHP 5. DatabaseUpdater->runUpdates() /home/travis/build/paladox/mw/includes/installer/DatabaseUpdater.php:434

709PHP 6. call_user_func_array:{/home/travis/build/paladox/mw/includes/installer/DatabaseUpdater.php:473}() /home/travis/build/paladox/mw/includes/installer/DatabaseUpdater.php:473

710PHP 7. PostgresUpdater->addSequence() /home/travis/build/paladox/mw/includes/installer/DatabaseUpdater.php:473

711PHP 8. PostgresUpdater->setDefault() /home/travis/build/paladox/mw/includes/installer/PostgresUpdater.php:615

This may be related

501PHP Notice: Uncommitted DB writes (transaction from DatabasePostgres::determineCoreSchema). in /home/travis/build/paladox/mw/includes/libs/rdbms/database/Database.php on line 3415

502PHP Stack trace:

503PHP 1. Database->__destruct() /home/travis/build/paladox/mw/includes/libs/rdbms/database/Database.php:0

504PHP 2. trigger_error() /home/travis/build/paladox/mw/includes/libs/rdbms/database/Database.php:3415

505

506Notice: Uncommitted DB writes (transaction from DatabasePostgres::determineCoreSchema). in /home/travis/build/paladox/mw/includes/libs/rdbms/database/Database.php on line 3415

507

508Call Stack:

509 0.7348 31618456 1. Database->__destruct() /home/travis/build/paladox/mw/includes/libs/rdbms/database/Database.php:0

510 0.7348 31618648 2. trigger_error() /home/travis/build/paladox/mw/includes/libs/rdbms/database/Database.php:3415

511

512PHP Notice: DB transaction writes or callbacks still pending (Database::query). in /home/travis/build/paladox/mw/includes/libs/rdbms/database/Database.php on line 3421

513PHP Stack trace:

514PHP 1. Database->__destruct() /home/travis/build/paladox/mw/includes/libs/rdbms/database/Database.php:0

515PHP 2. trigger_error() /home/travis/build/paladox/mw/includes/libs/rdbms/database/Database.php:3421

516

517Notice: DB transaction writes or callbacks still pending (Database::query). in /home/travis/build/paladox/mw/includes/libs/rdbms/database/Database.php on line 3421

518

519Call Stack:

520 0.7348 31618456 1. Database->__destruct() /home/travis/build/paladox/mw/includes/libs/rdbms/database/Database.php:0

521 0.7351 31619384 2. trigger_error() /home/travis/build/paladox/mw/includes/libs/rdbms/database/Database.php:3421

522

523PHP Warning: pg_close(): 196 is not a valid PostgreSQL link resource in /home/travis/build/paladox/mw/includes/libs/rdbms/database/DatabasePostgres.php on line 188

524PHP Stack trace:

525PHP 1. Database->__destruct() /home/travis/build/paladox/mw/includes/libs/rdbms/database/Database.php:0

526PHP 2. DatabasePostgres->closeConnection() /home/travis/build/paladox/mw/includes/libs/rdbms/database/Database.php:3426

527PHP 3. pg_close() /home/travis/build/paladox/mw/includes/libs/rdbms/database/DatabasePostgres.php:188

528

529Warning: pg_close(): 196 is not a valid PostgreSQL link resource in /home/travis/build/paladox/mw/includes/libs/rdbms/database/DatabasePostgres.php on line 188

530

531Call Stack:

532 0.7370 31618488 1. Database->__destruct() /home/travis/build/paladox/mw/includes/libs/rdbms/database/Database.php:0

533 0.7370 31618624 2. DatabasePostgres->closeConnection() /home/travis/build/paladox/mw/includes/libs/rdbms/database/Database.php:3426

534 0.7370 31618672 3. pg_close() /home/travis/build/paladox/mw/includes/libs/rdbms/database/DatabasePostgres.php:188

535

T66472

@scfc and @saper are listed on https://www.mediawiki.org/wiki/Developers/Maintainers as developers for PostgreSQL support and might be able to help.

aaron added a comment.Oct 17 2016, 3:04 AM

What is the actual trace of the original error?

Paladox added a comment.EditedOct 17 2016, 6:33 AM

@aaron hi, T148332 is the current error

with the patch there applied

I get error

https://phabricator.wikimedia.org/T147599#2720462

But it may be a different error for different people so I am not sure what the next error is really is.

But Postgres worked in mw 1.26 and I think 1.27 so something broke in mw 1.28

Paladox added a comment.EditedOct 17 2016, 3:46 PM

It also started showing this error T148332 between September 1 and September 15.

Change 316399 had a related patch set uploaded (by Aaron Schulz):
Make isTransactableQuery() exclude CREATE/ALTER

https://gerrit.wikimedia.org/r/316399

Change 316396 had a related patch set uploaded (by Paladox):
Revert "Move updater/installer specific methods out of DatabaseBase"

https://gerrit.wikimedia.org/r/316396

Most of the problems are resolved by https://gerrit.wikimedia.org/r/#/c/316399/ now I am getting this problem

https://travis-ci.org/paladox/SemanticMediaWiki/builds/168433397

674Creating sequence logging_log_id_seq

675[16fc3078cb5ab0068dae8e4b] [no req] DBQueryError from line 1040 of /home/travis/build/paladox/mw/includes/libs/rdbms/database/Database.php: A database query error has occurred. Did you forget to run your application's database schema updater after upgrading?

676Query: CREATE SEQUENCE logging_log_id_seq

677Function: Database::query

678Error: 42P07 ERROR: relation "logging_log_id_seq" already exists

679

Change 316396 abandoned by Paladox:
Revert "Move updater/installer specific methods out of DatabaseBase"

Reason:
Fixed in I64ccab7f9b74f60309ba0c9a8ce68337c42ffb0f

https://gerrit.wikimedia.org/r/316396

[offtopic] In reply to T147599#2723869:

For future reference, please use markup (three times ` ) to keep things readable. Thanks!

Change 316399 merged by jenkins-bot:
Postgres installation fixes

https://gerrit.wikimedia.org/r/316399

Paladox added a comment.EditedOct 19 2016, 10:18 AM

@aaron I'm now not sure which commit passed sepetember 1 2016 is causing the new problem. You fixed most problems now, but theres one last problem to fix which is https://travis-ci.org/paladox/SemanticMediaWiki/builds/168880655

using commit rMWc9f1f64cad140571728b395b95cbc8fb43d36670 before rMW7197f32c82b468d83da0aed239ba44d5435816c3 works, see https://travis-ci.org/paladox/SemanticMediaWiki/builds/168868108

But starting with rMW7197f32c82b468d83da0aed239ba44d5435816c3 it fails and the commits above too. So one of those commits introduced https://travis-ci.org/paladox/SemanticMediaWiki/builds/168880655 problem but rMW7197f32c82b468d83da0aed239ba44d5435816c3 started with the errors first but the different errors.

Creating sequence logging_log_id_seq

[e358ae80324c1d41b3b67da2] [no req]   DBQueryError from line 1054 of /home/travis/build/paladox/mw/includes/libs/rdbms/database/Database.php: A database query error has occurred. Did you forget to run your application's database schema updater after upgrading? 

Query: CREATE SEQUENCE logging_log_id_seq

Function: Database::query

Error: 42P07 ERROR:  relation "logging_log_id_seq" already

From rMW7197f32c82b468d83da0aed239ba44d5435816c3 upwards it seems to have broken detection in the sequence so it should be able to detect weather or not to create the sequence but it isen't working but before that commit it does so either that or one of the other commits broke it. @aaron could you have a look at that please.

Change 314861 restored by Paladox:
Fix postgres support

https://gerrit.wikimedia.org/r/314861

Change 314861 abandoned by Paladox:
Fix postgres support

https://gerrit.wikimedia.org/r/314861

Change 314861 restored by Paladox:
Fix postgres support

https://gerrit.wikimedia.org/r/314861

Change 314861 abandoned by Paladox:
Fix postgres support

https://gerrit.wikimedia.org/r/314861

Paladox added a comment.EditedOct 23 2016, 3:49 PM

@scfc or @aaron would you be able to look into this please?

As of now 590aa0fb7f70d59974af8ccf9f3fea4997c647ea I was able to install on PostgreSQL via the commandline installer from scratch. It didn't work ca. two weeks earlier.

Paladox added a comment.EditedOct 24 2016, 12:47 AM

@saper hi running install.php was fixed in rMW66b0ad56df146c74e531c0f89524398ee0ad3e5b but running update.php is broken for Postgres

It seems that it carnt detect if the tables / sequence and anything else exist so it fails.

It calls the query function so maybe something broke in that function for Postgres?

T148628

aaron closed this task as Resolved.Oct 24 2016, 4:02 AM
aaron claimed this task.

Closing, since the installer works. update.php should be separate bug, which I think is already T148628.

Jdforrester-WMF added a subscriber: Jdforrester-WMF.

Migrating from the old tracking task to a tag for PostgreSQL-related tasks.