[mwcli mwdd] DBReadOnlyError running update.php in mw-cli docker (due to sqlite file permissions)
Closed, ResolvedPublicBUG REPORT
Actions

Assigned To

Authored By

	jeena
	Jul 29 2021, 7:42 AM

Description

List of steps to reproduce (step by step, including full links if applicable):

mw docker mediawiki create
mw docker mediawiki install
mw docker mediawiki exec bash
cd maintenance
php update.php

What happens?:

...Update 'populate ip_changes' already logged as completed. Use --force to run it again.
...externallinks table indexes up to date
Purging caches...Wikimedia\Rdbms\DBReadOnlyError from line 1082 of /var/www/html/w/includes/libs/rdbms/database/Database.php: Database is read-only: The primary database server is running in read-only mode.
#0 /var/www/html/w/includes/libs/rdbms/database/Database.php(1357): Wikimedia\Rdbms\Database->assertIsWritableMaster()
#1 /var/www/html/w/includes/libs/rdbms/database/Database.php(1308): Wikimedia\Rdbms\Database->executeQuery('DELETE FROM obj...', 'DatabaseUpdater...', 128)
#2 /var/www/html/w/includes/libs/rdbms/database/Database.php(3570): Wikimedia\Rdbms\Database->query('DELETE FROM obj...', 'DatabaseUpdater...', 128)
#3 /var/www/html/w/includes/libs/rdbms/database/DBConnRef.php(68): Wikimedia\Rdbms\Database->delete('objectcache', '*', 'DatabaseUpdater...')
#4 /var/www/html/w/includes/libs/rdbms/database/DBConnRef.php(526): Wikimedia\Rdbms\DBConnRef->__call('delete', Array)
#5 /var/www/html/w/includes/installer/DatabaseUpdater.php(1123): Wikimedia\Rdbms\DBConnRef->delete('objectcache', '*', 'DatabaseUpdater...')
#6 /var/www/html/w/maintenance/update.php(213): DatabaseUpdater->purgeCache()
#7 /var/www/html/w/maintenance/doMaintenance.php(108): UpdateMediaWiki->execute()
#8 /var/www/html/w/maintenance/update.php(264): require_once('/var/www/html/w...')
#9 {main}

What should have happened instead?:
The update script completes without error

Software version (if not a Wikimedia wiki), browser information, screenshots, other information, etc:
mw-cli version info:

GitCommit: 658ba30
GitBranch: master
GitState: dirty - There is an untracked Dockerfile in my mw-cli repo
GitSummary: 658ba30
BuildDate: 2021-07-29T06:24:30Z
Version: v0.1.0

Details

	Title	Reference	Author	Source Branch	Dest Branch
	dev: Make it harder to fall over T287654 (sqlite db issues)	repos/releng/cli!12	addshore	require-user-choice-of-db-type	main

Customize query in GitLab

Related Objects

Mentioned Here: T283797: [mwcli mwdd] When using `mediawiki exec` to run update.php (or other maint scripts), the command is run as the host user, not as "nobody" / the app user
rMTCL658ba3092759: Create lastest.txt in release dir

Event Timeline

jeena created this task.Jul 29 2021, 7:42 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 29 2021, 7:42 AM

Addshore subscribed.Jul 29 2021, 1:00 PM

So it looks like the current expectation is that a user would also pass a --wiki to maintenance scripts that the run within MediaWiki
This may not be apparent for users though and we might want to make things easier / more apparent.
But also as you said on IRC it seems to still happen if you pass a --wiki!

Looking at the code it seems that if you do not pass a --wiki the default is used by default.
https://github.com/addshore/mwcli/blob/dev/static/mwdd/mediawiki/MwddSettings.php#L16-L20
This should be the name of the wiki created by default after using mw docker mediawiki install

I'll take a look!

Purging caches...Wikimedia\Rdbms\DBReadOnlyError from line 1082 of /var/www/html/w/includes/libs/rdbms/database/Database.php: Database is read-only: The primary database server is running in read-only mode.

Oh this seems to be connecting to the right place.
Otherwise it would complain that tables or dbs do not exist.

By default the mw docker mediawiki install command uses sqlite.
So this is probably a file permission error.

The sqlite files are currently stored in $IP . '/data/' . $dockerDb . '.sqlite'
Can you see that file in the container?
And what are the permissions of that file?

Looking at my setup currently these are owned and run by nobody which is the user that the images currently used by mwcli use.

I have no name!@mediawiki:/var/www/html/w$ ls -la ./data
total 980
drwxr-xr-x  3 nobody root      4096 Jul 14 19:51 .
drwxr-xr-x 18   1000    1000   4096 Jul 14 19:49 ..
-rw-r--r--  1 nobody nogroup     14 Jul 14 19:49 .htaccess
-rw-r--r--  1 nobody nogroup 962560 Jul 14 19:51 default.sqlite
-rw-r--r--  1 nobody nogroup  12288 Jul 14 19:49 default_jobqueue.sqlite
-rw-r--r--  1 nobody nogroup  12288 Jul 14 19:49 default_l10n_cache.sqlite
drwxr-xr-x  2 nobody nogroup   4096 Jul 14 19:51 locks
-rw-r--r--  1 nobody nogroup      0 Jul 14 19:49 wikicache.sqlite

So this could be a duplicate of the high prio task T283797: [mwcli mwdd] When using `mediawiki exec` to run update.php (or other maint scripts), the command is run as the host user, not as "nobody" / the app user that needs to be worked on.

I imagine if you run maintenance scripts for SQL users using mw docker mediawiki exec --user=nobody bash for now this would work.

Currently the default user for exec is the system / OS user, this is to enable users to touch files and not end up in a situation where things are owned in bad ways on their local system.
For example phpunit runs, or composer runs, or even maintenance scripts, can all output files
As detailed in T283797 this is partly because composer is bundled in this image / container and users are expected to be able to use it there.

If users were not expected to run composer or other things that write to disk in this container, then the default exec could perhaps just use nobody, which is used for the mw app, and then we wouldnt have this issue?

This is not an issue for mysql wikis as they dont write files on disk.

Addshore renamed this task from DBReadOnlyError running update.php in mw-cli docker to [mwcli mwdd] DBReadOnlyError running update.php in mw-cli docker (due to sqlite file permissions.Jul 29 2021, 1:13 PM

Addshore renamed this task from [mwcli mwdd] DBReadOnlyError running update.php in mw-cli docker (due to sqlite file permissions to [mwcli mwdd] DBReadOnlyError running update.php in mw-cli docker (due to sqlite file permissions).

Addshore triaged this task as High priority.

Addshore merged a task: T283797: [mwcli mwdd] When using `mediawiki exec` to run update.php (or other maint scripts), the command is run as the host user, not as "nobody" / the app user.

Addshore moved this task from Inbox to Priority Backlog on the mwcli board.

Addshore added subscribers: Krinkle, kostajh.

This also happens using mysql

jeena renamed this task from [mwcli mwdd] DBReadOnlyError running update.php in mw-cli docker (due to sqlite file permissions) to [mwcli mwdd] DBReadOnlyError running update.php in mw-cli docker.Jul 29 2021, 4:14 PM

actually that was user error, I needed to pass "--wiki mysqlwiki" to update.php instead of "--wiki default"

It is also an issue for phpunit tests for sqlite though

mw dev mediawiki phpunit tests/phpunit/integration/includes/logging/LogPageTest.php
Using PHP 7.2.31-1+0~20200514.41+debian9~1.gbpe2a56b+wmf1
PHPUnit 8.5.17 by Sebastian Bergmann and contributors.

[7120807117f83275dedab21d] [no req]   Wikimedia\Rdbms\DBReadOnlyError: Database is read-only: The master database server is running in read-only mode.
Backtrace:
from /var/www/html/w/includes/libs/rdbms/database/Database.php(1050)
#0 /var/www/html/w/includes/libs/rdbms/database/Database.php(1328): Wikimedia\Rdbms\Database->assertIsWritableMaster()
#1 /var/www/html/w/includes/libs/rdbms/database/Database.php(1279): Wikimedia\Rdbms\Database->executeQuery(string, string, integer)
#2 /var/www/html/w/includes/libs/rdbms/database/DatabaseSqlite.php(1030): Wikimedia\Rdbms\Database->query(string, string, integer)
#3 /var/www/html/w/includes/db/CloneDatabase.php(108): Wikimedia\Rdbms\DatabaseSqlite->duplicateTableStructure(string, string, boolean, string)
#4 /var/www/html/w/tests/phpunit/MediaWikiIntegrationTestCase.php(1478): CloneDatabase->cloneTableStructure()
#5 /var/www/html/w/tests/phpunit/MediaWikiIntegrationTestCase.php(1546): MediaWikiIntegrationTestCase::setupDatabaseWithTestPrefix(Wikimedia\Rdbms\DatabaseSqlite, string)
#6 /var/www/html/w/tests/phpunit/MediaWikiIntegrationTestCase.php(1499): MediaWikiIntegrationTestCase::setupTestDB(Wikimedia\Rdbms\DatabaseSqlite, string)
#7 /var/www/html/w/tests/phpunit/MediaWikiIntegrationTestCase.php(428): MediaWikiIntegrationTestCase::setupAllTestDBs(Wikimedia\Rdbms\DatabaseSqlite, string, boolean)
#8 /var/www/html/w/vendor/phpunit/phpunit/src/Framework/TestSuite.php(627): MediaWikiIntegrationTestCase->run(PHPUnit\Framework\TestResult)
#9 /var/www/html/w/vendor/phpunit/phpunit/src/TextUI/TestRunner.php(656): PHPUnit\Framework\TestSuite->run(PHPUnit\Framework\TestResult)
#10 /var/www/html/w/vendor/phpunit/phpunit/src/TextUI/Command.php(236): PHPUnit\TextUI\TestRunner->doRun(PHPUnit\Framework\TestSuite, array, array, boolean)
#11 /var/www/html/w/tests/phpunit/phpunit.php(141): PHPUnit\TextUI\Command->run(array, boolean)
#12 /var/www/html/w/tests/phpunit/phpunit.php(204): PHPUnitMaintClass->execute()
#13 {main}

For now at least, a MR making it easier to not fall over this issue is at https://gitlab.wikimedia.org/releng/cli/-/merge_requests/12

So in detail, there is a volume mounted

volumes:
  mediawiki-data:

- "${MEDIAWIKI_VOLUMES_DATA:-mediawiki-data}:/var/www/html/w/data:delegated"

to store the data such as sqlite db in.
This directory is created by the docker mount, and thus afaik owned by both nobody & root?

I have no name!@427ccc030875:/var/www/html/w$ ls -lahr /var/www/html/w/data
total 8.0K
drwxr-xr-x 20   1000 1000 4.0K Oct 16 10:56 ..
drwxr-xr-x  2 nobody root 4.0K Oct 16 10:00 .

The install script is run by nobody and runs just fine mwdev dev mw install --dbtype=sqlite --dbname=sqlitetest1

The resulting files are all owned by nobody!

$ mwdev dev mw exec ls -- -lahr /var/www/html/w/data
total 980K
-rw-r--r--  1 nobody nogroup    0 Oct 16 11:32 wikicache.sqlite
-rw-r--r--  1 nobody nogroup  12K Oct 16 11:32 sqlitetest1_l10n_cache.sqlite
-rw-r--r--  1 nobody nogroup  12K Oct 16 11:32 sqlitetest1_jobqueue.sqlite
-rw-r--r--  1 nobody nogroup 940K Oct 16 11:32 sqlitetest1.sqlite
drwxr-xr-x  2 nobody nogroup 4.0K Oct 16 11:32 locks
-rw-r--r--  1 nobody nogroup   14 Oct 16 11:32 .htaccess
drwxr-xr-x 20   1000    1000 4.0K Oct 16 11:32 ..
drwxr-xr-x  3 nobody root    4.0K Oct 16 11:32 .

But then if users run scripts that need to write to this db, as their own user (the default via exec) it'll break

Such as mwdev dev mw exec php -- ./maintenance/update.php --quick --wiki sqlitetest1

Running this as nobody will actually work just fine

mwdev dev mw exec --user nobody php -- ./maintenance/update.php --quick --wiki sqlitetest1

Now how to make this less crappy....

Addshore claimed this task.Oct 16 2021, 12:15 PM

Addshore moved this task from Priority Backlog to In Progress on the mwcli board.

Restricted Application added a project: User-Addshore. · View Herald TranscriptOct 16 2021, 12:15 PM

Addshore moved this task from In Progress to Pending Release on the mwcli board.Oct 16 2021, 12:16 PM

Addshore closed this task as Resolved.Oct 16 2021, 12:52 PM

Addshore moved this task from Pending Release to Done on the mwcli board.May 25 2022, 9:38 PM

[mwcli mwdd] DBReadOnlyError running update.php in mw-cli docker (due to sqlite file permissions)Closed, ResolvedPublicBUG REPORTActions

Description

Details

Related Objects

Event Timeline

[mwcli mwdd] DBReadOnlyError running update.php in mw-cli docker (due to sqlite file permissions)
Closed, ResolvedPublicBUG REPORT
Actions