Zobjs fail to build using docker compose exec mediawiki php maintenance/run.php update and npm run local:setup
Open, MediumPublic
Actions

Assigned To

None

Authored By

	vaughnwalters
	Mar 8 2024, 5:23 PM

Description

Given that I have recloned mediawiki as a fresh install
When I install Mediawiki-docker and WikiLambda
Then the Zobjects will not build using docker compose exec mediawiki php maintenance/run.php update or npm run local:setup

Acceptance criteria
Zobjects should build fully when running the docker compose exec mediawiki php maintenance/run.php update command in the docs

docker compose exec mediawiki php maintenance/run.php update	npm run local:setup	no errors in /cache/mw-error.log

Environment: Apple M1 Max, macOS 14.3.1

Related Objects
Search...

		Status	Subtype	Assigned	Task
		In Progress		ecarg	T371563 [Spike] Figure out why environment re/installation fails depending on machine or architecture
		Open		None	T359651 Zobjs fail to build using docker compose exec mediawiki php maintenance/run.php update and npm run local:setup

Event Timeline

vaughnwalters created this task.Mar 8 2024, 5:23 PM

vaughnwalters updated the task description. (Show Details)

vaughnwalters updated the task description. (Show Details)Mar 8 2024, 5:29 PM

vaughnwalters updated the task description. (Show Details)Mar 8 2024, 5:31 PM

vaughnwalters updated the task description. (Show Details)

From my end, since docker compose exec mediawiki php maintenance/run.php update was giving me the following (stopping at random Z objs) several times:

Successfully created Z150.
Wikimedia\Rdbms\DBQueryError from line 1203 of /var/www/html/w/includes/libs/rdbms/database/Database.php: Error 11: database disk image is malformed
Function: ManualLogEntry::insert
Query: INSERT INTO "logging" (log_type,log_action,log_timestamp,log_actor,log_namespace,log_title,log_page,log_params,log_comment_id) VALUES ('create','create','20240308173725',3,0,'Z160',64,'a:1:{s:17:"associated_rev_id";i:60;}',2)

#0 /var/www/html/w/includes/libs/rdbms/database/Database.php(1187): Wikimedia\Rdbms\Database->getQueryException()
#1 /var/www/html/w/includes/libs/rdbms/database/Database.php(1161): Wikimedia\Rdbms\Database->getQueryExceptionAndLog()
#2 /var/www/html/w/includes/libs/rdbms/database/Database.php(652): Wikimedia\Rdbms\Database->reportQueryError()
#3 /var/www/html/w/includes/libs/rdbms/database/Database.php(1481): Wikimedia\Rdbms\Database->query()
#4 /var/www/html/w/includes/libs/rdbms/database/DBConnRef.php(119): Wikimedia\Rdbms\Database->insert()
#5 /var/www/html/w/includes/libs/rdbms/database/DBConnRef.php(407): Wikimedia\Rdbms\DBConnRef->__call()
#6 /var/www/html/w/includes/libs/rdbms/querybuilder/InsertQueryBuilder.php(343): Wikimedia\Rdbms\DBConnRef->insert()
#7 /var/www/html/w/includes/logging/ManualLogEntry.php(333): Wikimedia\Rdbms\InsertQueryBuilder->execute()
#8 /var/www/html/w/includes/Storage/PageUpdater.php(1550): ManualLogEntry->insert()
#9 /var/www/html/w/includes/Storage/PageUpdater.php(925): MediaWiki\Storage\PageUpdater->doCreate()
#10 /var/www/html/w/extensions/WikiLambda/includes/Hooks.php(277): MediaWiki\Storage\PageUpdater->saveRevision()
#11 /var/www/html/w/extensions/WikiLambda/includes/Hooks.php(153): MediaWiki\Extension\WikiLambda\Hooks::insertContentObject()
#12 /var/www/html/w/includes/installer/DatabaseUpdater.php(603): MediaWiki\Extension\WikiLambda\Hooks::createInitialContent()
#13 /var/www/html/w/includes/installer/DatabaseUpdater.php(562): MediaWiki\Installer\DatabaseUpdater->runUpdates()
#14 /var/www/html/w/maintenance/update.php(189): MediaWiki\Installer\DatabaseUpdater->doUpdates()
#15 /var/www/html/w/maintenance/includes/MaintenanceRunner.php(698): UpdateMediaWiki->execute()
#16 /var/www/html/w/maintenance/run.php(51): MediaWiki\Maintenance\MaintenanceRunner->run()
#17 {main}
PHP Notice:  Uncommitted DB writes (transaction from MediaWiki\Storage\PageUpdater::doCreate) in /var/www/html/w/includes/libs/rdbms/database/TransactionManager.php on line 586
PHP Notice:  DB transaction writes or callbacks still pending (WikiPage::insertOn, MediaWiki\Revision\RevisionStore::insertRevisionRowOn, MediaWiki\Storage\SqlBlobStore::storeBlob, MediaWiki\Revision\RevisionStore::insertContentRowOn, MediaWiki\Revision\RevisionStore::insertSlotRowOn, WikiPage::updateRevisionOn, RecentChange::notifyNew) in /var/www/html/w/includes/libs/rdbms/database/Database.php on line 3236

I used docker compose exec mediawiki php extensions/WikiLambda/maintenance/reloadBuiltinData.php that would stop like this:

*******************************************************************************
NOTE: Do not run maintenance scripts directly, use maintenance/run.php instead!
      Running scripts directly has been deprecated in MediaWiki 1.40.
      It may not work for some (or any) scripts in the future.
*******************************************************************************

Updated Z1 
Updated Z2 
Updated Z3 
Updated Z4 
qemu: uncaught target signal 7 (Bus error) - core dumped

several times OR like this:

Updated Z105 
MediaWiki\Revision\RevisionAccessException from line 1526 of /var/www/html/w/includes/Revision/RevisionStore.php: Main slot of revision not found in database. See T212428.
#0 /var/www/html/w/includes/Revision/RevisionStore.php(1385): MediaWiki\Revision\RevisionStore->constructSlotRecords()
#1 /var/www/html/w/includes/Revision/RevisionStore.php(1559): MediaWiki\Revision\RevisionStore->loadSlotRecords()
#2 [internal function]: MediaWiki\Revision\RevisionStore->MediaWiki\Revision\{closure}()
#3 /var/www/html/w/includes/Revision/RevisionSlots.php(170): call_user_func()
#4 /var/www/html/w/includes/Revision/RevisionSlots.php(109): MediaWiki\Revision\RevisionSlots->getSlots()
#5 /var/www/html/w/includes/Revision/RevisionSlots.php(95): MediaWiki\Revision\RevisionSlots->getSlot()
#6 /var/www/html/w/extensions/WikiLambda/includes/ZObjectStore.php(262): MediaWiki\Revision\RevisionSlots->getContent()
#7 /var/www/html/w/extensions/WikiLambda/includes/ZObjectStore.php(317): MediaWiki\Extension\WikiLambda\ZObjectStore->updateZObject()
#8 /var/www/html/w/extensions/WikiLambda/maintenance/reloadBuiltinData.php(148): MediaWiki\Extension\WikiLambda\ZObjectStore->updateZObjectAsSystemUser()
#9 /var/www/html/w/extensions/WikiLambda/maintenance/reloadBuiltinData.php(109): ReloadBuiltinData->reloadBuiltinSafe()
#10 /var/www/html/w/maintenance/includes/MaintenanceRunner.php(698): ReloadBuiltinData->execute()
#11 /var/www/html/w/maintenance/doMaintenance.php(100): MediaWiki\Maintenance\MaintenanceRunner->run()
#12 /var/www/html/w/extensions/WikiLambda/maintenance/reloadBuiltinData.php(260): require_once('...')
#13 {main}

I also get the following sometimes when I run npm run local:setup:

Creating wikilambda_ztester_results table...done.
Creating wikilambda_zlanguages table...done.
...have wlzl_label_primary field in wikilambda_zobject_labels table.
...have wlzl_return_type field in wikilambda_zobject_labels table.
        Successfully created Z1.
        Successfully created Z2.
qemu: uncaught target signal 7 (Bus error) - core dumped

Jdforrester-WMF subscribed.Mar 8 2024, 6:29 PM

vaughnwalters renamed this task from Zobjs intermittently fail to build using docker compose exec mediawiki php maintenance/run.php update to Zobjs fail to build using docker compose exec mediawiki php maintenance/run.php update and npm run local:setup.Mar 8 2024, 8:05 PM

vaughnwalters updated the task description. (Show Details)

@ecarg Your errors look like the result of using an image built for a different architecture. Bus error indicates that something (I guess sqlite?) is trying to access nonexistent memory. Can you check on the various images you have (i.e., the ones that are running core, core-web, etc.) and see if they are correct for your machine's architecture?

• Mcastro triaged this task as Medium priority.Mar 14 2024, 4:37 PM

• Mcastro added a project: Abstract Wikipedia Fix-It tasks.

• Mcastro moved this task from To Triage to Backlog on the Abstract Wikipedia team board.

Update on this. I installed ubuntu on an older macbook and installed mediawiki-docker and WikiLambda and it spun up correctly and created all of the zobjects. So far the problem seems to be isolated to m1 and intel macbooks.

Also, this script from @DVrandecic also created the zobjects correctly on macOS
for i in {1..50}; do docker-compose exec -u root mediawiki php maintenance/run.php update --quick; done

Aha, if this only occurs on Macs with Mx series CPUs, this might be the recurring upstream bug in Docker on macOS on Apple Silicon machines repeatedly enabling the ultra-toxic-will-break-everything preference called 'Use Rosetta for x86_64/amd64 emulation on Apple Silicon'; it should always be switched off for MediaWiki work (or, as far as I can tell, any work at all). Does that fix it?

In T359651#9634839, @Jdforrester-WMF wrote:

Aha, if this only occurs on Macs with Mx series CPUs, this might be the recurring upstream bug in Docker on macOS on Apple Silicon machines repeatedly enabling the ultra-toxic-will-break-everything preference called 'Use Rosetta for x86_64/amd64 emulation on Apple Silicon'; it should always be switched off for MediaWiki work (or, as far as I can tell, any work at all). Does that fix it?

This also occurs on Intel macs (@ecarg is using an intel mac with macOS) so it is not isolated to Mx series, but does seem to be isolated to macOS. I will re run though and make sure the 'Use Rosetta for x86_64/amd64 emulation on Apple Silicon' is off though on my m1 and see if this clears up any errors. (update - it does not)

vaughnwalters updated the task description. (Show Details)Jul 15 2024, 3:27 PM

ecarg added a parent task: T371563: [Spike] Figure out why environment re/installation fails depending on machine or architecture.Jul 31 2024, 9:32 PM

zeljkofilipin added a project: User-zeljkofilipin.Aug 7 2024, 11:19 AM

zeljkofilipin moved this task from Backlog 🪒 to Watching 📺 on the User-zeljkofilipin board.

Jdforrester-WMF moved this task from Backlog to Engineering Backlog on the Abstract Wikipedia team board.Wed, Nov 20, 5:55 PM

zeljkofilipin removed a project: User-zeljkofilipin.Thu, Dec 19, 4:09 PM

	F42473674: Screenshot 2024-03-08 at 1.38.53 PM.png
	Mar 8 2024, 8:05 PM

	F42470582: Screenshot 2024-03-08 at 11.27.29 AM.png
	Mar 8 2024, 5:29 PM

	F42470312: Screenshot 2024-03-07 at 8.09.18 AM.png
	Mar 8 2024, 5:23 PM

	F42470311: Screenshot 2024-03-07 at 8.07.58 AM.png
	Mar 8 2024, 5:23 PM

Zobjs fail to build using docker compose exec mediawiki php maintenance/run.php update and npm run local:setupOpen, MediumPublicActions

Description

Related ObjectsSearch...

Event Timeline

Zobjs fail to build using docker compose exec mediawiki php maintenance/run.php update and npm run local:setup
Open, MediumPublic
Actions

Related Objects
Search...