Page MenuHomePhabricator
Create Task
Maniphest T371563

[Spike] Figure out why environment re/installation fails depending on machine or architecture
Closed, ResolvedPublic
Actions

Description

What/Why:
Re/installation of our setup fails too often and randomly. It seems to behave differently depending on machine (or architecture). For example, it doesn't run into issues on Linux whereas it succeeds occasionally (mostly fails) for Macs. This is especially prominent during the step of installing built-in ZObjects. Some of us have had to settle on regularly mooching off of someone else's mediawiki/core/cache/sqlite files to get everything installed.

How:
It seems we don't know why this happens yet. This task is the prerequisite of solving this issue, to find out the why and document findings here. A followup task to implement a solution will follow.

Related Objects
Search...

Event Timeline

ecarg created this task.Jul 31 2024, 9:30 PM
ecarg changed the task status from Open to In Progress.
ecarg added a subtask: T359651: Zobjs fail to build using docker compose exec mediawiki php maintenance/run.php update and npm run local:setup.Jul 31 2024, 9:32 PM
ecarg updated the task description. (Show Details)
ecarg updated the task description. (Show Details)
ecarg added a comment.Jul 31 2024, 10:37 PM
  • Re-installation today was 'successful' but a lot of my MW PHP tests fail; while for others' it does not. Why?
• Sharvaniharan triaged this task as Low priority.Aug 21 2024, 4:42 PM
• Sharvaniharan moved this task from To Triage to Backlog on the Abstract Wikipedia team board.
ecarg added a comment.EditedOct 2 2024, 2:04 AM
[2bfe3607854f531020f2f4ce] [no req]   Error: Class "UIDGenerator" not found
Backtrace:
from /var/www/html/w/extensions/EventBus/includes/EventFactory.php(356)
#0 /var/www/html/w/extensions/EventBus/includes/EventFactory.php(805): MediaWiki\Extension\EventBus\EventFactory->createEvent()
#1 /var/www/html/w/extensions/EventBus/includes/EventBusHooks.php(417): MediaWiki\Extension\EventBus\EventFactory->createRevisionCreateEvent()
#2 /var/www/html/w/extensions/EventBus/includes/EventBusHooks.php(398): MediaWiki\Extension\EventBus\EventBusHooks::sendRevisionCreateEvent()
#3 /var/www/html/w/includes/HookContainer/HookContainer.php(159): MediaWiki\Extension\EventBus\EventBusHooks->onRevisionRecordInserted()
#4 /var/www/html/w/includes/HookContainer/HookRunner.php(3338): MediaWiki\HookContainer\HookContainer->run()
#5 /var/www/html/w/includes/Revision/RevisionStore.php(527): MediaWiki\HookContainer\HookRunner->onRevisionRecordInserted()
#6 /var/www/html/w/includes/Storage/PageUpdater.php(1514): MediaWiki\Revision\RevisionStore->insertRevisionOn()
#7 /var/www/html/w/includes/Storage/PageUpdater.php(934): MediaWiki\Storage\PageUpdater->doCreate()
#8 /var/www/html/w/extensions/WikiLambda/includes/Hooks.php(283): MediaWiki\Storage\PageUpdater->saveRevision()
#9 /var/www/html/w/extensions/WikiLambda/includes/Hooks.php(178): MediaWiki\Extension\WikiLambda\Hooks::insertContentObject()
#10 /var/www/html/w/includes/installer/DatabaseUpdater.php(595): MediaWiki\Extension\WikiLambda\Hooks::createInitialContent()
#11 /var/www/html/w/includes/installer/DatabaseUpdater.php(552): MediaWiki\Installer\DatabaseUpdater->runUpdates()
#12 /var/www/html/w/maintenance/update.php(195): MediaWiki\Installer\DatabaseUpdater->doUpdates()
#13 /var/www/html/w/maintenance/includes/MaintenanceRunner.php(703): UpdateMediaWiki->execute()
#14 /var/www/html/w/maintenance/run.php(51): MediaWiki\Maintenance\MaintenanceRunner->run()

But when I check, I have /EventBus/includes/EventBusFactory.php && /EventBus/includes/EventBusFactory.php

In RELEASE-NOTES-1.43, I see: * UIDGenerator, deprecated since 1.35, has been removed.

[edit] TIL I need to update all of our extensions too on a regular basis; issue resolved

ecarg added a comment.EditedOct 3 2024, 2:56 AM
Wikimedia\Rdbms\DBQueryError from line 1198 of /var/www/html/w/includes/libs/rdbms/database/Database.php: Error 11: database disk image is malformed
Function: WikiPage::pageData
Query: SELECT  page_id,page_namespace,page_title,page_is_redirect,page_is_new,page_random,page_touched,page_links_updated,page_latest,page_len,page_content_model  FROM "page"    WHERE page_namespace = 0 AND page_title = 'Z17'  LIMIT 1  

#0 /var/www/html/w/includes/libs/rdbms/database/Database.php(1182): Wikimedia\Rdbms\Database->getQueryException()
#1 /var/www/html/w/includes/libs/rdbms/database/Database.php(1156): Wikimedia\Rdbms\Database->getQueryExceptionAndLog()
#2 /var/www/html/w/includes/libs/rdbms/database/Database.php(647): Wikimedia\Rdbms\Database->reportQueryError()
#3 /var/www/html/w/includes/libs/rdbms/database/Database.php(1345): Wikimedia\Rdbms\Database->query()
#4 /var/www/html/w/includes/libs/rdbms/database/Database.php(1354): Wikimedia\Rdbms\Database->select()
#5 /var/www/html/w/includes/libs/rdbms/database/DBConnRef.php(127): Wikimedia\Rdbms\Database->selectRow()
#6 /var/www/html/w/includes/libs/rdbms/database/DBConnRef.php(369): Wikimedia\Rdbms\DBConnRef->__call()
#7 /var/www/html/w/includes/libs/rdbms/querybuilder/SelectQueryBuilder.php(793): Wikimedia\Rdbms\DBConnRef->selectRow()
#8 /var/www/html/w/includes/page/WikiPage.php(361): Wikimedia\Rdbms\SelectQueryBuilder->fetchRow()
#9 /var/www/html/w/includes/page/WikiPage.php(388): WikiPage->pageData()
#10 /var/www/html/w/includes/page/WikiPage.php(432): WikiPage->pageDataFromTitle()
#11 /var/www/html/w/includes/page/WikiPage.php(691): WikiPage->loadPageData()
#12 /var/www/html/w/includes/page/WikiPage.php(1200): WikiPage->getLatest()
#13 /var/www/html/w/extensions/WikiLambda/includes/Hooks.php(257): WikiPage->doPurge()
#14 /var/www/html/w/extensions/WikiLambda/includes/Hooks.php(236): MediaWiki\Extension\WikiLambda\Hooks::insertContentObject()
#15 /var/www/html/w/extensions/WikiLambda/includes/Hooks.php(178): MediaWiki\Extension\WikiLambda\Hooks::insertContentObject()
#16 /var/www/html/w/includes/installer/DatabaseUpdater.php(595): MediaWiki\Extension\WikiLambda\Hooks::createInitialContent()
#17 /var/www/html/w/includes/installer/DatabaseUpdater.php(552): MediaWiki\Installer\DatabaseUpdater->runUpdates()
#18 /var/www/html/w/maintenance/update.php(195): MediaWiki\Installer\DatabaseUpdater->doUpdates()
#19 /var/www/html/w/maintenance/includes/MaintenanceRunner.php(703): UpdateMediaWiki->execute()
#20 /var/www/html/w/maintenance/run.php(51): MediaWiki\Maintenance\MaintenanceRunner->run()
#21 {main}
  • this occurs routinely for all of us at various times
  • lazy workaround that doesn't always work is to just dump and reinstall over and over

We need to find out why/how/where this DB error keeps happening

ecarg added a subscriber: gengh.Oct 11 2024, 10:05 PM

As per @gengh's suggestion on using Maria DB, I believe I got all the ZObjects installed in one go! I want to believe this is one solution around the 'database disk image malformed' error, but today is the first time I tried this so I cannot account or foresee what other things may occur down the road using this path. Good news nevertheless:D

10.11.24 install w:mariadb.rtf43 KBDownload
upload of the installation result

gengh added subscribers: vaughnwalters, DVrandecic.Oct 12 2024, 9:54 AM

Great news!
I've been seeing some sqlite incident reports of db corruption happening when inserting objects at a very high rate (e.g. https://sqlite.org/forum/forumpost/705955d7110e89a5)
In our case, during a blank installation of WikiLambda, there's an insertion of 1236 objects, with secondary db updates for each one of them. Running these inserts with sqlite & macOS seems to end up in a corrupted database quite often.

@vaughnwalters @DVrandecic
You both faced this same issue, right? Would you be open to try out the MariaDB installation?

I'm working on a less demanding insertion script that can be configured to add some sleep time (ms) between insertions. I'm wondering if this might also help.

Jdforrester-WMF moved this task from Backlog to Engineering Backlog on the Abstract Wikipedia team board.Nov 12 2024, 4:57 PM
ecarg closed this task as Resolved.Jun 20 2025, 2:29 AM

Closing this now that we have been using MariaDB successfully, thanks to Geno

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits