Page MenuHomePhabricator

Running cargoRecreateData.php without --table can leave tables empty or missing rows
Closed, ResolvedPublic

Description

Running cargoRecreateData.php without --table can leave tables empty or missing rows. Since c7f4f3c, $this->handledPageIds is kept for the whole run and never reset between tables, so a page that stores data in multiple tables is only parsed for the first one. Because #cargo_store only saves data for the table currently being recreated, the later tables silently lose that page's rows. On our Wiki, this left two tables completely empty after a full run, running each with --table restored them. This is fixed by resetting $this->handledPageIds to an empty array in recreateAllDataForTable().

diff
--- a/maintenance/cargoRecreateData.php
+++ b/maintenance/cargoRecreateData.php
@@ -161,6 +161,7 @@ class CargoRecreateData extends Maintenance {
 		$templatesForThisTable = array_merge( $templatesThatDeclareThisTable, $templatesThatAttachToThisTable );
 		$wikiPageFactory = MediaWikiServices::getInstance()->getWikiPageFactory();
 
+		$this->handledPageIds = [];
 		foreach ( $templatesForThisTable as $templatePageID ) {
 			$templateTitle = Title::newFromID( $templatePageID );
 			if ( $templateTitle == null ) {

I want to disclose that I did stumble across this bug/fix with the help of AI but I have reviewed it fully and tested myself.

Event Timeline

Change #1349457 had a related patch set uploaded (by Praveen5638; author: Praveen5638):

[mediawiki/extensions/Cargo@master] cargoRecreateData: Reset handledPageIds between tables

https://gerrit.wikimedia.org/r/1349457

Aklapper renamed this task from cargoRecreateData Bug w/ Fix to Running cargoRecreateData.php without --table can leave tables empty or missing rows.Mon, Oct 5, 10:13 AM

@Praveen5638 - if c7f4f3c caused the problem, then probably the best solution is to simply revert it, no? I don't think $handledPageIds has any benefit if it gets reset for every template, so there would be no point keeping that array.

The purpose of c7f4f3c was to save time by avoiding parsing the same page twice, or multiple times - but I think you're right that the parsing needs to happen separately for each template, at least the way the code is structured now. Maybe there's a truly ideal way to fix this problem by not calling those two lines that set the CargoStore::$settings values if we know for sure that every table that holds data for the current page needs to be recreated... but doing that right seems awfully complex. The best solution for now seems to be just a revert.

@Yaron_Koren - Thanks for the feedback! That makes total sense. I have updated Change #1349457 (Patch Set 2) to completely revert commit c7f4f3c as suggested: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Cargo/+/1349457

Change #1349457 merged by jenkins-bot:

[mediawiki/extensions/Cargo@master] cargoRecreateData: Revert "Avoid handling same page more than once"

https://gerrit.wikimedia.org/r/1349457

Fixed by @Praveen5638, who undid my bad commit.