Page MenuHomePhabricator

migrateUserGroup.php should not process all user records
Open, Needs TriagePublic

Description

Waiting for the script to go through 7 million Commons users just to migrate 70 of them was not fun. Special:ListUsers/<group> exists so there must be an efficient way to iterate users in a given group.

Event Timeline

Tgr created this task.Feb 7 2019, 12:57 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 7 2019, 12:57 AM
Reedy added a subscriber: Reedy.EditedFeb 7 2019, 1:13 AM

It doesn't quite do that... It filters by the user group (which is indexed), and then makes some batches of the user ids, which isn't guaranteed to have any users in that batch...

		$start = $dbw->selectField( 'user_groups', 'MIN(ug_user)',
			[ 'ug_group' => $oldGroup ], __FUNCTION__ );
		$end = $dbw->selectField( 'user_groups', 'MAX(ug_user)',
			[ 'ug_group' => $oldGroup ], __FUNCTION__ );

and

			$dbw->update( 'user_groups',
				[ 'ug_group' => $newGroup ],
				[ 'ug_group' => $oldGroup,
					"ug_user BETWEEN " . (int)$blockStart . " AND " . (int)$blockEnd ],
				__METHOD__,
				[ 'IGNORE' ]
			);

It's definitely not sensible. I guess it should just read $batchSize filtered by group, and then do the updates for those

Any idea how long it took?

Tgr added a comment.Feb 7 2019, 1:32 AM

Any idea how long it took?

24 minutes.