Page MenuHomePhabricator
Paste P16814

Wikibase Entity Source Legacy Configuration Migration Script v0.4
ActivePublic

Authored by toan on Jul 12 2021, 2:46 PM.
<?php
namespace Wikibase\Lib\Maintenance;
use Composer\Semver\Constraint\Constraint;
use Composer\Semver\VersionParser;
use Maintenance;
use Wikibase\Client\EntitySourceDefinitionsLegacyClientSettingsParser;
use Wikibase\DataAccess\EntitySource;
use Wikibase\Lib\EntityTypeDefinitions;
use Wikibase\Lib\WikibaseSettings;
use Wikibase\Repo\WikibaseRepo;
$basePath = getenv( 'MW_INSTALL_PATH' );
require_once $basePath . '/maintenance/Maintenance.php';
/**
* Tool for getting new entitySources configuration for upcoming REL1_37 release
*
* Place this script in the extensions/Wikibase folder and executing by running
* php extensions/Wikibase/entitySources.php
* @license GPL-2.0-or-later
*/
class EntitySourcesMigration extends Maintenance {
/**
* @param EntitySource[] $entitySources
*/
function createSettingsFromEntitySources( array $entitySources ): array {
$entitySourceConfig = [];
foreach ( $entitySources as $source ) {
$entityNamespaces = [];
foreach ( $source->getEntityTypes() as $entityType ) {
$entityNamespaces[$entityType] = $source->getEntityNamespaceIds()[$entityType]
. '/' . $source->getEntitySlotNames()[$entityType];
}
$entitySourceConfig[$source->getSourceName()] = [
'entityNamespaces' => $entityNamespaces,
'repoDatabase' => $source->getDatabaseName(),
'baseUri' => $source->getConceptBaseUri(),
'rdfNodeNamespacePrefix' => $source->getRdfNodeNamespacePrefix(),
'rdfPredicateNamespacePrefix' => $source->getRdfPredicateNamespacePrefix(),
'interwikiPrefix' => $source->getInterwikiPrefix(),
];
}
return $entitySourceConfig;
}
private function hasNoEntitySources( $settings ): bool {
return !$settings->hasSetting( 'entitySources' )
|| $settings->hasSetting( 'entitySources' ) && empty( $settings->getSetting( 'entitySources' ) );
}
public function execute() {
$versionParser = new VersionParser();
if ( !$versionParser->parseConstraints( new Constraint( '==', $versionParser->normalize( MW_VERSION ) ) )
->matches( new Constraint( "<", "1.37" ) ) ) {
die( 'This script is intended to run on versions older than 1.37.' );
}
$repoEntitySourceConfig = null;
$clientEntitySourceConfig = null;
$this->output( '// Suggested entitySource configuration to be added in LocalSettings.php:' . "\n\n" );
if ( WikibaseSettings::isRepoEnabled() ) {
$repoSettings = WikibaseSettings::getRepoSettings();
if ( $this->hasNoEntitySources( $repoSettings ) ) {
$repoEntitySourceConfig = $this->createSettingsFromEntitySources(
WikibaseRepo::getDefaultInstance()->getEntitySourceDefinitions()->getSources()
);
$this->output( '$wgWBRepoSettings[\'entitySources\'] = ' );
$this->output( var_export( $repoEntitySourceConfig, true ) );
$this->output( ";\n\n" );
};
}
if ( WikibaseSettings::isClientEnabled() ) {
$clientSettings = WikibaseSettings::getClientSettings();
// REL1_35 and REL1_36 compatible
$entityTypes = require __DIR__ . '/lib/WikibaseLib.entitytypes.php';
$entityTypeDefinitions = new EntityTypeDefinitions( $entityTypes );
if ( $this->hasNoEntitySources( $clientSettings ) ) {
$parser = new EntitySourceDefinitionsLegacyClientSettingsParser();
$sources = $parser->newDefinitionsFromSettings( $clientSettings, $entityTypeDefinitions )->getSources();
$clientEntitySourceConfig = $this->createSettingsFromEntitySources( $sources );
$this->output( '$wgWBClientSettings[\'entitySources\'] = ' );
$this->output( var_export( $clientEntitySourceConfig, true ) );
$this->output( ";\n\n" );
}
}
if ( $clientEntitySourceConfig === null && $repoEntitySourceConfig === null) {
$this->output( "No entitySources suggested as there already exists configuration\n\n" );
}
}
}
$maintClass = EntitySourcesMigration::class;
require_once RUN_MAINTENANCE_IF_MAIN;

Event Timeline

toan updated the paste's language from autodetect to php.Jul 12 2021, 3:01 PM

General comment: I don’t like all the print_r() in here – to me that sounds like a debugging function. We don’t want to print “human-readable information about a variable”, we want to print strings, and the right method for that in a maintenance script (since this is implemented as a maintenance script) would be $this->output(). That’s also the method used in ImportConstraintEntities.php, another maintenance script that prints configuration. (var_export( $value, true ) can be used to get the representation of $value as a string instead of printing it directly.)

L38
				'entityNamespaces' => $source->getEntityNamespaceIds(),

This is missing the entity slot names. Compare the WikibaseClient entitySources default setting in the change for T285471.

L77
			if ( $settings->hasSetting( 'entitySources' ) && !empty( $settings->getSetting( 'entitySources' ) ) ) {

If the client already has entitySources configured, why do we bother printing a config at all in that case?

toan changed the title of this paste from Wikibase Entity Source Lagacy Configuration Migration Script v0.2 to Wikibase Entity Source Lagacy Configuration Migration Script v0.3.

Thanks for the review, updated with the suggestions.

Okay, but this is still strange:

L81
			if ( $settings->hasSetting( 'entitySources' ) && empty( $settings->getSetting( 'entitySources' ) ) ) {

We parse the legacy entity sources if the entitySources setting is present and empty, but not if it’s totally missing? And if entitySources is configured, it’ll now crash if I’m not mistaken (because the rest of the block runs with an unset $sources variable).

I didn’t realize at first where this difference between repo and client came from, but then I looked and apparently pre-service migration WikibaseClient had no getEntitySourceDefinitions() method, just a private $entitySourceDefinitions field. I assume that’s why you did the client part differently… but still, that probably means we should only print new client entitySources if the setting isn’t configured already.

Maybe we can do the same thing in the repo too? We could still use WikibaseRepo::getEntitySourceDefinitions(), but only use it and print anything if entitySources isn’t configured yet. Then repo and client would at least be more similar.

Updated with suggested changes.

We parse the legacy entity sources if the entitySources setting is present and empty, but not if it’s totally missing? And if entitySources is configured, it’ll now crash if I’m not mistaken (because the rest of the block runs with an unset $sources variable).

done

I didn’t realize at first where this difference between repo and client came from, but then I looked and apparently pre-service migration WikibaseClient had no getEntitySourceDefinitions() method, just a private $entitySourceDefinitions field. I assume that’s why you did the client part differently… but still, that probably means we should only print new client entitySources if the setting isn’t configured already.

yes, from REL1_35 -> now this stuff has changed so much and that was the only way to have a common entrypoint

Maybe we can do the same thing in the repo too? We could still use WikibaseRepo::getEntitySourceDefinitions(), but only use it and print anything if entitySources isn’t configured yet. Then repo and client would at least be more similar.

Sounds good, however WikibaseRepo::getEntitySourceDefinitions() only exists in REL1_36+

toan changed the title of this paste from Wikibase Entity Source Lagacy Configuration Migration Script v0.3 to Wikibase Entity Source Lagacy Configuration Migration Script v0.4.Jul 15 2021, 8:02 AM
toan edited the content of this paste. (Show Details)

Alright, I think this looks good to me now. (The second $settings->hasSetting( 'entitySources' ) is redundant but doesn’t hurt.) I can’t think of any other settings that would need to be added – both of the legacy parsers hard-code the 'local' entity source name, so I don’t think we need to configure the repo localEntitySourceName / client itemAndPropertySourceName.

Is there any reason we provide this script in phabricator and not in the code itself?

Well, it uses the Wikibase 1.36 code, so it wouldn’t make sense to ship the script with the 1.37 release. I guess adding it to a 1.36.x release would be possible, but that seems like more effort to me.

Lucas_Werkmeister_WMDE changed the title of this paste from Wikibase Entity Source Lagacy Configuration Migration Script v0.4 to Wikibase Entity Source Legacy Configuration Migration Script v0.4.Aug 26 2021, 10:52 AM

If you name the script config-migrate-1.36.php or equivalent and include instructions, it should be clear when to use it. In my opinion, all code we offer for use should be included in the repo.