Steps to replicate the issue (include links if applicable):
- 1. I produced an XML dump from my Wikibase instance using [[ https://github.com/mediawiki-client-tools/mediawiki-dump-generator | dumpgenerator ]].
- 2. I followed the directions here (installing Docker, checking out the files, preparing the files, and customizing the environment
- 4. I ran the following Bash script with no issues:
# Load config.ini
source config.ini
cd "$WBDOCKERPATH"
# Run docker-compose to set up Wikibase Suite instance.
docker-compose -f docker-compose.yml -f docker-compose.extra.yml up -d
# Run update and install packages.
winpty docker exec -it wbdocker-wikibase-1 //bin//bash -c "apt-get -y update && apt-get -y install vim && apt-get -y install python3 && apt-get -y install python3-pip"
# Update for pip.
winpty docker exec -it wbdocker-wikibase-1 //bin//bash -c "python3 -m pip config --global set global.break-system-packages true"
# Clone WikibaseQualityConstraints, making sure to use the REL1_41 branch.
if [ ! -d "$WBWRAPPERPATH/WikibaseQualityExtensions" ]; then
git clone -b REL1_41 https://gerrit.wikimedia.org/r/p/mediawiki/extensions/WikibaseQualityConstraints.git "$WBWRAPPERPATH/WikibaseQualityExtensions"
fi
docker cp "$WBWRAPPERPATH/WikibaseQualityExtensions" wbdocker-wikibase-1:/var/www/html/extensions/WikibaseQualityConstraints
# Clone WikibaseLexeme, making sure to use the REL1_41 branch.
if [ ! -d "$WBWRAPPERPATH/WikibaseLexeme" ]; then
git clone -b REL1_41 https://gerrit.wikimedia.org/r/p/mediawiki/extensions/WikibaseLexeme.git "$WBWRAPPERPATH/WikibaseLexeme"
fi
docker cp "$WBWRAPPERPATH/WikibaseLexeme" wbdocker-wikibase-1:/var/www/html/extensions/WikibaseLexeme
# Load WikibaseQualityConstraints.
if winpty docker exec -it wbdocker-wikibase-1 //bin//bash -c "grep -Fxq \"wfLoadExtension( 'WikibaseQualityConstraints' );\" /var/www/html/LocalSettings.php";
then
:
else
winpty docker exec -it wbdocker-wikibase-1 //bin//bash -c "echo \"wfLoadExtension( 'WikibaseQualityConstraints' );\" >> /var/www/html/LocalSettings.php"
fi
# Load WikibaseLexeme.
if winpty docker exec -it wbdocker-wikibase-1 //bin//bash -c "grep -Fxq \"wfLoadExtension( 'WikibaseLexeme' );\" /var/www/html/LocalSettings.php";
then
:
else
winpty docker exec -it wbdocker-wikibase-1 //bin//bash -c "echo \"wfLoadExtension( 'WikibaseLexeme' );\" >> /var/www/html/LocalSettings.php"
fi
# Run update script.
winpty docker exec -it wbdocker-wikibase-1 //bin//bash -c "php /var/www/html/maintenance/update.php"
# Copy over XML dump to upload.
docker cp "$XMLDUMPPATH" wbdocker-wikibase-1:/var/tmp/dump.xml
# Update LocalSettings.php to allow for entity import.
if winpty docker exec -it wbdocker-wikibase-1 //bin//bash -c "grep -Fxq \"\$wgWBRepoSettings['allowEntityImport'] = true;\" /var/www/html/LocalSettings.php";
then
:
else
winpty docker exec -it wbdocker-wikibase-1 //bin//bash -c "echo '$'\"wgWBRepoSettings['allowEntityImport'] = true;\" >> /var/www/html/LocalSettings.php"
fi- 5. Next I manually created a bot account.
- 6. Then I ran this Bash script:
# Load config.ini source config.ini cd "$WBDOCKERPATH" # Create an entity and then delete it using this script. winpty python "$WBWRAPPERPATH/o2wb/create_and_destroy.py" -c "$WBWRAPPERPATH/config.json" # Upload XML dump. winpty docker exec -it wbdocker-wikibase-1 //bin//bash -c "php /var/www/html/maintenance/importDump.php < /var/tmp/dump.xml" # Run rebuild script. winpty docker exec -it wbdocker-wikibase-1 //bin//bash -c "php /var/www/html/maintenance/rebuildall.php" # Run jobs script. winpty docker exec -it wbdocker-wikibase-1 //bin//bash -c "php /var/www/html/maintenance/runJobs.php --memory-limit 512M" # Run site statistics script. winpty docker exec -it wbdocker-wikibase-1 //bin//bash -c "php /var/www/html/maintenance/initSiteStats.php --update" # Download the SQL script to rebuild Wikibase identifiers. # Originates from: https://www.wikibase.consulting/transferring-wikibase-data-between-wikis/ curl https://gist.githubusercontent.com/JeroenDeDauw/c86a5ab7e2771301eb506b246f1af7a6/raw/rebuildWikibaseIdCounters.sql -o rebuildWikibaseIdCounters.sql docker cp "$WBWRAPPERPATH/rebuildWikibaseIdCounters.sql" wbdocker-wikibase-1:/var/www/html/maintenance/rebuildWikibaseIdCounters.sql # Run SQL script to rebuild Wikibase identifiers. winpty docker exec -it wbdocker-wikibase-1 //bin//bash -c "php /var/www/html/maintenance/sql.php /var/www/html/maintenance/rebuildWikibaseIdCounters.sql" # Create an entity and then delete it using this script. winpty python "$WBWRAPPERPATH/o2wb/create_and_destroy.py" -c "$WBWRAPPERPATH/config.json"
The Python script is written as follows:
#!/usr/bin/env python import argparse import json import wikibaseintegrator from wikibaseintegrator import WikibaseIntegrator from wikibaseintegrator.wbi_config import config as wbi_config from wikibaseintegrator import wbi_login def main(): # Create the parser. parser = argparse.ArgumentParser() # Add argument. parser.add_argument("-c", '--config', type=str, required=True) # Parse argument. args = parser.parse_args() # Read in configuration file. f = open(args.config) data = json.load(f) f.close() # Configure instance. wbi_config['MEDIAWIKI_API_URL'] = data['MEDIAWIKI_API_URL'] wbi_config['SPARQL_ENDPOINT_URL'] = data['SPARQL_ENDPOINT_URL'] wbi_config['WIKIBASE_URL'] = data['WIKIBASE_URL'] # Log in. login_instance = wbi_login.Login(user=data['USER'], password=data['PASSWORD']) # Instantiate. wbi = WikibaseIntegrator(login=login_instance) # Start new item. entity = wbi.item.new() entity.aliases.set('en', 'Test123') entity.descriptions.set('en', 'A test entity.') # Write new item. entity.write() # Get item QID. entity_qid = str(entity.id) # Get item. entity = wbi.item.get(entity_qid) # Check JSON response. entity_json = entity.get_json() success_1 = False success_2 = False if 'descriptions' in entity_json: if 'en' in entity_json['descriptions']: if 'value' in entity_json['descriptions']['en']: if entity_json['descriptions']['en']['value'] == 'A test entity.': success_2 = True if 'aliases' in entity_json: if 'en' in entity_json['aliases']: if len(entity_json['aliases']['en']) > 0: if 'value' in entity_json['aliases']['en'][0]: if entity_json['aliases']['en'][0]['value'] == 'Test123': success_1 = True if success_1 and success_2: print('Entity successfully created.') else: print('Entity not found. Creation failed.') # Delete item. entity.delete() try: entity = wbi.item.get(entity_qid) print("Entity deletion unsuccessful. Something went wrong. Exiting...") exit() except wikibaseintegrator.wbi_exceptions.MissingEntityException: print("Entity deletion successful. Exiting...") exit() if __name__=="__main__": main()
What happens?:
The Python script succeeds in creating and destroying an item at the beginning of the Bash script. It also seems to succeed if it creates and destroys an item if run before runJobs.php in the second script. It fails if run after runJobs.php; it fails if run after initSiteStats.php, and it fails if run after sql.php.
In all circumstances of failure, the Python script fails with the same error: wikibaseintegrator.wbi_exceptions.MWApiError: 'The save has failed.'
In all circumstances of failure, when attempting to create a test item manually, the GUI provides the error Could not create a new page. It already exists. despite manual checks, the page does not exist.
If the Python script is removed entirely and I try to create a new item manually after the import, I also get the error Could not create a new page. It already exists..
What should have happened instead?:
I should be able to add new items, using a bot and manually, after importing an XML dump.
Software version (on Special:Version page; skip for WMF-hosted wikis like Wikipedia):
MediaWiki 1.41.1
PHP 8.1.28 (apache2handler)
ICU 72.1
MariaDB 10.11.8-MariaDB-ubu2204
Elasticsearch 7.10.2
Pygments 2.16.1
Other information (browser name/version, screenshots, etc.):
Tested on Chrome, Firefox, Internet Explorer, and Edge. Same results in GUI.
I posted about this issue in the Wikibase community Telegram chat yesterday as well (20 Jun. 2024).