Page MenuHomePhabricator

Create beta viwiki
Closed, ResolvedPublic

Description

viwiki is one of the Growth-Team target wikis, and we heavily rely on testing on beta (our features interact with wiki content in nontrivial ways that need to be tested wiki by wiki, and on beta they pull content from production to facilitate that). So it would be nice to be able to test viwiki in beta.

Details

Related Gerrit Patches:
operations/mediawiki-config : masterSet GrowthExperiments help panel search API for beta viwiki
operations/mediawiki-config : masterAdd viwiki to interwiki-labs.php
mediawiki/services/restbase/deploy : masterAdd viwiki to beta env
operations/mediawiki-config : masterAdd viwiki to beta

Event Timeline

Tgr created this task.Feb 29 2020, 12:19 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 29 2020, 12:19 AM
Restricted Application added a project: User-Urbanecm. · View Herald TranscriptFeb 29 2020, 11:06 AM

Change 575684 had a related patch set uploaded (by Reedy; owner: Reedy):
[operations/mediawiki-config@master] Add viwiki to beta

https://gerrit.wikimedia.org/r/575684

Reedy added a subscriber: Reedy.Feb 29 2020, 11:38 AM
reedy@deployment-deploy01:/srv/mediawiki-staging$ mwscript extensions/WikimediaMaintenance/addWiki.php --wiki=aawiki vi wikipedia viwiki vi.wikipedia.beta.wmflabs.org
Creating database viwiki for vi.wikipedia (Tiếng Việt)
Initialising tables
Initialising external storage cluster1...
Writing main page to Trang_Chính
Writing sidebar donate link to MediaWiki:Sitesupport-url
indexing namespaces...
	Indexing namespaces...done
content index...
	Fetching Elasticsearch version...6.5.4...ok
	Scanning available plugins...
		analysis-hebrew, analysis-icu, analysis-nori, analysis-smartcn, analysis-stconvert
		analysis-stempel, analysis-ukrainian, experimental-highlighter, extra, extra-analysis-esperanto
		extra-analysis-serbian, extra-analysis-slovak, ltr
	Picking analyzer...default
	Inferring index identifier...viwiki_content_first
	Creating index...ok
		Validating number of shards...ok
		Validating replica range...ok
		Validating shard allocation settings...done
		Validating max shards per node...ok
	Validating analyzers...ok
	Validating mappings...
		Validating mapping...different...corrected
	Validating aliases...
		Validating viwiki_content alias...alias is free...corrected
		Validating viwiki alias...alias not already assigned to this index...corrected
		Updating tracking indexes...done
general index...
	Fetching Elasticsearch version...6.5.4...ok
	Scanning available plugins...
		analysis-hebrew, analysis-icu, analysis-nori, analysis-smartcn, analysis-stconvert
		analysis-stempel, analysis-ukrainian, experimental-highlighter, extra, extra-analysis-esperanto
		extra-analysis-serbian, extra-analysis-slovak, ltr
	Picking analyzer...default
	Inferring index identifier...viwiki_general_first
	Creating index...ok
		Validating number of shards...ok
		Validating replica range...ok
		Validating shard allocation settings...done
		Validating max shards per node...ok
	Validating analyzers...ok
	Validating mappings...
		Validating mapping...different...corrected
	Validating aliases...
		Validating viwiki_general alias...alias is free...corrected
		Validating viwiki alias...alias not already assigned to this index...corrected
		Updating tracking indexes...done
archive index...
	Fetching Elasticsearch version...6.5.4...ok
	Scanning available plugins...
		analysis-hebrew, analysis-icu, analysis-nori, analysis-smartcn, analysis-stconvert
		analysis-stempel, analysis-ukrainian, experimental-highlighter, extra, extra-analysis-esperanto
		extra-analysis-serbian, extra-analysis-slovak, ltr
	Picking analyzer...default
	Inferring index identifier...viwiki_archive_first
	Creating index...ok
		Validating number of shards...ok
		Validating replica range...ok
		Validating shard allocation settings...done
		Validating max shards per node...ok
	Validating analyzers...ok
	Validating mappings...
		Validating mapping...different...corrected
	Validating aliases...
		Validating viwiki_archive alias...alias is free...corrected
		Updating tracking indexes...done
done.
Making sure mwstore://local-multiwrite/local-public exists...making 'mwstore://local-multiwrite/local-public' public...done.
Making sure mwstore://local-multiwrite/local-thumb exists...making 'mwstore://local-multiwrite/local-thumb' public...done.
Making sure mwstore://local-multiwrite/local-transcoded exists...making 'mwstore://local-multiwrite/local-transcoded' public...done.
Making sure mwstore://local-multiwrite/local-temp exists...making 'mwstore://local-multiwrite/local-temp' private...done.
Making sure mwstore://local-multiwrite/local-deleted exists...making 'mwstore://local-multiwrite/local-deleted' private...done.
Making sure mwstore://local-multiwrite/timeline-render exists...making 'mwstore://local-multiwrite/timeline-render' public...done.
Done. sync the config as in https://wikitech.wikimedia.org/wiki/Add_a_wiki#MediaWiki_configuration

Change 575684 merged by jenkins-bot:
[operations/mediawiki-config@master] Add viwiki to beta

https://gerrit.wikimedia.org/r/575684

Mentioned in SAL (#wikimedia-operations) [2020-02-29T12:34:17Z] <reedy@deploy1001> Synchronized dblists/all-labs.dblist: T246511 (duration: 00m 57s)

Mentioned in SAL (#wikimedia-operations) [2020-02-29T12:35:23Z] <reedy@deploy1001> Synchronized wikiversions-labs.json: T246511 (duration: 00m 56s)

Mentioned in SAL (#wikimedia-operations) [2020-02-29T12:37:14Z] <reedy@deploy1001> Synchronized wmf-config/config/viwiki.yaml: T246511 (duration: 00m 56s)

Reedy closed this task as Resolved.Feb 29 2020, 12:39 PM
Reedy claimed this task.

Done

Tgr added a comment.Mar 1 2020, 12:53 AM

Wow that was super fast! Thanks a bunch Reedy.

Tgr added a comment.Mar 1 2020, 12:53 AM

I got

[XlsHW6wQBHcAAHn1aJ0AAABW] /w/index.php?title=MediaWiki:NewcomerTasks.json&action=submit MediaWiki\Storage\BlobAccessException from line 261 of /srv/mediawiki/php-master/includes/Storage/SqlBlobStore.php: Unable to store text to external storage

Backtrace:

#0 /srv/mediawiki/php-master/includes/Revision/RevisionStore.php(948): MediaWiki\Storage\SqlBlobStore->storeBlob(string, array)
#1 /srv/mediawiki/php-master/includes/Revision/RevisionStore.php(685): MediaWiki\Revision\RevisionStore->storeContentBlob(MediaWiki\Revision\SlotRecord, Title, array)
#2 /srv/mediawiki/php-master/includes/Revision/RevisionStore.php(620): MediaWiki\Revision\RevisionStore->insertSlotOn(Wikimedia\Rdbms\DatabaseMysqli, integer, MediaWiki\Revision\SlotRecord, Title, array)
#3 /srv/mediawiki/php-master/includes/Revision/RevisionStore.php(517): MediaWiki\Revision\RevisionStore->insertRevisionInternal(MediaWiki\Revision\MutableRevisionRecord, Wikimedia\Rdbms\DatabaseMysqli, User, CommentStoreComment, Title, integer, integer)
#4 /srv/mediawiki/php-master/includes/libs/rdbms/database/Database.php(4045): MediaWiki\Revision\RevisionStore->MediaWiki\Revision\{closure}(Wikimedia\Rdbms\DatabaseMysqli, string)
#5 /srv/mediawiki/php-master/includes/libs/rdbms/database/DBConnRef.php(68): Wikimedia\Rdbms\Database->doAtomicSection(string, Closure)
#6 /srv/mediawiki/php-master/includes/libs/rdbms/database/DBConnRef.php(650): Wikimedia\Rdbms\DBConnRef->__call(string, array)
#7 /srv/mediawiki/php-master/includes/Revision/RevisionStore.php(519): Wikimedia\Rdbms\DBConnRef->doAtomicSection(string, Closure)
#8 /srv/mediawiki/php-master/includes/Storage/PageUpdater.php(1131): MediaWiki\Revision\RevisionStore->insertRevisionOn(MediaWiki\Revision\MutableRevisionRecord, Wikimedia\Rdbms\DBConnRef)
#9 /srv/mediawiki/php-master/includes/Storage/PageUpdater.php(768): MediaWiki\Storage\PageUpdater->doCreate(CommentStoreComment, User, integer)
#10 /srv/mediawiki/php-master/includes/page/WikiPage.php(1942): MediaWiki\Storage\PageUpdater->saveRevision(CommentStoreComment, integer)
#11 /srv/mediawiki/php-master/includes/EditPage.php(2385): WikiPage->doEditContent(JsonContent, CommentStoreComment, integer, boolean, User, string, array, integer)
#12 /srv/mediawiki/php-master/includes/EditPage.php(1657): EditPage->internalAttemptSave(array, boolean)
#13 /srv/mediawiki/php-master/includes/EditPage.php(704): EditPage->attemptSave(array)
#14 /srv/mediawiki/php-master/includes/actions/EditAction.php(60): EditPage->edit()
#15 /srv/mediawiki/php-master/includes/actions/SubmitAction.php(38): EditAction->show()
#16 /srv/mediawiki/php-master/includes/MediaWiki.php(519): SubmitAction->show()
#17 /srv/mediawiki/php-master/includes/MediaWiki.php(305): MediaWiki->performAction(Article, Title)
#18 /srv/mediawiki/php-master/includes/MediaWiki.php(972): MediaWiki->performRequest()
#19 /srv/mediawiki/php-master/includes/MediaWiki.php(535): MediaWiki->main()
#20 /srv/mediawiki/php-master/index.php(47): MediaWiki->run()
#21 /srv/mediawiki/w/index.php(3): require(string)
#22 {main}

on page save. Can't reproduce though.

Don't we also need some Parsoid/RB changes etc. made?

Change 575762 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[operations/mediawiki-config@master] Set GrowthExperiments help panel search API for beta viwiki

https://gerrit.wikimedia.org/r/575762

Change 575763 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[mediawiki/services/restbase/deploy@master] Add viwiki to beta env

https://gerrit.wikimedia.org/r/575763

reedy@deployment-deploy01:/srv/mediawiki-staging$ mwscript extensions/WikimediaMaintenance/addWiki.php --wiki=aawiki vi wikipedia viwiki vi.wikipedia.beta.wmflabs.org
Creating database viwiki for vi.wikipedia (Tiếng Việt)
Initialising tables
Initialising external storage cluster1...
Writing main page to Trang_Chính
Writing sidebar donate link to MediaWiki:Sitesupport-url
indexing namespaces...
	Indexing namespaces...done
content index...
	Fetching Elasticsearch version...6.5.4...ok
	Scanning available plugins...
		analysis-hebrew, analysis-icu, analysis-nori, analysis-smartcn, analysis-stconvert
		analysis-stempel, analysis-ukrainian, experimental-highlighter, extra, extra-analysis-esperanto
		extra-analysis-serbian, extra-analysis-slovak, ltr
	Picking analyzer...default
	Inferring index identifier...viwiki_content_first
	Creating index...ok
		Validating number of shards...ok
		Validating replica range...ok
		Validating shard allocation settings...done
		Validating max shards per node...ok
	Validating analyzers...ok
	Validating mappings...
		Validating mapping...different...corrected
	Validating aliases...
		Validating viwiki_content alias...alias is free...corrected
		Validating viwiki alias...alias not already assigned to this index...corrected
		Updating tracking indexes...done
general index...
	Fetching Elasticsearch version...6.5.4...ok
	Scanning available plugins...
		analysis-hebrew, analysis-icu, analysis-nori, analysis-smartcn, analysis-stconvert
		analysis-stempel, analysis-ukrainian, experimental-highlighter, extra, extra-analysis-esperanto
		extra-analysis-serbian, extra-analysis-slovak, ltr
	Picking analyzer...default
	Inferring index identifier...viwiki_general_first
	Creating index...ok
		Validating number of shards...ok
		Validating replica range...ok
		Validating shard allocation settings...done
		Validating max shards per node...ok
	Validating analyzers...ok
	Validating mappings...
		Validating mapping...different...corrected
	Validating aliases...
		Validating viwiki_general alias...alias is free...corrected
		Validating viwiki alias...alias not already assigned to this index...corrected
		Updating tracking indexes...done
archive index...
	Fetching Elasticsearch version...6.5.4...ok
	Scanning available plugins...
		analysis-hebrew, analysis-icu, analysis-nori, analysis-smartcn, analysis-stconvert
		analysis-stempel, analysis-ukrainian, experimental-highlighter, extra, extra-analysis-esperanto
		extra-analysis-serbian, extra-analysis-slovak, ltr
	Picking analyzer...default
	Inferring index identifier...viwiki_archive_first
	Creating index...ok
		Validating number of shards...ok
		Validating replica range...ok
		Validating shard allocation settings...done
		Validating max shards per node...ok
	Validating analyzers...ok
	Validating mappings...
		Validating mapping...different...corrected
	Validating aliases...
		Validating viwiki_archive alias...alias is free...corrected
		Updating tracking indexes...done
done.
Making sure mwstore://local-multiwrite/local-public exists...making 'mwstore://local-multiwrite/local-public' public...done.
Making sure mwstore://local-multiwrite/local-thumb exists...making 'mwstore://local-multiwrite/local-thumb' public...done.
Making sure mwstore://local-multiwrite/local-transcoded exists...making 'mwstore://local-multiwrite/local-transcoded' public...done.
Making sure mwstore://local-multiwrite/local-temp exists...making 'mwstore://local-multiwrite/local-temp' private...done.
Making sure mwstore://local-multiwrite/local-deleted exists...making 'mwstore://local-multiwrite/local-deleted' private...done.
Making sure mwstore://local-multiwrite/timeline-render exists...making 'mwstore://local-multiwrite/timeline-render' public...done.
Done. sync the config as in https://wikitech.wikimedia.org/wiki/Add_a_wiki#MediaWiki_configuration

One question: Why --wiki=aawiki?

One question: Why --wiki=aawiki?

MediaWiki maintenance scripts can only be run in the context of a MediaWiki wiki. For addWiki in prod you choose one that is on the correct DB cluster (aawiki is on the default one, s3), on beta it probably doesn't matter which one you choose as there is only one DB cluster. aawiki is alphabetically first on the list IIRC

One question: Why --wiki=aawiki?

MediaWiki maintenance scripts can only be run in the context of a MediaWiki wiki. For addWiki in prod you choose one that is on the correct DB cluster (aawiki is on the default one, s3), on beta it probably doesn't matter which one you choose as there is only one DB cluster. aawiki is alphabetically first on the list IIRC

OT: Actually, it does matter which one is chosen, per what @Ladsgroup told me. He may be able to explain why aawiki is needed here.

One question: Why --wiki=aawiki?

MediaWiki maintenance scripts can only be run in the context of a MediaWiki wiki. For addWiki in prod you choose one that is on the correct DB cluster (aawiki is on the default one, s3), on beta it probably doesn't matter which one you choose as there is only one DB cluster. aawiki is alphabetically first on the list IIRC

OT: Actually, it does matter which one is chosen, per what @Ladsgroup told me. He may be able to explain why aawiki is needed here.

I once caused some issues because of using mediawikiwiki instead of aawiki (some differences in central auth IIRC) while both are on s3, there are lots of (sometimes unintentional) assumptions in addWiki.php about the dummy wiki. I recommend sticking to aawiki just to be safe (rewriting/improving addWiki.php is also a good idea).

For beta cluster, I think it's fine, as long as you know how to fix issues and bring it back, you can experiment IMO.

Change 575786 had a related patch set uploaded (by Reedy; owner: Reedy):
[operations/mediawiki-config@master] Add viwiki to interwiki-labs.php

https://gerrit.wikimedia.org/r/575786

Change 575786 merged by jenkins-bot:
[operations/mediawiki-config@master] Add viwiki to interwiki-labs.php

https://gerrit.wikimedia.org/r/575786

Change 575762 merged by jenkins-bot:
[operations/mediawiki-config@master] Set GrowthExperiments help panel search API for beta viwiki

https://gerrit.wikimedia.org/r/575762