Page MenuHomePhabricator

Create beta viwiki
Closed, ResolvedPublic

Description

viwiki is one of the Growth-Team target wikis, and we heavily rely on testing on beta (our features interact with wiki content in nontrivial ways that need to be tested wiki by wiki, and on beta they pull content from production to facilitate that). So it would be nice to be able to test viwiki in beta.

Event Timeline

Change 575684 had a related patch set uploaded (by Reedy; owner: Reedy):
[operations/mediawiki-config@master] Add viwiki to beta

https://gerrit.wikimedia.org/r/575684

reedy@deployment-deploy01:/srv/mediawiki-staging$ mwscript extensions/WikimediaMaintenance/addWiki.php --wiki=aawiki vi wikipedia viwiki vi.wikipedia.beta.wmflabs.org
Creating database viwiki for vi.wikipedia (Tiếng Việt)
Initialising tables
Initialising external storage cluster1...
Writing main page to Trang_Chính
Writing sidebar donate link to MediaWiki:Sitesupport-url
indexing namespaces...
	Indexing namespaces...done
content index...
	Fetching Elasticsearch version...6.5.4...ok
	Scanning available plugins...
		analysis-hebrew, analysis-icu, analysis-nori, analysis-smartcn, analysis-stconvert
		analysis-stempel, analysis-ukrainian, experimental-highlighter, extra, extra-analysis-esperanto
		extra-analysis-serbian, extra-analysis-slovak, ltr
	Picking analyzer...default
	Inferring index identifier...viwiki_content_first
	Creating index...ok
		Validating number of shards...ok
		Validating replica range...ok
		Validating shard allocation settings...done
		Validating max shards per node...ok
	Validating analyzers...ok
	Validating mappings...
		Validating mapping...different...corrected
	Validating aliases...
		Validating viwiki_content alias...alias is free...corrected
		Validating viwiki alias...alias not already assigned to this index...corrected
		Updating tracking indexes...done
general index...
	Fetching Elasticsearch version...6.5.4...ok
	Scanning available plugins...
		analysis-hebrew, analysis-icu, analysis-nori, analysis-smartcn, analysis-stconvert
		analysis-stempel, analysis-ukrainian, experimental-highlighter, extra, extra-analysis-esperanto
		extra-analysis-serbian, extra-analysis-slovak, ltr
	Picking analyzer...default
	Inferring index identifier...viwiki_general_first
	Creating index...ok
		Validating number of shards...ok
		Validating replica range...ok
		Validating shard allocation settings...done
		Validating max shards per node...ok
	Validating analyzers...ok
	Validating mappings...
		Validating mapping...different...corrected
	Validating aliases...
		Validating viwiki_general alias...alias is free...corrected
		Validating viwiki alias...alias not already assigned to this index...corrected
		Updating tracking indexes...done
archive index...
	Fetching Elasticsearch version...6.5.4...ok
	Scanning available plugins...
		analysis-hebrew, analysis-icu, analysis-nori, analysis-smartcn, analysis-stconvert
		analysis-stempel, analysis-ukrainian, experimental-highlighter, extra, extra-analysis-esperanto
		extra-analysis-serbian, extra-analysis-slovak, ltr
	Picking analyzer...default
	Inferring index identifier...viwiki_archive_first
	Creating index...ok
		Validating number of shards...ok
		Validating replica range...ok
		Validating shard allocation settings...done
		Validating max shards per node...ok
	Validating analyzers...ok
	Validating mappings...
		Validating mapping...different...corrected
	Validating aliases...
		Validating viwiki_archive alias...alias is free...corrected
		Updating tracking indexes...done
done.
Making sure mwstore://local-multiwrite/local-public exists...making 'mwstore://local-multiwrite/local-public' public...done.
Making sure mwstore://local-multiwrite/local-thumb exists...making 'mwstore://local-multiwrite/local-thumb' public...done.
Making sure mwstore://local-multiwrite/local-transcoded exists...making 'mwstore://local-multiwrite/local-transcoded' public...done.
Making sure mwstore://local-multiwrite/local-temp exists...making 'mwstore://local-multiwrite/local-temp' private...done.
Making sure mwstore://local-multiwrite/local-deleted exists...making 'mwstore://local-multiwrite/local-deleted' private...done.
Making sure mwstore://local-multiwrite/timeline-render exists...making 'mwstore://local-multiwrite/timeline-render' public...done.
Done. sync the config as in https://wikitech.wikimedia.org/wiki/Add_a_wiki#MediaWiki_configuration

Change 575684 merged by jenkins-bot:
[operations/mediawiki-config@master] Add viwiki to beta

https://gerrit.wikimedia.org/r/575684

Mentioned in SAL (#wikimedia-operations) [2020-02-29T12:34:17Z] <reedy@deploy1001> Synchronized dblists/all-labs.dblist: T246511 (duration: 00m 57s)

Mentioned in SAL (#wikimedia-operations) [2020-02-29T12:35:23Z] <reedy@deploy1001> Synchronized wikiversions-labs.json: T246511 (duration: 00m 56s)

Mentioned in SAL (#wikimedia-operations) [2020-02-29T12:37:14Z] <reedy@deploy1001> Synchronized wmf-config/config/viwiki.yaml: T246511 (duration: 00m 56s)

Reedy claimed this task.

Done

Wow that was super fast! Thanks a bunch Reedy.

I got

[XlsHW6wQBHcAAHn1aJ0AAABW] /w/index.php?title=MediaWiki:NewcomerTasks.json&action=submit MediaWiki\Storage\BlobAccessException from line 261 of /srv/mediawiki/php-master/includes/Storage/SqlBlobStore.php: Unable to store text to external storage

Backtrace:

#0 /srv/mediawiki/php-master/includes/Revision/RevisionStore.php(948): MediaWiki\Storage\SqlBlobStore->storeBlob(string, array)
#1 /srv/mediawiki/php-master/includes/Revision/RevisionStore.php(685): MediaWiki\Revision\RevisionStore->storeContentBlob(MediaWiki\Revision\SlotRecord, Title, array)
#2 /srv/mediawiki/php-master/includes/Revision/RevisionStore.php(620): MediaWiki\Revision\RevisionStore->insertSlotOn(Wikimedia\Rdbms\DatabaseMysqli, integer, MediaWiki\Revision\SlotRecord, Title, array)
#3 /srv/mediawiki/php-master/includes/Revision/RevisionStore.php(517): MediaWiki\Revision\RevisionStore->insertRevisionInternal(MediaWiki\Revision\MutableRevisionRecord, Wikimedia\Rdbms\DatabaseMysqli, User, CommentStoreComment, Title, integer, integer)
#4 /srv/mediawiki/php-master/includes/libs/rdbms/database/Database.php(4045): MediaWiki\Revision\RevisionStore->MediaWiki\Revision\{closure}(Wikimedia\Rdbms\DatabaseMysqli, string)
#5 /srv/mediawiki/php-master/includes/libs/rdbms/database/DBConnRef.php(68): Wikimedia\Rdbms\Database->doAtomicSection(string, Closure)
#6 /srv/mediawiki/php-master/includes/libs/rdbms/database/DBConnRef.php(650): Wikimedia\Rdbms\DBConnRef->__call(string, array)
#7 /srv/mediawiki/php-master/includes/Revision/RevisionStore.php(519): Wikimedia\Rdbms\DBConnRef->doAtomicSection(string, Closure)
#8 /srv/mediawiki/php-master/includes/Storage/PageUpdater.php(1131): MediaWiki\Revision\RevisionStore->insertRevisionOn(MediaWiki\Revision\MutableRevisionRecord, Wikimedia\Rdbms\DBConnRef)
#9 /srv/mediawiki/php-master/includes/Storage/PageUpdater.php(768): MediaWiki\Storage\PageUpdater->doCreate(CommentStoreComment, User, integer)
#10 /srv/mediawiki/php-master/includes/page/WikiPage.php(1942): MediaWiki\Storage\PageUpdater->saveRevision(CommentStoreComment, integer)
#11 /srv/mediawiki/php-master/includes/EditPage.php(2385): WikiPage->doEditContent(JsonContent, CommentStoreComment, integer, boolean, User, string, array, integer)
#12 /srv/mediawiki/php-master/includes/EditPage.php(1657): EditPage->internalAttemptSave(array, boolean)
#13 /srv/mediawiki/php-master/includes/EditPage.php(704): EditPage->attemptSave(array)
#14 /srv/mediawiki/php-master/includes/actions/EditAction.php(60): EditPage->edit()
#15 /srv/mediawiki/php-master/includes/actions/SubmitAction.php(38): EditAction->show()
#16 /srv/mediawiki/php-master/includes/MediaWiki.php(519): SubmitAction->show()
#17 /srv/mediawiki/php-master/includes/MediaWiki.php(305): MediaWiki->performAction(Article, Title)
#18 /srv/mediawiki/php-master/includes/MediaWiki.php(972): MediaWiki->performRequest()
#19 /srv/mediawiki/php-master/includes/MediaWiki.php(535): MediaWiki->main()
#20 /srv/mediawiki/php-master/index.php(47): MediaWiki->run()
#21 /srv/mediawiki/w/index.php(3): require(string)
#22 {main}

on page save. Can't reproduce though.

Don't we also need some Parsoid/RB changes etc. made?

Change 575762 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[operations/mediawiki-config@master] Set GrowthExperiments help panel search API for beta viwiki

https://gerrit.wikimedia.org/r/575762

Change 575763 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[mediawiki/services/restbase/deploy@master] Add viwiki to beta env

https://gerrit.wikimedia.org/r/575763

reedy@deployment-deploy01:/srv/mediawiki-staging$ mwscript extensions/WikimediaMaintenance/addWiki.php --wiki=aawiki vi wikipedia viwiki vi.wikipedia.beta.wmflabs.org
Creating database viwiki for vi.wikipedia (Tiếng Việt)
Initialising tables
Initialising external storage cluster1...
Writing main page to Trang_Chính
Writing sidebar donate link to MediaWiki:Sitesupport-url
indexing namespaces...
	Indexing namespaces...done
content index...
	Fetching Elasticsearch version...6.5.4...ok
	Scanning available plugins...
		analysis-hebrew, analysis-icu, analysis-nori, analysis-smartcn, analysis-stconvert
		analysis-stempel, analysis-ukrainian, experimental-highlighter, extra, extra-analysis-esperanto
		extra-analysis-serbian, extra-analysis-slovak, ltr
	Picking analyzer...default
	Inferring index identifier...viwiki_content_first
	Creating index...ok
		Validating number of shards...ok
		Validating replica range...ok
		Validating shard allocation settings...done
		Validating max shards per node...ok
	Validating analyzers...ok
	Validating mappings...
		Validating mapping...different...corrected
	Validating aliases...
		Validating viwiki_content alias...alias is free...corrected
		Validating viwiki alias...alias not already assigned to this index...corrected
		Updating tracking indexes...done
general index...
	Fetching Elasticsearch version...6.5.4...ok
	Scanning available plugins...
		analysis-hebrew, analysis-icu, analysis-nori, analysis-smartcn, analysis-stconvert
		analysis-stempel, analysis-ukrainian, experimental-highlighter, extra, extra-analysis-esperanto
		extra-analysis-serbian, extra-analysis-slovak, ltr
	Picking analyzer...default
	Inferring index identifier...viwiki_general_first
	Creating index...ok
		Validating number of shards...ok
		Validating replica range...ok
		Validating shard allocation settings...done
		Validating max shards per node...ok
	Validating analyzers...ok
	Validating mappings...
		Validating mapping...different...corrected
	Validating aliases...
		Validating viwiki_general alias...alias is free...corrected
		Validating viwiki alias...alias not already assigned to this index...corrected
		Updating tracking indexes...done
archive index...
	Fetching Elasticsearch version...6.5.4...ok
	Scanning available plugins...
		analysis-hebrew, analysis-icu, analysis-nori, analysis-smartcn, analysis-stconvert
		analysis-stempel, analysis-ukrainian, experimental-highlighter, extra, extra-analysis-esperanto
		extra-analysis-serbian, extra-analysis-slovak, ltr
	Picking analyzer...default
	Inferring index identifier...viwiki_archive_first
	Creating index...ok
		Validating number of shards...ok
		Validating replica range...ok
		Validating shard allocation settings...done
		Validating max shards per node...ok
	Validating analyzers...ok
	Validating mappings...
		Validating mapping...different...corrected
	Validating aliases...
		Validating viwiki_archive alias...alias is free...corrected
		Updating tracking indexes...done
done.
Making sure mwstore://local-multiwrite/local-public exists...making 'mwstore://local-multiwrite/local-public' public...done.
Making sure mwstore://local-multiwrite/local-thumb exists...making 'mwstore://local-multiwrite/local-thumb' public...done.
Making sure mwstore://local-multiwrite/local-transcoded exists...making 'mwstore://local-multiwrite/local-transcoded' public...done.
Making sure mwstore://local-multiwrite/local-temp exists...making 'mwstore://local-multiwrite/local-temp' private...done.
Making sure mwstore://local-multiwrite/local-deleted exists...making 'mwstore://local-multiwrite/local-deleted' private...done.
Making sure mwstore://local-multiwrite/timeline-render exists...making 'mwstore://local-multiwrite/timeline-render' public...done.
Done. sync the config as in https://wikitech.wikimedia.org/wiki/Add_a_wiki#MediaWiki_configuration

One question: Why --wiki=aawiki?

In T246511#5930441, @Zoranzoki21 wrote:

One question: Why --wiki=aawiki?

MediaWiki maintenance scripts can only be run in the context of a MediaWiki wiki. For addWiki in prod you choose one that is on the correct DB cluster (aawiki is on the default one, s3), on beta it probably doesn't matter which one you choose as there is only one DB cluster. aawiki is alphabetically first on the list IIRC

In T246511#5930441, @Zoranzoki21 wrote:

One question: Why --wiki=aawiki?

MediaWiki maintenance scripts can only be run in the context of a MediaWiki wiki. For addWiki in prod you choose one that is on the correct DB cluster (aawiki is on the default one, s3), on beta it probably doesn't matter which one you choose as there is only one DB cluster. aawiki is alphabetically first on the list IIRC

OT: Actually, it does matter which one is chosen, per what @Ladsgroup told me. He may be able to explain why aawiki is needed here.

In T246511#5930441, @Zoranzoki21 wrote:

One question: Why --wiki=aawiki?

MediaWiki maintenance scripts can only be run in the context of a MediaWiki wiki. For addWiki in prod you choose one that is on the correct DB cluster (aawiki is on the default one, s3), on beta it probably doesn't matter which one you choose as there is only one DB cluster. aawiki is alphabetically first on the list IIRC

OT: Actually, it does matter which one is chosen, per what @Ladsgroup told me. He may be able to explain why aawiki is needed here.

I once caused some issues because of using mediawikiwiki instead of aawiki (some differences in central auth IIRC) while both are on s3, there are lots of (sometimes unintentional) assumptions in addWiki.php about the dummy wiki. I recommend sticking to aawiki just to be safe (rewriting/improving addWiki.php is also a good idea).

For beta cluster, I think it's fine, as long as you know how to fix issues and bring it back, you can experiment IMO.

Change 575786 had a related patch set uploaded (by Reedy; owner: Reedy):
[operations/mediawiki-config@master] Add viwiki to interwiki-labs.php

https://gerrit.wikimedia.org/r/575786

Change 575786 merged by jenkins-bot:
[operations/mediawiki-config@master] Add viwiki to interwiki-labs.php

https://gerrit.wikimedia.org/r/575786

Change 575762 merged by jenkins-bot:
[operations/mediawiki-config@master] Set GrowthExperiments help panel search API for beta viwiki

https://gerrit.wikimedia.org/r/575762

Krinkle added a project: Platform Engineering.
Krinkle subscribed.

Change 575763 had a related patch set uploaded (by Gergő Tisza; owner: Gergő Tisza):
[mediawiki/services/restbase/deploy@master] Add viwiki to beta env

https://gerrit.wikimedia.org/r/575763

Re-opening as this patch is still pending review for the wiki to operate correctly. Otherwise anything relating to RESTBase/Parsoid is likely broken or incorrect.

Change 575763 merged by Ppchelko:
[mediawiki/services/restbase/deploy@master] Add viwiki to beta env

https://gerrit.wikimedia.org/r/575763

So seems everything is finally done, then