Page MenuHomePhabricator
Create Task
Maniphest T410269

Wikibase CI broken: Unknown filter type [truncate_norm] for [truncate_keyword]
Closed, ResolvedPublic
Actions

Description

Wikibase quibble-apitests-only-vendor-php81 tests have been failing since at least Friday, 14 November (#9391 for this change):

php maintenance/run.php CirrusSearch\Maintenance\UpdateSearchIndexConfig --startOver
Updating cluster ...
indexing namespaces...
	Fetching server version...opensearch 1.3.20...ok
mw_cirrus_metastore missing, creating new metastore index.
Creating metastore index... mw_cirrus_metastore_first	Scanning available plugins...
		analysis-hebrew, analysis-icu, analysis-nori, analysis-smartcn, analysis-stempel
		analysis-sudachi, analysis-[u]krainian, cirrus-highlighter, ltr, opensearch-analysis-stconvert
		opensearch-extra, opensearch-extra-analysis-esperanto, opensearch-extra-analysis-homoglyph, opensearch-extra-analysis-khmer, opensearch-extra-analysis-serbian
		opensearch-extra-analysis-slovak, opensearch-extra-analysis-textify, opensearch-extra-analysis-turkish, opensearch-extra-analysis-ukrainian, opensearch-knn
		repository-s3
Elastica\Exception\ResponseException from line 178 of /workspace/src/vendor/ruflin/elastica/src/Transport/Http.php: Unknown filter type [truncate_norm] for [truncate_keyword]
#0 /workspace/src/vendor/ruflin/elastica/src/Request.php(183): Elastica\Transport\Http->exec()
#1 /workspace/src/vendor/ruflin/elastica/src/Client.php(545): Elastica\Request->send()
#2 /workspace/src/vendor/ruflin/elastica/src/Index.php(759): Elastica\Client->request()
#3 /workspace/src/extensions/CirrusSearch/includes/MetaStore/MetaStoreIndex.php(213): Elastica\Index->request()
#4 /workspace/src/extensions/CirrusSearch/includes/MetaStore/MetaStoreIndex.php(145): CirrusSearch\MetaStore\MetaStoreIndex->createNewIndex()
#5 /workspace/src/extensions/CirrusSearch/includes/Maintenance/Maintenance.php(238): CirrusSearch\MetaStore\MetaStoreIndex->createIfNecessary()
#6 /workspace/src/extensions/CirrusSearch/maintenance/IndexNamespaces.php(43): CirrusSearch\Maintenance\Maintenance->maybeCreateMetastore()
#7 /workspace/src/extensions/CirrusSearch/maintenance/UpdateSearchIndexConfig.php(69): CirrusSearch\Maintenance\IndexNamespaces->execute()
#8 /workspace/src/maintenance/includes/MaintenanceRunner.php(696): CirrusSearch\Maintenance\UpdateSearchIndexConfig->execute()
#9 /workspace/src/maintenance/run.php(53): MediaWiki\Maintenance\MaintenanceRunner->run()
#10 {main}

CirrusSearch: Make keyword fields actual opensearch keyword fields looks like a likely culprit.

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
jjb: Update Quibble-based jobs for new wmf-opensearch-search-plugins releaseintegration/configmaster+31 -31
Docker: [quibble-bullseye] Bump for wmf-opensearch-search-pluginsintegration/configmaster+48 -0
Customize query in gerrit

Related Objects

Event Timeline

Lucas_Werkmeister_WMDE created this task.Mon, Nov 17, 11:39 AM
Restricted Application added a project: Discovery-Search. · View Herald TranscriptMon, Nov 17, 11:39 AM
Restricted Application added subscribers: Base, Aklapper. · View Herald Transcript
Lucas_Werkmeister_WMDE updated the task description. (Show Details)Mon, Nov 17, 11:41 AM
Lucas_Werkmeister_WMDE removed a subscriber: Base.
Jakob_WMDE subscribed.Mon, Nov 17, 11:44 AM
Silvan_WMDE subscribed.Mon, Nov 17, 11:56 AM

confirmed: I can reproduce this locally by running CirrusSearch\Maintenance\UpdateSearchIndexConfig --startOver and moving back to a commit prior to CirrusSearch: Make keyword fields actual opensearch keyword fields does fix it

Lucas_Werkmeister_WMDE added a comment.Mon, Nov 17, 12:02 PM

So far I’ve only seen failed builds in Wikibase. Looking at zuul/layout.yaml, I suspect it’s not the only extension, but it’s probably not super widespread: affected extensions would be ones in the intersection of “uses the extension-apitests template”and “depends on CirrusSearch”.

Lucas_Werkmeister_WMDE added a comment.Mon, Nov 17, 12:04 PM

Hm, not really – VisualEditor uses apitests, pulls in CirrusSearch, and yet works apparently.

Jakob_WMDE added a comment.Mon, Nov 17, 12:07 PM

Wikibase runs api-testing using a real OpenSearch in Quibble, and may still be the only job doing so. I noticed that a new version of wmf-opensearch-search-plugins was released in the process of T40403 which is linked to the problematic commit. Could it be that we need to bump the version of this package somewhere, or that the latest version has not been published yet? It is being installed here: https://gerrit.wikimedia.org/r/plugins/gitiles/integration/config/+/refs/heads/master/dockerfiles/quibble-bullseye/Dockerfile.template#142

Silvan_WMDE added a comment.Mon, Nov 17, 12:13 PM

updating to docker-registry.wikimedia.org/repos/search-platform/cirrussearch-opensearch-image:v1.3.20-12 does fix the issue here locally

EBernhardson subscribed.Mon, Nov 17, 3:06 PM
dcausse subscribed.EditedMon, Nov 17, 3:30 PM

Indeed, the new debian package wmf-opensearch-search-plugins version 1.3.20+12 has to be installed to run the lastest cirrus version. We generally maintain the cirrussearch-opensearch-image docker image that is used by MW developers and our cirrus integration test suite, but here I think that you install opensearch on the existing quibble image and thus refreshing this image with the new version of the plugin is indeed what should be needed.

Lucas_Werkmeister_WMDE added a comment.Mon, Nov 17, 4:13 PM

(For the record here, this was temporarily fixed by reverting the CirrusSearch change.)

Jakob_WMDE added a subscriber: hashar.Mon, Nov 17, 4:32 PM

Thanks for the revert @dcausse! Wikibase CI is unblocked now.

@hashar could you help us out here by building a new version of the Quibble Bullseye image with the new version of the OpenSearch plugins so that the CirrusSearch commit can be un-reverted?

gerritbot added a comment.Mon, Nov 17, 4:36 PM

Change #1206410 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] Docker: [quibble-bullseye] Bump for wmf-opensearch-search-plugins

https://gerrit.wikimedia.org/r/1206410

gerritbot added a project: Patch-For-Review.Mon, Nov 17, 4:36 PM
gerritbot added a comment.Mon, Nov 17, 4:38 PM

Change #1206410 merged by jenkins-bot:

[integration/config@master] Docker: [quibble-bullseye] Bump for wmf-opensearch-search-plugins

https://gerrit.wikimedia.org/r/1206410

Stashbot added a comment.Mon, Nov 17, 4:39 PM

Mentioned in SAL (#wikimedia-releng) [2025-11-17T16:39:32Z] <James_F> Docker: [quibble-bullseye] Bump for wmf-opensearch-search-plugins, for T410269

gerritbot added a comment.Mon, Nov 17, 4:43 PM

Change #1206414 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] jjb: Update Quibble-based jobs for new wmf-opensearch-search-plugins release

https://gerrit.wikimedia.org/r/1206414

Jdforrester-WMF subscribed.Mon, Nov 17, 5:01 PM
In T410269#11379689, @Jakob_WMDE wrote:

@hashar could you help us out here by building a new version of the Quibble Bullseye image with the new version of the OpenSearch plugins so that the CirrusSearch commit can be un-reverted?

Done in hashar's absence.

gerritbot added a comment.Mon, Nov 17, 5:02 PM

Change #1206414 merged by jenkins-bot:

[integration/config@master] jjb: Update Quibble-based jobs for new wmf-opensearch-search-plugins release

https://gerrit.wikimedia.org/r/1206414

Maintenance_bot removed a project: Patch-For-Review.Mon, Nov 17, 5:32 PM
Jakob_WMDE closed this task as Resolved.Tue, Nov 18, 8:44 AM

Wikibase CI was unblocked after the revert was merged, and now we also have the updated OpenSearch plugins in the Quibble docker image.

dcausse edited projects, added Discovery-Search (2025.10.20 - 2025.12.31); removed Discovery-Search.Wed, Nov 19, 8:41 AM
dcausse moved this task from Incoming to Done on the Discovery-Search (2025.10.20 - 2025.12.31) board.
pfischer moved this task from Done to Reported on the Discovery-Search (2025.10.20 - 2025.12.31) board.Fri, Nov 21, 1:37 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits