Page MenuHomePhabricator

vagrant wikibase cirrus role not working / should set $wgWBCSUseCirrus = true; to enable
Closed, ResolvedPublicBUG REPORT

Description

I've enabled the vagrant wikibase cirrus role, however I'm still getting empty results for srsearch queries, i.e. wbhasstatement gives me:

{
    "batchcomplete": "",
    "query": {
        "searchinfo": {
            "totalhits": 0
        },
        "search": []
    }
}

I've tried reindexing from CirrusSearch, which succeeds, but doesn't help:

> mwscript extensions/CirrusSearch/maintenance/forceSearchIndex.php
[                wiki] Indexed 3 pages ending at 3 at 3/second
Indexed a total of 3 pages at 3/second

The puppet file didn't look like it enabled it, so I tried enabling it according to the docs, but that didn't work either:

http://wikidata.wiki.local.wmftest.net:8080/w/api.php?action=query&list=search&srsearch=haswbstatement:P7=7&useCirrus=1

This results in

"main": {
    "*": "Unrecognized parameter: useCirrus."
},

I've also tried manually enabling in in my localsettings, i.e.

$wgWBCSUseCirrus = true;

This results in

{
    "batchcomplete": "",
    "query": {
        "searchinfo": {
            "totalhits": 0
        },
        "search": []
    }
}

Even after reprovisioning, reindex using mwscript extensions/CirrusSearch/maintenance/forceSearchIndex.php

Event Timeline

Mvolz created this task.Jul 19 2019, 10:53 AM
Mvolz updated the task description. (Show Details)
Tgr added a subscriber: Tgr.Jul 19 2019, 11:04 AM
Smalyshev triaged this task as Normal priority.Jul 31 2019, 8:10 PM
Smalyshev moved this task from Backlog to Doing on the User-Smalyshev board.

I tried to install wikibasecirrussearch with vagrant and it seems to work for me, after manually enabling $wgWBCSUseCirrus = true; (that part is not automatic yet). Maybe some other configurations are interfering? Are there any local configs and where do you configure $wgWBCSUseCirrus = true;? Did you check that you have latest master checkout of all modules (CirrusSearch, Wikibase, WikibaseCirrusSearch)?

Smalyshev moved this task from Doing to In review on the User-Smalyshev board.Aug 1 2019, 4:42 AM
Smalyshev moved this task from In review to Waiting/Blocked on the User-Smalyshev board.
Mvolz added a comment.Aug 1 2019, 12:57 PM

I tried to install wikibasecirrussearch with vagrant and it seems to work for me, after manually enabling $wgWBCSUseCirrus = true; (that part is not automatic yet). Maybe some other configurations are interfering? Are there any local configs and where do you configure $wgWBCSUseCirrus = true;? Did you check that you have latest master checkout of all modules (CirrusSearch, Wikibase, WikibaseCirrusSearch)?

On update I'm no longer getting the error with $wgWBCSUseCirrus = true, but I'm still not getting any results through with haswbstatement queries, even after reprovisioning- any further debugging tips?

Mvolz updated the task description. (Show Details)Aug 1 2019, 12:59 PM
Mvolz renamed this task from vagrant wikibase cirrus role not working to vagrant wikibase cirrus role not working / should set $wgWBCSUseCirrus = true; to enable.Aug 2 2019, 8:38 AM
Gehel assigned this task to dcausse.Aug 27 2019, 5:29 PM
dcausse moved this task from Backlog to Done on the MediaWiki-Vagrant board.Aug 29 2019, 1:30 PM

The searchIndexProperties wikibase config needs to be populated as well.

What I did (for the record):

  • fetch a fresh vagrant
  • enable wikibasecirrussearch
  • provision (ran into T231534)
  • ran vagrant provision a second time
  • created /vagrant/settings.d/wikis/wikidatawiki/settings.d/20-cirrus.php to set $wgWBCSUseCirrus = true;
  • edited /vagrant/mediawiki/extensions/repo/config/Wikibase.default.php and set 'searchIndexProperties' => ['P1'] (there are probably better ways to customize wikibase in vagrant)
  • reindexed wikidatawiki: mwscript extensions/CirrusSearch/maintenance/updateSearchIndexConfig.php --wiki wikidatawiki --reindexAndRemoveOk --indexIdentifier now
  • ran into GC issues (T211362)
  • edited /etc/elasticsearch/jvm.options and set -Xmx512
  • deleted partially created indices: curl -XDELETE http://localhost:9200/wikidatawiki*
  • created the indices: mwscript extensions/CirrusSearch/maintenance/updateSearchIndexConfig.php --wiki wikidatawiki
  • populated the index: mwscript extensions/CirrusSearch/maintenance/forceSearchIndex.php --wiki wikidatawiki
  • created P1 and an entity using it, searched for haswbstatement:P1, haswbstatement:* both returned Q1

I'll increase the heap as suggested in T211362.

As for this particular ticket I'm not sure what to do, I'm tentatively moving it to done, @Mvolz please let me know if setting searchIndexProperties did solve the problem.

Tgr added a comment.Aug 29 2019, 2:16 PM

Thanks for lookig into it! These should be folded into the Vagrant role settings. I can do that in a few weeks; if you want to do it before that, the mediainfo role has an example of how to create Wikibase properties/entities (that should probably be moved to the wikidata and wikibase_repo roles so it's always available), JVM options are in puppet/modules/elasticsearch/files/jvm.options and most of the configuration should probably happen via the mediawiki::extension resource in the wikibasecirrussearch role. The reindexing should be in theory handled by the cirrussearch role if the other settings are correct.

Change 533246 had a related patch set uploaded (by DCausse; owner: DCausse):
[mediawiki/vagrant@master] [mediainfo] Tell cirrus to index P1 and P2 as statements

https://gerrit.wikimedia.org/r/533246

@Tgr thanks for the info!
I've uploaded a patch but I attached the config listing the properties to index under mediainfo, reason is that it's mediainfo that creates these properties. Since we have to create some properties I'm not entirely sure what to do in wikibasecirrussearch...
As for $wgWBCSUseCirrus = true I'm still unsure but IIRC @Smalyshev had some reasons not to set it to true by default.

Change 533246 merged by jenkins-bot:
[mediawiki/vagrant@master] [mediainfo] Tell cirrus to index P1 and P2 as statements

https://gerrit.wikimedia.org/r/533246

debt closed this task as Resolved.Sep 5 2019, 6:23 PM