Page MenuHomePhabricator

CirrusSearch queues up jobs even though $wgSearchType isn't set to use it
Open, Needs TriagePublic

Description

In this issue https://github.com/wmde/wikibase-release-pipeline/issues/295 it was reported that running the docker images without ElasticSearch being enabled CirrusSearch still queues up jobs that just fail because there is not ElasticSearch around.

Not sure if this is to be a bug in CirrusSearch or just a quirk, but it could easily be fixed by hiding the loading of CirrusSearch behind the MW_ELASTIC_HOST that hides the rest of the ElasticSearch configuration.

wikibase_jobrunner_1  | 2022-01-18 12:10:38 cirrusSearchLinksUpdatePrioritized Item:Q1 addedLinks=[] removedLinks=[] prioritize=1 cluster= namespace=120 title=Q1 requestId=5d9067fe31b28690f9c4e92a (id=2,timestamp=20220118121037) STARTING
wikibase_jobrunner_1  | 2022-01-18 12:10:38 cirrusSearchLinksUpdatePrioritized Item:Q1 addedLinks=[] removedLinks=[] prioritize=1 cluster= namespace=120 title=Q1 requestId=5d9067fe31b28690f9c4e92a (id=2,timestamp=20220118121037) t=66 good
wikibase_jobrunner_1  | 2022-01-18 12:10:38 cirrusSearchElasticaWrite Special: method=sendData arguments=["content",[{"data":{"version":2,"wiki":"my_wiki","namespace":120,"namespace_text":"Item","title":"Q1","timestamp":"2022-01-18T12:10:37Z","create_timestamp":"2022-01-18T12:10:37Z","redirect":[]},"params":{"_id":"2","_type":"","_index":"","_cirrus_hints":{"BuildDocument_flags":0,"noop":{"version":"documentVersion"}}},"upsert":true}]] cluster=default createdAt=1642507838 errorCount=0 retryCount=0 requestId=5d9067fe31b28690f9c4e92a namespace=-1 title= (id=5,timestamp=20220118121038) STARTING
wikibase_jobrunner_1  | 2022-01-18 12:10:38 cirrusSearchElasticaWrite Special: method=sendData arguments=["content",[{"data":{"version":2,"wiki":"my_wiki","namespace":120,"namespace_text":"Item","title":"Q1","timestamp":"2022-01-18T12:10:37Z","create_timestamp":"2022-01-18T12:10:37Z","redirect":[]},"params":{"_id":"2","_type":"","_index":"","_cirrus_hints":{"BuildDocument_flags":0,"noop":{"version":"documentVersion"}}},"upsert":true}]] cluster=default createdAt=1642507838 errorCount=0 retryCount=0 requestId=5d9067fe31b28690f9c4e92a namespace=-1 title= (id=5,timestamp=20220118121038) t=9 error=ElasticaWrite job failed: Requeued
wikibase_jobrunner_1  | 2022-01-18 12:10:38 cirrusSearchElasticaWrite Special: method=sendData arguments=["content",[{"data":{"version":2,"wiki":"my_wiki","namespace":120,"namespace_text":"Item","title":"Q1","timestamp":"2022-01-18T12:10:37Z","create_timestamp":"2022-01-18T12:10:37Z","redirect":[]},"params":{"_id":"2","_type":"","_index":"","_cirrus_hints":{"BuildDocument_flags":0,"noop":{"version":"documentVersion"}}},"upsert":true}]] cluster=default createdAt=1642507838 errorCount=1 retryCount=0 requestId=5d9067fe31b28690f9c4e92a namespace=-1 title= (id=6,timestamp=20220118121038) STARTING
wikibase_jobrunner_1  | 2022-01-18 12:10:38 cirrusSearchElasticaWrite Special: method=sendData arguments=["content",[{"data":{"version":2,"wiki":"my_wiki","namespace":120,"namespace_text":"Item","title":"Q1","timestamp":"2022-01-18T12:10:37Z","create_timestamp":"2022-01-18T12:10:37Z","redirect":[]},"params":{"_id":"2","_type":"","_index":"","_cirrus_hints":{"BuildDocument_flags":0,"noop":{"version":"documentVersion"}}},"upsert":true}]] cluster=default createdAt=1642507838 errorCount=1 retryCount=0 requestId=5d9067fe31b28690f9c4e92a namespace=-1 title= (id=6,timestamp=20220118121038) t=9 error=ElasticaWrite job failed: Requeued
wikibase_jobrunner_1  | 2022-01-18 12:10:38 cirrusSearchElasticaWrite Special: method=sendData arguments=["content",[{"data":{"version":2,"wiki":"my_wiki","namespace":120,"namespace_text":"Item","title":"Q1","timestamp":"2022-01-18T12:10:37Z","create_timestamp":"2022-01-18T12:10:37Z","redirect":[]},"params":{"_id":"2","_type":"","_index":"","_cirrus_hints":{"BuildDocument_flags":0,"noop":{"version":"documentVersion"}}},"upsert":true}]] cluster=default createdAt=1642507838 errorCount=2 retryCount=0 requestId=5d9067fe31b28690f9c4e92a namespace=-1 title= (id=7,timestamp=20220118121038) STARTING
wikibase_jobrunner_1  | 2022-01-18 12:10:38 cirrusSearchElasticaWrite Special: method=sendData arguments=["content",[{"data":{"version":2,"wiki":"my_wiki","namespace":120,"namespace_text":"Item","title":"Q1","timestamp":"2022-01-18T12:10:37Z","create_timestamp":"2022-01-18T12:10:37Z","redirect":[]},"params":{"_id":"2","_type":"","_index":"","_cirrus_hints":{"BuildDocument_flags":0,"noop":{"version":"documentVersion"}}},"upsert":true}]] cluster=default createdAt=1642507838 errorCount=2 retryCount=0 requestId=5d9067fe31b28690f9c4e92a namespace=-1 title= (id=7,timestamp=20220118121038) t=10 error=ElasticaWrite job failed: Requeued
wikibase_jobrunner_1  | 2022-01-18 12:10:39 cirrusSearchElasticaWrite Special: method=sendData arguments=["content",[{"data":{"version":2,"wiki":"my_wiki","namespace":120,"namespace_text":"Item","title":"Q1","timestamp":"2022-01-18T12:10:37Z","create_timestamp":"2022-01-18T12:10:37Z","redirect":[]},"params":{"_id":"2","_type":"","_index":"","_cirrus_hints":{"BuildDocument_flags":0,"noop":{"version":"documentVersion"}}},"upsert":true}]] cluster=default createdAt=1642507838 errorCount=3 retryCount=0 requestId=5d9067fe31b28690f9c4e92a namespace=-1 title= (id=8,timestamp=20220118121038) STARTING
wikibase_jobrunner_1  | 2022-01-18 12:10:39 cirrusSearchElasticaWrite Special: method=sendData arguments=["content",[{"data":{"version":2,"wiki":"my_wiki","namespace":120,"namespace_text":"Item","title":"Q1","timestamp":"2022-01-18T12:10:37Z","create_timestamp":"2022-01-18T12:10:37Z","redirect":[]},"params":{"_id":"2","_type":"","_index":"","_cirrus_hints":{"BuildDocument_flags":0,"noop":{"version":"documentVersion"}}},"upsert":true}]] cluster=default createdAt=1642507838 errorCount=3 retryCount=0 requestId=5d9067fe31b28690f9c4e92a namespace=-1 title= (id=8,timestamp=20220118121038) t=9 error=ElasticaWrite job failed: Requeued
wikibase_jobrunner_1  | 2022-01-18 12:10:39 cirrusSearchElasticaWrite Special: method=sendData arguments=["content",[{"data":{"version":2,"wiki":"my_wiki","namespace":120,"namespace_text":"Item","title":"Q1","timestamp":"2022-01-18T12:10:37Z","create_timestamp":"2022-01-18T12:10:37Z","redirect":[]},"params":{"_id":"2","_type":"","_index":"","_cirrus_hints":{"BuildDocument_flags":0,"noop":{"version":"documentVersion"}}},"upsert":true}]] cluster=default createdAt=1642507838 errorCount=4 retryCount=0 requestId=5d9067fe31b28690f9c4e92a namespace=-1 title= (id=9,timestamp=20220118121039) STARTING
wikibase_jobrunner_1  | 2022-01-18 12:10:39 cirrusSearchElasticaWrite Special: method=sendData arguments=["content",[{"data":{"version":2,"wiki":"my_wiki","namespace":120,"namespace_text":"Item","title":"Q1","timestamp":"2022-01-18T12:10:37Z","create_timestamp":"2022-01-18T12:10:37Z","redirect":[]},"params":{"_id":"2","_type":"","_index":"","_cirrus_hints":{"BuildDocument_flags":0,"noop":{"version":"documentVersion"}}},"upsert":true}]] cluster=default createdAt=1642507838 errorCount=4 retryCount=0 requestId=5d9067fe31b28690f9c4e92a namespace=-1 title= (id=9,timestamp=20220118121039) t=7 error=ElasticaWrite job failed: Dropped

AC

  • CirrusSearch doesn't add useless jobs

Event Timeline

This wasn't included in the last release but the fix is now merged into main and works. I'd mark this as resolved. Thanks 🙏

Moving back to doing, as I reverted it to make the wmde6 release for T301663: Prepare wmde.6 (1.35) minor / security release
I'll then revert the revert and done and resolve this