Page MenuHomePhabricator

[beta cluster] Search - "An error has occurred while searching"
Closed, ResolvedPublic

Description

Looks the same as T275975.

Special:Search produces:

An error has occurred while searching: We could not complete your search due to a temporary problem. Please try again later.

Screen Shot 2023-03-17 at 2.58.28 PM.png (894×2 px, 277 KB)

Special:Homepage displays the Console error:
Fetching task suggestions failed: cirrussearch-backend-error

Event Timeline

Apparently the SSL cert for some beta ElasticSearch server is invalid (curl error code 60).

@bking has delt with these issues before, might have ideas

tgr@deployment-mwmaint02:~$ curl 'https://deployment-elastic09.deployment-prep.eqiad1.wikimedia.cloud:9243'
curl: (60) SSL certificate problem: certificate has expired

tgr@deployment-mwmaint02:~$ echo | openssl s_client -servername deployment-elastic09.deployment-prep.eqiad1.wikimedia.cloud -connect deployment-elastic09.deployment-prep.eqiad1.wikimedia.cloud:9243 | openssl x509 -inform pem -noout -text
depth=2 C = US, ST = California, L = San Francisco, O = "Wikimedia Foundation, Inc", OU = Cloud Services, CN = WMF_TEST_CA
verify return:1
depth=1 C = US, L = San Francisco, O = "Wikimedia Foundation, Inc", OU = Cloud Services, CN = discovery
verify return:1
depth=0 CN = deployment-elastic09.deployment-prep.eqiad1.wikimedia.cloud
verify error:num=10:certificate has expired
notAfter=Mar 16 11:16:00 2023 GMT
verify return:1
depth=0 CN = deployment-elastic09.deployment-prep.eqiad1.wikimedia.cloud
notAfter=Mar 16 11:16:00 2023 GMT
verify return:1

Mentioned in SAL (#wikimedia-releng) [2023-03-17T22:58:58Z] <urbanecm> deployment-prep: Reboot deployment-elastic09 (T332455)

Mentioned in SAL (#wikimedia-releng) [2023-03-17T23:00:46Z] <urbanecm> deployment-prep: Reboot deployment-elastic10, deployment-elastic11 (T332455)

Urbanecm_WMF claimed this task.
Urbanecm_WMF subscribed.

I rebooted it and newer cert was picked up. Seems to be working now.