Page MenuHomePhabricator
Create Task
Maniphest T105887

CirrusSearch generates PHP notices when server is down
Closed, ResolvedPublic
Actions

Description

When our ES server went down, these started appearing in our logs.

PHP Fatal error: Call to a member function getTotalHits() on null in /srv/mediawiki/tags/2015-07-15_00:50:47/extensions/CirrusSearch/includes/BuildDocument/RedirectsAndIncomingLinks.php on line 111
PHP Notice: Undefined index: timed_out in /srv/mediawiki/tags/2015-07-15_00:50:47/extensions/CirrusSearch/includes/Searcher.php on line 1199

Details

ProjectBranchLines +/-Subject
mediawiki/extensions/CirrusSearchmaster+12 -0Handle null result from link count msearch
Customize query in gerrit

Event Timeline

Nikerabbit created this task.Jul 15 2015, 2:31 PM
Nikerabbit raised the priority of this task from to Needs Triage.
Nikerabbit updated the task description. (Show Details)
Nikerabbit added a project: CirrusSearch.
Nikerabbit added a subscriber: Nikerabbit.
Restricted Application added a project: Discovery-ARCHIVED. · View Herald TranscriptJul 15 2015, 2:31 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Nemo_bis added a parent task: T41480: [DO NOT USE] Issues affecting translatewiki.net [superseded by #affects-translatewiki.net].Jul 15 2015, 5:49 PM
MarkAHershberger added a subscriber: MarkAHershberger.Sep 11 2015, 5:03 PM

I'm getting a similar error from runJobs.php:

PHP Fatal error:  Call to a member function getTotalHits() on a non-object
in .../extensions/CirrusSearch/includes/BuildDocument/RedirectsAndIncomingLinks.php on line 111
Deskana triaged this task as Low priority.Dec 31 2015, 12:37 AM
Deskana moved this task from Needs triage to Search on the Discovery-ARCHIVED board.
Deskana moved this task from Inbox to Technical on the CirrusSearch board.
Deskana added a subscriber: Deskana.
• Pchelolo added a project: Services (watching).Aug 2 2018, 10:53 PM
• Pchelolo added a subscriber: • Pchelolo.

Just started happening again recently with a fairly high rate: https://logstash.wikimedia.org/goto/b78e539d56b4286f3096b061d2ebbc51

Restricted Application added a project: Discovery-Search. · View Herald TranscriptAug 2 2018, 10:53 PM
• mobrovac raised the priority of this task from Low to High.Aug 3 2018, 10:48 AM
• mobrovac added a subscriber: • mobrovac.

Raising the priority since there have been more than 20k such messages in the past 24h.

• EBjune added a subscriber: EBernhardson.Aug 7 2018, 2:59 PM
debt edited projects, added Discovery-Search (Current work); removed Discovery-Search.Aug 9 2018, 5:13 PM
debt added a subscriber: debt.

We'll take a look.

thcipriani added a subscriber: thcipriani.Aug 17 2018, 4:30 PM

Ping! This is still very noisy in the logs 45K messages in the past week.

Stacktrace if it's helpful:

#0 /srv/mediawiki/php-1.32.0-wmf.16/extensions/CirrusSearch/includes/BuildDocument/RedirectsAndIncomingLinks.php(93): CirrusSearch\BuildDocument\RedirectsAndIncomingLinks->realFinishBatch(array)
#1 /srv/mediawiki/php-1.32.0-wmf.16/includes/Hooks.php(174): CirrusSearch\BuildDocument\RedirectsAndIncomingLinks::finishBatch(array)
#2 /srv/mediawiki/php-1.32.0-wmf.16/includes/Hooks.php(202): Hooks::callHook(string, array, array, NULL)
#3 /srv/mediawiki/php-1.32.0-wmf.16/extensions/CirrusSearch/includes/Updater.php(413): Hooks::run(string, array)
#4 /srv/mediawiki/php-1.32.0-wmf.16/extensions/CirrusSearch/includes/Updater.php(204): CirrusSearch\Updater->buildDocumentsForPages(array, integer)
#5 /srv/mediawiki/php-1.32.0-wmf.16/extensions/CirrusSearch/includes/Updater.php(457): CirrusSearch\Updater->updatePages(array, integer)
#6 /srv/mediawiki/php-1.32.0-wmf.16/extensions/CirrusSearch/includes/Job/IncomingLinkCount.php(36): CirrusSearch\Updater->updateLinkedArticles(array)
#7 /srv/mediawiki/php-1.32.0-wmf.16/extensions/CirrusSearch/includes/Job/Job.php(99): CirrusSearch\Job\IncomingLinkCount->doJob()
#8 /srv/mediawiki/php-1.32.0-wmf.16/extensions/EventBus/includes/JobExecutor.php(67): CirrusSearch\Job\Job->run()
#9 /srv/mediawiki/rpc/RunSingleJob.php(67): JobExecutor->execute(array)
#10 {main}
gerritbot added a subscriber: gerritbot.Aug 17 2018, 10:55 PM

Change 453539 had a related patch set uploaded (by EBernhardson; owner: EBernhardson):
[mediawiki/extensions/CirrusSearch@master] Handle null result from link count msearch

https://gerrit.wikimedia.org/r/453539

gerritbot added a project: Patch-For-Review.Aug 17 2018, 10:55 PM
EBernhardson claimed this task.Aug 17 2018, 10:55 PM
EBernhardson moved this task from Incoming to Needs review on the Discovery-Search (Current work) board.
gerritbot added a comment.Aug 20 2018, 12:37 PM

Change 453539 merged by jenkins-bot:
[mediawiki/extensions/CirrusSearch@master] Handle null result from link count msearch

https://gerrit.wikimedia.org/r/453539

ReleaseTaggerBot added a project: MW-1.32-notes (WMF-deploy-2018-08-21 (1.32.0-wmf.18)).Aug 20 2018, 1:00 PM
EBernhardson moved this task from Needs review to Needs Reporting on the Discovery-Search (Current work) board.Aug 20 2018, 4:14 PM
debt closed this task as Resolved.Aug 24 2018, 4:07 PM
• Phabricator_maintenance added a project: affects-translatewiki.net.Sep 26 2019, 1:53 PM
• Phabricator_maintenance removed a parent task: T41480: [DO NOT USE] Issues affecting translatewiki.net [superseded by #affects-translatewiki.net].Sep 26 2019, 1:55 PM
Maintenance_bot removed a project: Patch-For-Review.Sep 26 2019, 2:10 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL