Page MenuHomePhabricator

Investigate EQIAD WDQS graph split host alerts/separate out categories in the puppet code
Closed, ResolvedPublic

Description

Disclaimer: This info is pieced together from the scrollback in Wikimedia-Search . If you have better info, feel free to correct this.

Beginning at 0752 UTC today, all wdqs graph split hosts in EQIAD alerted:

[07:52:25]  <jinxer-wm> FIRING: SystemdUnitFailed: systemd-timedated.service on wdqs1023:9100 - https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state - https://grafana.wikimedia.org/d/g-AaZRFWk/systemd-status - https://alerts.wikimedia.org/?q=alertname%3DSystemdUnitFailed

Per earlier conversation with @dcausse , the working theory is that load-categories-daily.timer ran before we could reload the wdqs-categories database. When this happens, the daily job tries to write to a namespace that doesn't exist , which fails and generates enough log spam to lock up the host. The log spam floods the console as well, making it unresponsive.

The actual impact was minimal:

  • this service is quite new and does not serve a lot of traffic
  • it was apparently still serving queries via query-main.wikidata.org, so no actual user impact

Event Timeline

One possible explanation for unresponsiveness is log spam, we pushed this puppet patch in an attempt to mitigate this potential issue.

Change #1070551 had a related patch set uploaded (by Bking; author: DCausse):

[operations/puppet@production] wdqs: remove deprecated wcqs reload crontask

https://gerrit.wikimedia.org/r/1070551

Change #1070551 merged by Bking:

[operations/puppet@production] wdqs: remove deprecated wcqs reload crontask

https://gerrit.wikimedia.org/r/1070551

Change #1070587 had a related patch set uploaded (by Bking; author: DCausse):

[operations/puppet@production] wdqs: drop run_tests crontask

https://gerrit.wikimedia.org/r/1070587

Change #1070587 merged by Bking:

[operations/puppet@production] wdqs: drop run_tests crontask

https://gerrit.wikimedia.org/r/1070587

Change #1070603 had a related patch set uploaded (by Bking; author: DCausse):

[operations/puppet@production] wdqs: drop deploy_mode

https://gerrit.wikimedia.org/r/1070603

Change #1070603 merged by Bking:

[operations/puppet@production] wdqs: drop deploy_mode

https://gerrit.wikimedia.org/r/1070603

bking renamed this task from Investigate WDQS graph split hosts' unresponsiveness to Investigate EQIAD WDQS graph split hosts' unresponsiveness.Sep 4 2024, 10:14 PM
bking updated the task description. (Show Details)
bking updated the task description. (Show Details)

Change #1070955 had a related patch set uploaded (by DCausse; author: DCausse):

[operations/puppet@production] wdqs: better isolation of categories related

https://gerrit.wikimedia.org/r/1070955

Change #1070956 had a related patch set uploaded (by DCausse; author: DCausse):

[operations/puppet@production] wdqs: common module and profile should not define categories_endpoint

https://gerrit.wikimedia.org/r/1070956

Change #1070958 had a related patch set uploaded (by DCausse; author: DCausse):

[operations/puppet@production] wdqs: do not add categories on main and scholarly endpoints

https://gerrit.wikimedia.org/r/1070958

bking renamed this task from Investigate EQIAD WDQS graph split hosts' unresponsiveness to Investigate EQIAD WDQS graph split host alerts.Sep 5 2024, 6:48 PM
Gehel triaged this task as High priority.Sep 6 2024, 12:46 PM

Change #1070955 merged by Bking:

[operations/puppet@production] wdqs: better isolation of categories components

https://gerrit.wikimedia.org/r/1070955

Change #1070956 merged by Bking:

[operations/puppet@production] wdqs: common module and profile should not define categories_endpoint

https://gerrit.wikimedia.org/r/1070956

Change #1070958 merged by Bking:

[operations/puppet@production] wdqs: do not add categories on main and scholarly endpoints

https://gerrit.wikimedia.org/r/1070958

bking renamed this task from Investigate EQIAD WDQS graph split host alerts to Investigate EQIAD WDQS graph split host alerts/separate out categories in the puppet code.Oct 2 2024, 5:35 PM

Per the above PRs, the categories puppet code has been separated out. As such, we can close this ticket.