Page MenuHomePhabricator

cxserver: SectionMapping DB timeout
Closed, ResolvedPublic2 Estimated Story PointsBUG REPORT

Description

The cxserver log has entries with a DB error when accessing the section mapping database on m5.

Log

Error: Could not connect to section mapping databaseError: connect ETIMEDOUT

https://logstash.wikimedia.org/app/dashboards#/doc/logstash-*/logstash-k8s-1-7.0.0-1-2025.01.15?id=VOATaZQBLmySI1N_eieU

Error is from cxserver/lib/suggestion/SectionSuggester.js where it fails to connect with database.

Event Timeline

Change #1111675 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[mediawiki/services/cxserver@master] SectionSuggester: Open db connection in the same execution context

https://gerrit.wikimedia.org/r/1111675

Was this a one time thing? m5 databases look up and healthy

Change #1111675 merged by jenkins-bot:

[mediawiki/services/cxserver@master] SectionSuggester: Open db connection in the same execution context

https://gerrit.wikimedia.org/r/1111675

Change #1111948 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update cxserver to 2025-01-16-103443-production

https://gerrit.wikimedia.org/r/1111948

Was this a one time thing? m5 databases look up and healthy

Getting more logs with some fixes in logging. Let's see!

Change #1111948 merged by jenkins-bot:

[operations/deployment-charts@master] Update cxserver to 2025-01-16-103443-production

https://gerrit.wikimedia.org/r/1111948

Nikerabbit set the point value for this task to 2.EditedJan 16 2025, 3:22 PM
Nikerabbit subscribed.

I think I'm only seeing errors before the latest deployment but might to be too early to tell.

With the latest cxserver logs (sadly, not available to logstash as of now due to a different issue), I don't see the above error CC @abi_

abi_ moved this task from Need QA to Done on the LPL Essential (LPL Essential 2024 Nov-Jan) board.

Not seeing this error anymore.