Page MenuHomePhabricator

Steady rate of Phonos Swift errors (inc. DescribeFileOp failed, FileBackendStore::ingestFreshFileStats: Could not stat)
Open, LowPublicPRODUCTION ERROR

Description

Unsure of the impact (if any), but found during ongoing investigation into T325464: [8 hours] Investigate why phonosIPAFilePersist jobs aren't being generated — setting low as appears to have no(?) user impact.

Error

Feb 2, 2023 @ 21:04:52.020

  • mwversion: 1.40.0-wmf.21
  • reqId: 14bc0f88-6e55-4b87-96a1-3a009bc04286
  • Find reqId in Logstash
  • wiki: afwiktionary
normalized_message
DescribeFileOp failed: {"src":"mwstore://global-swift-codfw/phonos-render/8/i/8iughjrbdzl4wuo7z9hrgdjkk0ru9fz.mp3","headers":{"x-delete-at":1677854219},"failedAction":"attempt"}

Error

Feb 2, 2023 @ 20:37:32.339

message
HTTP 502 (Bad Gateway) in 'SwiftFileBackend::doDescribeInternal' (given '{"async":false,"src":"mwstore://global-swift-codfw/phonos-render/8/0/807xt6wxxcd9cy00ki6rfn51v2wvsyz.mp3","headers":{"x-delete-at":1677480728}}')

Error

Feb 2, 2023 @ 20:01:28.376

normalized_message
FileBackendStore::ingestFreshFileStats: Could not stat file mwstore://global-swift-codfw/phonos-render/f/p/fpvidqsoqb413d74xvdv5hchnko7iqz.mp3