Page MenuHomePhabricator

Improve error monitoring
Open, Needs TriagePublic

Description

  1. Express catches synchronous and asynchronous errors inside route handlers and middleware: https://expressjs.com/en/guide/error-handling.html. Therefore, Express wouldn't crash in the event of expected (or unexpected) errors. We need to set up a logging middleware to help us detect and fix unexpected errors (T302696).
  2. If Express anyway crashes, the container fails but it is restarted because of the Kubernetes pod's restartPolicy. Therefore, we may need something that monitors container restarts and sends a notification. Unfortunately, the solution proposed in T286135#7739790 does not work, because a preStop event doesn't seem to be sent when the container is automatically restarted this way.
  3. Finally, we may also consider having an external service monitoring that the web server is responding as expected.

2 and 3 are discussed in T53434.