= Background information
In our project we follow some guidelines for observability established by our SREs as described here:
- wiki/Observability
- wikitech/Observability/Dashboard_guidelines
Here is how our template grafana dashboards look like:
- Grafana template dashboard
Some example dashboards from our services:
- Example of nodejs service dashboard (our push notifications service)
- Example of Go service (sessionstore)
Most of the systems metrics are generated directly from our kubernetes setup for example
- Memory/CPU/Disk metrics
- Network metrics
- HTTP metrics
- Kubernetes metrics
Here are some ideas for tegola specifically:
- Go lang runtime
- Memory
- Garbage collection
- Timings
- Usage
- Prometheus go client exposes a variety of memory usage metrics by default
- `go_memstats_xxx`
- Goroutines
- Number of goroutines
- HTTP endpoints
- Timings for request/response cycle
- Per status code type/method
- Additional metadata related to tile/map/layer/zoom level if applicable?
- Request/response sizes
- Cache
- Number of hit/miss
- Get/put timings
- Seeding/purging stats
- Connection status (active/failed/reconnects) to cache backend
- Database connection
- Number of connections
- Query timings
= Open questions
- Do we need fine-grained observability of the Swift cache (i.e. connection pool)
= Acceptance criteria
- [] Documentation is updated
- [] Grafana Dashboard is created to monitor the aforementioned metrics