The search index being lagged on wikidata means that users are unable to use items that they have created.
I care specifically about wikidata in this ticket, but if other indexes should have alarms then we should probably create those too.

This ticket was created after T219364 which caused around 24 hours of search index lag.

From IRC:

11:03 AM <+gehel> lag of multiple hours is unexpected and should not happen (or if it does, it should be treated as a bug)
11:04 AM <+gehel> but lag of < 1h is perfectly normal
11:04 AM <addshore> gehel: should we have some alarms then?
11:04 AM <+gehel> yeah
11:04 AM <addshore> :D
11:04 AM <addshore> right now, people can use items that they created in the last 24 hours on wikidata in statements ;)
11:04 AM <+gehel> we have an alert on writes being explicitly frozen for too long
11:05 AM <+gehel> but we don't one checking the lag specifically
11:08 AM <+gehel> we also have (or should have) a check on the consumer lag of the elastica write job
11:08 AM <+gehel> not sure why it wasn't triggered