We should mess around with building (or finding) something that sees errors in logstash, dedupes them, and turns them into Phabricator tasks with far less manual intervention and general toil than the current painstaking and error-prone process.
We've discussed a lot of ideas for this sort of thing during triage sessions, train deploys, and casual ranting during offsites. Let's just experiment.
A few notes from the last time this came up:
- Fully automated pipelining of tasks into Phabricator is probably difficult because of potential security issues, etc. On the other hand, it ought to be possible to get a list of errors and click a button to say either "make this a task" or "this is associated with the existing errors". Phatality but more so, essentially.
- Kibana is extremely painful to use. Can we just cut it out of the interaction loop entirely (but maybe keep filters in sync)?
- Or can we just replace the new errors dashboard entirely?