We rely on the stock mysql SHOW STATUS functions to detect slave lag, but these only give log -> DB lag not the "lag behind master DB". We want to properly detect lag even when:
a) The slave can't talk to the master
b) Binlogs relay at a trickle due to network issues
c) Replication uses chaining (eg. master -> slave1 -> slave2)
A few possibilities:
a) We have pt-heartbeat set up, and it would be nice to use something like that. We can change how DatabaseMysql::getLag() works using a config flag. I'd prefer this.
b) Use some schema (post-send lazy loading and local memcached) to check slave vs master position/timestamp instead of only using the Mysql slave lag functions which only check the slaves application lag (not the full relay lag).