Hi, I am debugging some odd behavior on Montage, what, in short, should not be possible given the code. It looked like a partially-applied transaction.
Poking around, I seem to have found a culprit. I've noticed from our error logs, Montage gets a fair amount of this error: 'Lock wait timeout exceeded; try restarting transaction'. (Performance has been an issue in the past with wmflabs, but that's a separate story.)
This led me to this great in-depth article. In short, though:
The InnoDB documentation clearly says "InnoDB rolls back only the last statement on a transaction timeout by default". In this case, we do not get the transaction atomicity offered by InnoDB.
And I checked the setting:
MariaDB [s53490__montage]> SHOW GLOBAL VARIABLES LIKE 'innodb_rollback_on_timeout'; +----------------------------+-------+ | Variable_name | Value | +----------------------------+-------+ | innodb_rollback_on_timeout | OFF | +----------------------------+-------+
This isn't a setting I can set on a transaction-by-transaction basis. The article I mentioned above talks about how to enable/test the setting, however.
In short, could we please look at getting innodb_rollback_on_timeout enabled for wmflabs?