Change Details

Status Quo: * we have several (about 5 to 10) instances of dispatchChanges.php running as cron jobs concurrently. * dispatchChanges.php picks the most lagged client by looking at wbchanges_dispatch, grabs a lock representing that client, and pushes updates to it. On success, wbchanges_dispatch is updated and the lock released. * Locking is done using the Database::lock method, which translates to MySQLs GET_LOCK. The lock is bound to the DB connedction. * Each run of dispatchChanges.php may make several dispatch passes, picking and locking a client wiki each time. The Problem: * Each dispatchChanges.php instance (about 10) holds a connection to the master DB for several minutes. (How much of a problem is this, really?) * We cannot close the connection and re-connect, since we would lose the lock. Ideas: * Rely on information written to wbchanges_dispatch. But how to prevent starvation due to stale locks, left by a failed dispatch pass or dead process? High timeout values cause starvation, low timeout values cause false negatives. * Use some other locking mechanism. We have the LockManager base class with several implementations, including Memc, Redis, file system, and MySQL. * Put wbchanges on a separate DB system. But running a separate server for a singe DB table with 1000 rows and 10 concurrent connections seems overkill. Caveat: Switching from one locking mechanism to the other requires careful coordination. Execution of dispatchChanges.php must be halted and then re-started with the new configuration, without causing more than a few minutes lag for client wikis.

Status Quo: * we have several (about 5 to 10) instances of dispatchChanges.php running as cron jobs concurrently. * dispatchChanges.php picks the most lagged client by looking at wbchanges_dispatch, grabs a lock representing that client, and pushes updates to it. On success, wbchanges_dispatch is updated and the lock released. * Locking is done using the Database::lock method, which translates to MySQLs GET_LOCK. The lock is bound to the DB connedction. * Each run of dispatchChanges.php may make several dispatch passes, picking and locking a client wiki each time. The Problem: * Each dispatchChanges.php instance (about 10) holds a connection to the master DB for several minutes. (How much of a problem is this, really?) * We cannot close the connection and re-connect, since we would lose the lock. Ideas: * Rely on information written to wbchanges_dispatch. But how to prevent starvation due to stale locks, left by a failed dispatch pass or dead process? High timeout values cause starvation, low timeout values cause false negatives. * Use some other locking mechanism. We have the LockManager base class with several implementations, including Memc, Redis, file system, and MySQL. * Put wbchanges on a separate DB system. But running a separate server for a singe DB table with 1000 rows and 10 concurrent connections seems overkill. NOTE: Switching from one locking mechanism to the other requires careful coordination. Execution of dispatchChanges.php must be halted and then re-started with the new configuration, without causing more than a few minutes lag for client wikis.

Status Quo: * we have several (about 5 to 10) instances of dispatchChanges.php running as cron jobs concurrently. * dispatchChanges.php picks the most lagged client by looking at wbchanges_dispatch, grabs a lock representing that client, and pushes updates to it. On success, wbchanges_dispatch is updated and the lock released. * Locking is done using the Database::lock method, which translates to MySQLs GET_LOCK. The lock is bound to the DB connedction. * Each run of dispatchChanges.php may make several dispatch passes, picking and locking a client wiki each time. The Problem: * Each dispatchChanges.php instance (about 10) holds a connection to the master DB for several minutes. (How much of a problem is this, really?) * We cannot close the connection and re-connect, since we would lose the lock. Ideas: * Rely on information written to wbchanges_dispatch. But how to prevent starvation due to stale locks, left by a failed dispatch pass or dead process? High timeout values cause starvation, low timeout values cause false negatives. * Use some other locking mechanism. We have the LockManager base class with several implementations, including Memc, Redis, file system, and MySQL. * Put wbchanges on a separate DB system. But running a separate server for a singe DB table with 1000 rows and 10 concurrent connections seems overkill. Caveat: NOTE: Switching from one locking mechanism to the other requires careful coordination. Execution of dispatchChanges.php must be halted and then re-started with the new configuration, without causing more than a few minutes lag for client wikis.