Page MenuHomePhabricator

PHP-queue predis backend is losing data when exit() is call in popAtomic callback
Open, Needs TriagePublic

Description

When we get a DB error like a constraint violation, the message is dropped.

We're wrapping the lpop call that takes the message off the list in a predis MultiExec transaction along with the callback:
https://github.com/CoderKungfu/php-queue/blob/master/src/PHPQueue/Backend/Predis.php#L104

If there's an exception, the predis driver discards the transaction, leaving the message on the queue:
https://github.com/nrk/predis/blob/v1.1/src/Transaction/MultiExec.php#L430

But the exit() being called by Civi seems to break that. Why is the predis driver committing the txn? Is there an autocommit option set someplace?

Event Timeline

Possible upgrade for PHP-Queue: use a lua script to temporarily move data to a different structure while processing a job, then only delete the data from there when finished, a la https://phabricator.wikimedia.org/source/mediawiki/browse/master/includes/jobqueue/JobQueueRedis.php$261