Change Details

Tentatively scheduled for Q3 2016. There are a dozen good reasons to get off of ActiveMQ, but primarily that it's a huge, flaming bag of SPOF. This task is complete once ActiveMQ is uninstalled everywhere, and the replacement system is high-availability. We've done a lot of this work already in DonationInterface, by wrapping queue access in the DonationQueue module, but this epic is still "big" because it involves actually deploying the thing. Deployment outline: * Producers will begin mirroring messages to both ActiveMQ and the new queue. Messages will be set to expire in a month or so. * Incrementally switch queue consumers to consume via Kafka, beginning with the non-critical queues and tools. * Deactivate the mirror to ActiveMQ and retire the box only once we're 100% certain about the new stuff.

There are a dozen good reasons to get off of ActiveMQ, but primarily that it's a huge, flaming bag of SPOF. This task is complete once ActiveMQ is uninstalled everywhere, and the replacement system is high-availability or at least redundant. Must complete this work before roughly the end of August, in order to leave time to stabilize ahead of Big English. We've done a lot of the groundwork already by wrapping php-queue under the DonationQueue module in DonationInterface, so the work here is mostly in generalizing, applying to the remaining components, and then deploying the thing. Deployment outline: * Producers will begin mirroring messages to both ActiveMQ and the new queue. Messages will be set to expire in a month or less, so intermittent failures while we start the new consumers are fine. * Incrementally switch queue consumers to consume via Kafka, beginning with the non-critical queues and tools. Keep an eye on queue usage and set the old message archiver to dump legacy data if storage issues arise. * Deactivate each mirror to ActiveMQ and finally retire the box only once we're 100% certain about the new stuff.

Tentatively scheduled for Q3 2016. There are a dozen good reasons to get off of ActiveMQ, but primarily that it's a huge, flaming bag of SPOF. This task is complete once ActiveMQ is uninstalled everywhere, and the replacement system is high-availability or at least redundant. Must complete this work before roughly the end of August, in order to leave time to stabilize ahead of Big English. We've done a lot of this work already in DonationInterfacee groundwork already by wrapping php-queue under the DonationQueue module in DonationInterface, so the work here is mostly in generalizing, by wrappapplying queue access in the DonationQueue moduleto the remaining components, but this epic is still "big" because it involves actuallyand then deploying the thing. Deployment outline: * Producers will begin mirroring messages to both ActiveMQ and the new queue. Messages will be set to expire in a month or soless, so intermittent failures while we start the new consumers are fine. * Incrementally switch queue consumers to consume via Kafka, beginning with the non-critical queues and tools. Keep an eye on queue usage and set the old message archiver to dump legacy data if storage issues arise. * Deactivate theeach mirror to ActiveMQ and finally retire the box only once we're 100% certain about the new stuff.