Page MenuHomePhabricator

Donations queue consumer should write stats every minute
Open, Needs TriagePublic2 Estimated Story Points

Description

Instead of just writing stats once at the end, it should write the processing rate stats each minute as well as once at the end.

Event Timeline

Change 857074 had a related patch set uploaded (by Ejegg; author: Ejegg):

[wikimedia/fundraising/crm@master] Export donations qc stats every minute

https://gerrit.wikimedia.org/r/857074

Dwisehaupt set the point value for this task to 2.Nov 21 2022, 9:52 PM

As discussed in IRC, the approach in the attached patch (writing every minute plus once at the end of the run) would risk overwriting good statistics before they're published if e.g. the run lasted 61 seconds. There would be a file with stats for the first 60 seconds, that would quickly be overwritten with a file with stats for the last second (much less informative), likely before the first file is published to prometheus. We could mitigate that by making the queue consumer wait for new messages (T323657) so it always runs the full almost-three minutes, but as mentioned on that task we would need an update to the stats collector library to discount waiting time.