We saw bohrium fail to archive data for the iOS piwik dashboard again recently (March 12 and 13). @elukey said he will invalidate 2018-03-12/13 for iOS data in piwik to force a re-run of the archiver.
Meanwhile, we've all been aware that the volume of request from iOS app is a long-standing issue (T123640#2121263). According to @elukey on IRC today,
the infra on which bohrium/piwik runs now is more stable (the sre team fixed the underlying issue)
Can it handle the volume from iOS app now?
The iOS team also wants to help to alleviate the situation. Here are some thoughts:
- We can increase the dispatch interval from 60 seconds to a higher number (e.g. 120 seconds) so that the tracker would dispatch events less frequently.
- Instead of archiving the data on the fly, we can schedule the cron task once a day or even less frequently, since we normally don't need the real-time data.
- The data has already been sampled at the event level in a 1:10 rate. This has impacted our ability to get insight of app user behavior from smaller wikis. We prefer not to downsample even more.