|· · ·|
|Resolved||None||T137997 Visualization/data regressions after moving from korma.wmflabs.org to wikimedia.biterg.io|
|Resolved||Aklapper||T28 Decide on wanted metrics for Maniphest in kibana|
|Resolved||Lcanasdiaz||T138002 Deployment of Maniphest panel|
|Resolved||Lcanasdiaz||T138003 Maniphest support to be added to GrimorieLab|
|· · ·|
- Mentioned In
- T147001: Go through default Kibana widgets; decide which ones are not relevant for us and remove them
T116572: Add remaining KPIs to Overview once available in kibana
T28: Decide on wanted metrics for Maniphest in kibana
T105634: Add "Ticket Openers" to equivalent of "Activity by contributors" in kibana
T132421: Identify Wikimedia's most important/used info panels in korma.wmflabs.org
- Mentioned Here
- T161682: Maniphest statistics show implausible numbers of submitted reports
T161519: Maniphest: Parser does not split projects by comma separator?
Guys, we are ready to deploy this but we are finding some issues with the data collection. We do not manage to download more than 600 tickets. Do you have a limitation or do we need a special token? We named it "maniphestbot"
@Aklapper we are getting HTTP 503 Errors - Service unavailable.
I've been testing our tool and we get this error from time to time, not after 600 tasks were fetched. Sometimes it happens before and sometimes after. I could download 1043 tasks with the same process once but most of the times the range is in between 300 and 600.
Are we overloading the server? We are sending 3 HTTP request per second at most.
@Lcanasdiaz: 3 http requests per second doesn't sound too unreasonable, however, how many tasks are you fetching in a single batch? You should make your code fetch small batches (or even one task at a time?) and retry the request when you receive a 503. It shouldn't be too difficult to recover from the errors and continue. With variable load on our database and a strict execution time limit there will always be an occasional 503 when the service is overloaded.
Note to myself: Once this is deployed, compare to https://rust-analytics.mozilla.community/app/kibana#/dashboard/GitHub-Issues-Timing and https://rust-analytics.mozilla.community/app/kibana#/dashboard/GitHub-Issues (which is GitHub though) and https://eclipse.biterg.io/app/kibana#/dashboard/Issues-Backlog and https://eclipse.biterg.io/app/kibana#/dashboard/Issues
@Aklapper we've updated the dashboard today adding finally the information of Maniphest. You can check it in:
@Lcanasdiaz feel free to close the task
I'd have multiple feature requests depending on what's the expected purpose of this statistical tool (I think the main purpose should be to compensate the shortage of reporting / statistics / charts within Phabricator itself compared to e.g. bugzilla), but for now I reported a bug: T161682.