| Peter | |
| Tue, Jul 21, 8:23 AM |
| F95049965: image.png | |
| Tue, Jul 21, 2:45 PM |
| F95049660: image.png | |
| Tue, Jul 21, 2:45 PM |
| F95020128: Screenshot 2026-07-21 at 11.49.22.png | |
| Tue, Jul 21, 9:50 AM |
| F95019720: Screenshot 2026-07-21 at 11.44.07.png | |
| Tue, Jul 21, 9:50 AM |
| F95019641: Screenshot 2026-07-21 at 11.43.56.png | |
| Tue, Jul 21, 9:50 AM |
There seems to be different opinions about how good the current version of Castor works. Let's check the statistics we have and make a summary in this task so we have a good starting point.
Acceptance Criteria:
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T432684 Castor improvements | |||
| Open | Spike | AMarkossyan-WMF | T432705 Figure out the best metric(s) to measure the CI cache improvements | ||
| Open | Peter | T432688 Collect current Castor statistics |
Lets start with some statistics we already have in https://releng-data.wmcloud.org/-/dashboards/ci-by-repo-and-job
This is the time the castor save time:
And how often saves waits on other jobs (not perfect but at least some metrics:
And then builds and failures per repo (to see the overall failure rate for jobs that we then later can compare with Castor failures):
Failure rate for the save workspace job looks like 0,09% using this query. Out of 77614 runs the last 30 days, 72 runs are marked as a failure.
I couldn't find anything more in the database, please check if you can @AMarkossyan-WMF . I'll dig into the XML result files, I think we can get some queuing information out of those.
I did some surface-level research and here are a couple of results:
What percentage of all failures was castor?
Similar data broked down by repo (since the beginning of the year):
The methodology, assumotions and exact SQL queries can be found here.
Added what I seen in the Jenkins XML files in https://phabricator.wikimedia.org/T352319#12145323
For the problem reported in T295351: I checked the last 7 days and it caused 5,2% of the quibble failures (62 out of 1191) and that specific issue caused 53 of those failures). For all jobs, the cache issue was 1,7% of all failures (74 out of 4460).