As part of the DebMonitor Goal (T191298) I'd like to request some resources in one of our Misc MySQL clusters to be used as the backend for DebMonitor.
The expected usage will be as follows:
From the current structure and data I have in the test instance in labs I have made a ballpark estimation of ~2GB of space on disk needed, but let's consider 5~10GB to be on the safe side. For the details of the tables:
- 1 bridge table will have ~1M rows (# hosts ~1300 multiplied by # installed packages ~770), but is made of 6 int, 2 datetime, 1 varchar that will hold very short strings (0 or 8 char for now with majority of 0).
- 4 tables to keep track of packages and source packages and their respective versions, for which I expect them to be surely smaller than 100k rows each, at least for the foreseeable future.
- all the other tables are negligible.
Given that the query activity will be heavily dependent on the changes in the installed/upgraded/upgradable packages across the fleet, it's hard to make good estimations. In general all queries will be done in small bursts when the script on each host will connect to the DebMonitor server sending it's updates. This is what I was able to predict:
- Normal operation after the initial ramp-up:
- a baseline of ~2,5 SELECT/s; ~0,75 UPDATE/s; 0,015 DELETE/s more or less evenly distributed across the day generated by an apt-get update hook (run before any puppet run) and a crontab that will run the debmonitor CLI once a day per host.
- additional SELECT/INSERT/UPDATE based on how the packages change on the hosts. Let's take a couple of scenarios:
- a package it's added to the base package list and will be installed in all hosts in the next puppet run. As a result in the next 30 minutes there will be an additional ~2,9 SELECT/s and ~0,72 INSERT/s.
- apt-get update detects that an update is available for a package installed on all hosts. As a result in the next 30 minutes there will be an additional ~1,4 SELECT/s and ~0,72 UPDATE/s.
- queries generated by the webUI should be negligible, given that low number of potential users (SRE only).
- Initial ramp-up. We can decide how to spread this in time in order to have it not harm the cluster. Here a couple of possible scenarios:
- just enabling it with the final daily crontab will generate for the first 24h ~46 SELECT/s and ~12 INSERT/s.
- we can spread that across a longer time-period, let's say a week, in this case they will become ~6.7 SELECT/s and ~1.7 INSERT/s.
- If you have any preference I'll adapt the puppet patches accordingly.
Let me know if you need any additional data or have any question/comment