Page MenuHomePhabricator

gerrit: config tweaks
Open, LowPublic

Description

Some settings could be tweaked on Gerrit, on Debian and on the JVM, this task is here to track those tweaks.
Apologies if I'm unaware of historical reasons behind some parts of the existing configuration, I'll be happy to learn about them.

The steps would be:

  • create a test platform with the current gerrit settings
  • create a benchmark scenario

compare performance with/without:

  • transparent huge pages → madvise on Debian + -XX:+UseTransparentHugePages on the JVM
  • jgit's core.packedGitLimit tweaking
  • more to come

Event Timeline

re: test platform - that is also T363196 (T329444, T240933, T256563, .. ) - there is/was a gerrit test instance in devtools. puppetized.. so using the same config. but it was shut down because using the LDAP for user auth there is against policy, afair.

LSobanski moved this task from Incoming to Backlog on the collaboration-services board.
  • transparent huge pages → madvise on Debian + -XX:+UseTransparentHugePages on the JVM

It is barely ever stalling on memory, I don't think it is worth optimizing. See Grafana query for memory Pressure Stall Information.

  • jgit's core.packedGitLimit tweaking

That was set to 4G in 2019. The cache can be monitored at https://grafana.wikimedia.org/d/8YPId9hGz/jgit-block-cache?orgId=1&from=now-30d&to=now&timezone=utc&var-instance=gerrit.wikimedia.org:443 . It has a a miss rate and eviction rate around 0.135% which shows we hold all the pack files in memory. Looks like 4G is good enough for us.

I don't think they need any tweak.


For the second part of this task "create a test platform", I don't think we can reliably reproduce the traffic we have in production. If we wanted to tweak a value it is easier to do it directly in production as I have done multiple time:

  • find a metric to monitor
  • apply the change in production
  • verify the metric

That saves us the hassle of setting up, and maintaining, a dev instance.