Upstream issue: https://bugs.chromium.org/p/gerrit/issues/detail?id=16474
Gerrit had it is / partition filed up when we upgraded from 3.4 to 3.5 which was caused by reindexing of changes filing up file diff caches (T323262).
Gerrit stores cache data in H2 database files and two of them are overinflated on disk:
| /var/lib/gerrit2/review_site/cache/ | Size (MB) |
|---|---|
| git_file_diff.h2.db | 8376 |
| gerrit_file_diff.h2.db | 11597 |
There is also a 1GBytes file at /var/lib/gerrit2/review_site/db/account_patch_reviews.h2.db
gerrit show-caches displays smaller usage (around 150M if I get it right):
Name |Entries | AvgGet |Hit Ratio|
| Mem Disk Space| |Mem Disk|
--------------------------------+---------------------+---------+---------+
D gerrit_file_diff | 24562 150654 157.36m| 14.9ms | 72% 44%|
D git_file_diff | 12998 143329 158.06m| 14.8ms | 3% 14%|I would like to inspect the database to figure out what kind of data are there and whether some garbage collection can be achieved to shrink the h2.db files.
Gerrit uses an old version com.h2database:h2:1.3.176. It connects, as understand it, without any username or password using:
this.conn = org.h2.Driver.load().connect(url, null);
I have made a compressed copy of one of the cache on gerrit1001.wikimedia.org at /home/hashar/git_file_diff.h2.db.gz which is only 32MBytes.
I have tried to access the database locally but hitting a wall:
$ /usr/lib/jvm/java-8-openjdk-amd64/bin/java -cp h2-1.3.176-ijar.jar org.h2.tools.Recover Error: A JNI error has occurred, please check your installation and try again Exception in thread "main" java.lang.ClassFormatError: Absent Code attribute in method that is not native or abstract in class file org/h2/tools/Recover
With a more recent version of h2 /usr/lib/jvm/java-8-openjdk-amd64/bin/java -cp h2-2.1.214.jar org.h2.tools.Recover it results in:
-- MVStore CREATE ALIAS IF NOT EXISTS READ_BLOB_MAP FOR 'org.h2.tools.Recover.readBlobMap'; CREATE ALIAS IF NOT EXISTS READ_CLOB_MAP FOR 'org.h2.tools.Recover.readClobMap'; -- LOB CREATE TABLE IF NOT EXISTS INFORMATION_SCHEMA.LOB_BLOCKS(LOB_ID BIGINT, SEQ INT, DATA VARBINARY, PRIMARY KEY(LOB_ID, SEQ)); -- lobMap.size: 0 -- lobData.size: 0 -- Layout -- chunk.1 = chunk:1,block:2,len:1,liveMax:380,livePages:2,map:9,max:410,next:3,pages:4,root:400000f746,time:1a,unusedAtVersion:1,version:1,toc:427,occupancy:0a -- meta.id = 1 -- root.1 = 4000008d50 -- root.2 = 4000002a8e -- root.5 = 8000002ac6 -- Meta -- map.2 = name:_ -- map.3 = name:openTransactions -- map.4 = name:undoLog.1 -- map.5 = name:table.0,key:8fa25204,val:5803b3f1 -- map.6 = name:lobMap,key:8fa25204,val:f4470498 -- map.7 = name:tempLobMap,key:8fa25204,val:59a6a071 -- map.8 = name:lobRef,key:eabe0274,val:436a4e4b -- map.9 = name:lobData,key:8fa25204,val:59a6a071 -- name._ = 2 -- name.lobData = 9 -- name.lobMap = 6 -- name.lobRef = 8 -- name.openTransactions = 3 -- name.table.0 = 5 -- name.tempLobMap = 7 -- name.undoLog.1 = 4 -- Types -- 436a4e4b = org.h2.mvstore.db.NullValueDataType@574caa3f -- 5803b3f1 = org.h2.mvstore.tx.VersionedValueType@5803b3f1 -- 59a6a071 = org.h2.mvstore.type.ByteArrayDataType@59a6a071 -- 8fa25204 = org.h2.mvstore.type.LongDataType@8fa25204 -- eabe0274 = org.h2.mvstore.db.LobStorageMap$BlobReference$Type@eabe0274 -- f4470498 = org.h2.mvstore.db.LobStorageMap$BlobMeta$Type@f4470498 -- Tables ---- Schema SET ---- SET CREATE_BUILD 214; ---- Table Data ---- ---- Schema ---- CREATE USER IF NOT EXISTS "" SALT '' HASH '' ADMIN; DROP ALIAS READ_BLOB_MAP; DROP ALIAS READ_CLOB_MAP; DROP TABLE IF EXISTS INFORMATION_SCHEMA.LOB_BLOCKS;
Essentially there is no data found :-\