Page MenuHomePhabricator

Migrate to NoteDb
Closed, ResolvedPublic

Description

This task is manly for me to track progress for the migration to notedb. Currently this is low priority until we update to 2.15. I am filling this early as bugs have begun appearing due to upstream reading the index for things like external id. This was done as a preparation change to migrate to notedb. Setting autoMograte under [notedb “changes”] will migrate things without needing us to do an offline migration. There is a trial option which will not migrate everything but be able to try notedb before migrating fully.

"* Migrate data to to NoteDb, read from NoteDb, but ReviewDb is still the source of truth ("trial mode")."

See docs https://gerrit-review.googlesource.com/Documentation/note-db.html#online-migration

I think we should go with the full migration instead of trial mode because we can do it online and once it is migrated to notedb we can switch the config to read from notedb.

This will mean increased in storage as the repo's will be increased. This will also make the db redundant.

Though trial mode will write to notedb but still write to both reviewdb and notedb but reads from notedb once migrated. But looking at docs i think this is only supported offline which requires downtime.

Also actually we should do it in 2.15 so we have the chance to decide weather to do it online or offline, 3.x removes online migrations

Also this will require a an init to migrate accounts over to notedb which has to be done offline. Changes can be done online.

Migrating accounts should be quicker offline then doing changes which will require lots of downtime but luckily we can do that online so no downtime required for changes.

Once we have migrated to notedb we need to set

[noteDb "changes"]
    autoMigrate = false
    trial = false
    write = true
    read = true
    sequence = true
    primaryStorage = note db
    disableReviewDb = true

immediately otherwise there is a risk of data loss.

Note that this is for changes only, accounts are migrated to notedb automatically when you upgrade to 2.15.

Event Timeline

Paladox triaged this task as Lowest priority.Aug 24 2017, 2:45 PM

Note that when it's completed the migration it will write to the logs so we will need to pay attention to the logs since it will update gerrit.config but that will be overwritten by puppet.

there will be soon a notedb.config file added upstream to set autoMigrate to false.

We will need to monitor the files carefully over 24 hours when it migrates to change it gerrit.config.

we need to change the notedb section to

[noteDb "changes"]

write = true
read = true
primaryStorage = NOTE_DB
disableReviewDb = true

once done.

We cannot make a notedb file in puppet we have to let gerrit create it once the migration is complete and move the configs to gerrit.config then delete notedb.config.

Paladox raised the priority of this task from Lowest to Low.Oct 27 2017, 1:02 PM

Upstream are looking to removing ReviewDB in the next 6 months. Wont affect any stable release.

Upstream's time table puts reviewdb discontinuation at sometime this year.

When we do the upgrade to 2.15 it wont migrate everything. groups will be migrated when we do the upgrade to 2.16 / 3.0.

Comments are now stored in the repo so it store the history.

Comments are now stored in the repo so it store the history.

Cool, i was always worried that we might lose all the comment history if we ever were to use a different code review tool. Having the comments in the repo sounds nice.

Change 408298 had a related patch set uploaded (by Paladox; owner: Paladox):
[operations/puppet@production] Gerrit: Set notedb configs to enable notedb

https://gerrit.wikimedia.org/r/408298

Paladox raised the priority of this task from Low to Medium.May 11 2018, 4:21 PM
Paladox updated the task description. (Show Details)
Paladox updated the task description. (Show Details)

Mentioned in SAL (#wikimedia-cloud) [2018-05-31T19:16:29Z] <paladox> preparing gerrit-test3 for upgrade to 2.15 (testing T174034 and T177201)

Change 408298 merged by Dzahn:
[operations/puppet@production] Gerrit: Set notedb configs to enable notedb

https://gerrit.wikimedia.org/r/408298

This has been resolved now :).

Groups will be migrated to notedb in T200739