Page MenuHomePhabricator

Diffusion commits stuck in 'Importing...
Open, LowPublic

Description

While doing some repoadmin job I notice that commits such as https://phabricator.wikimedia.org/rEDYMe45d9f76bf6a8ac25ceaac48af94be72cbf957bd which are days old are still on "Importing..." status. I have deactivated the rEDYM repo today, but the commit is from a couple of days ago.

I have experienced this issue in other repositories as well. I'll try to find some more examples if required.

Thank you.

Revisions and Commits

Event Timeline

This is due to the gerrit upgrade which created notedb commits in each repo as it is migrating commits and other stuff related to changes (in those projects) into the repo's (which is alot of references)

Yep that is related to that task.

Paladox added a revision: Restricted Differential Revision.Jun 10 2018, 10:34 PM

It appears that the history of rMW is still stuck on importing, and no commits are listed after Monday, June 11. Gerrit correctly reports commits after that date: https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/core/. Is there something that is still missing?

I think @mmodell disabled some stuff (phid?, I don't remember) to help ease the db transactions after the ignoring of the refs/.*/.*/meta ignores. That might be?

As far as I can tell, rMW is importing properly now.

If there are any others which haven't imported properly please update this ticket. Currently I'm not aware of any others.

What's the rationale behind Diffusion importing repositories at all? It was supposed to be a step towards Phabricator-based code review, but that plan was abandoned - maybe it would be simpler to abandon repo mirroring into Diffusion as well?

What's the rationale behind Diffusion importing repositories at all? It was supposed to be a step towards Phabricator-based code review, but that plan was abandoned - maybe it would be simpler to abandon repo mirroring into Diffusion as well?

Off-topic but :) - It was a repo viewer that we hosted (Gerrit didn't have one until recently, other option was Github) and some people still prefer Diffusion over Gitiles.

@mmodell Also rPHEX seems to be stuck (a commit I made and the immediate revert when I noticed it didn't landed at Differential as I expected - sorry).

Vvjjkkii renamed this task from Diffusion commits stuck in 'Importing...' status for too long to 4abaaaaaaa.Jul 1 2018, 1:04 AM
Vvjjkkii triaged this task as High priority.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed subscribers: MarcoAurelio, Aklapper.
Mainframe98 renamed this task from 4abaaaaaaa to Diffusion commits stuck in 'Importing....Jul 1 2018, 6:56 AM
Mainframe98 raised the priority of this task from High to Needs Triage.
Mainframe98 updated the task description. (Show Details)
Mainframe98 added subscribers: MarcoAurelio, Aklapper.

Found another repository with commits still stuck on importing: rMCSN mediawiki-codesniffer.

Found another repository with commits still stuck on importing: rMCSN mediawiki-codesniffer.

Fixed.

What's the rationale behind Diffusion importing repositories at all?

Moved to T359549: Drop our mirroring of code to Diffusion and empty the repos instead

I guess ./bin/repository importing rYourFavRepo and ./bin/repository reparse yourFavCommitId could be a fun game to play one day