Page MenuHomePhabricator

Diffusion commits stuck in 'Importing...
Open, Needs TriagePublic

Description

While doing some repoadmin job I notice that commits such as https://phabricator.wikimedia.org/rEDYMe45d9f76bf6a8ac25ceaac48af94be72cbf957bd which are days old are still on "Importing..." status. I have deactivated the rEDYM repo today, but the commit is from a couple of days ago.

I have experienced this issue in other repositories as well. I'll try to find some more examples if required.

Thank you.

Details

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 10 2018, 5:05 PM

This is due to the gerrit upgrade which created notedb commits in each repo as it is migrating commits and other stuff related to changes (in those projects) into the repo's (which is alot of references)

Yep that is related to that task.

Prod added a subscriber: Prod.Jun 14 2018, 7:59 PM

It appears that the history of rMW is still stuck on importing, and no commits are listed after Monday, June 11. Gerrit correctly reports commits after that date: https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/core/. Is there something that is still missing?

I think @mmodell disabled some stuff (phid?, I don't remember) to help ease the db transactions after the ignoring of the refs/.*/.*/meta ignores. That might be?

As far as I can tell, rMW is importing properly now.

If there are any others which haven't imported properly please update this ticket. Currently I'm not aware of any others.

Tgr added a subscriber: Tgr.Jun 18 2018, 9:20 PM

What's the rationale behind Diffusion importing repositories at all? It was supposed to be a step towards Phabricator-based code review, but that plan was abandoned - maybe it would be simpler to abandon repo mirroring into Diffusion as well?

greg added a subscriber: greg.Jun 18 2018, 9:25 PM

What's the rationale behind Diffusion importing repositories at all? It was supposed to be a step towards Phabricator-based code review, but that plan was abandoned - maybe it would be simpler to abandon repo mirroring into Diffusion as well?

Off-topic but :) - It was a repo viewer that we hosted (Gerrit didn't have one until recently, other option was Github) and some people still prefer Diffusion over Gitiles.

Majr added a subscriber: Majr.Jun 19 2018, 9:14 AM

@mmodell Commits from the 9th (rMWe0fab761f34c7c10ba33f0ead4969fe981ce9aa4) to the 12th (rMW67ba3f31ed5ed30c0c08bb5a75bb098e3ffbe824) seem to still be stuck on importing.

@mmodell Also rPHEX seems to be stuck (a commit I made and the immediate revert when I noticed it didn't landed at Differential as I expected - sorry).

Ltrlg added a subscriber: Ltrlg.Jun 25 2018, 6:53 AM
Vvjjkkii renamed this task from Diffusion commits stuck in 'Importing...' status for too long to 4abaaaaaaa.Jul 1 2018, 1:04 AM
Vvjjkkii triaged this task as High priority.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed subscribers: MarcoAurelio, Aklapper.
Mainframe98 renamed this task from 4abaaaaaaa to Diffusion commits stuck in 'Importing....Jul 1 2018, 6:56 AM
Mainframe98 raised the priority of this task from High to Needs Triage.
Mainframe98 updated the task description. (Show Details)

Found another repository with commits still stuck on importing: rMCSN mediawiki-codesniffer.

Found another repository with commits still stuck on importing: rMCSN mediawiki-codesniffer.

Fixed.

Late, but I found another stuck commit: rPHDEP98d9126923d5, from rPHDEP Phabricator Deployment.