Page MenuHomePhabricator

Status of analytics/limn-*-data git repositories?
Closed, ResolvedPublic

Description

In 2015:

we will focus on phasing out limn next year, as part of the project code named {frog}

I guess limn has been phased out nowadays but we still have a few Gerrit repositories opened for it and I would like to archive them as well as delete the related github mirrors.

Up-to-date list on Gerrit is https://gerrit.wikimedia.org/r/#/admin/projects/?filter=limn there are still a few git repositories open though:

RepoStatus?
analytics/limn-analytics-data
analytics/limn-edit-data
analytics/limn-ee-data
analytics/limn-extdist-data
analytics/limn-flow-data
analytics/limn-language-data
analytics/limn-mobile-data
analytics/limn-multimedia-data

Can we archive all of them? :]

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 16 2019, 9:40 AM

hashar: these actually don't have anything to do with limn, they all hold configuration for reportupdater reports. We would rename them but that's not possible in gerrit, right? We could start a conversation about centralizing all of the reportupdater reports into one repo, each set of reports lives in its own folder so that would work fine. This is probably a decision for Product-Analytics, we'll talk it over with them.

fdans assigned this task to Milimetric.Apr 18 2019, 4:23 PM
fdans triaged this task as High priority.
fdans moved this task from Incoming to Ops Week on the Analytics board.

hashar: these actually don't have anything to do with limn, they all hold configuration for reportupdater reports. We would rename them but that's not possible in gerrit, right?

Indeed Gerrit does not have a built in way to rename repository. We should be able to manually rename one but we have never committed time to figure it out (in short: stop gerrit, move the git repo, do some sql updates, start gerrit).

We could start a conversation about centralizing all of the reportupdater reports into one repo, each set of reports lives in its own folder so that would work fine.

Then if you want to merge those repositories, that is "easily" doable. For each repositories, make sure there is no overlap in files/paths. For example have all content moved into a folder. In the new repository, add remote for each of the repositories to merge then do a git merge of all the branches. Bim merged!

mkdir bla && cd bla && git init .

# Add all repositories as remotes
for i in analytics edits ee extdist flow language mobile multimedia; do git remote add $i https://gerrit.wikimedia.org/r/analytics/limn-$i-data.git; done;
git fetch --all

# Create an empty branch
git checkout --orphan octopus

# Start with an initial commit, usually the one that adds .gitreview file
git commit --allow-empty -m 'Initial commit'

# Merge all remote branches in, allowing different histories that have nothing in common
git branch -r|xargs git merge --allow-unrelated-histories

Assuming there are no conflicts, the end result is a merge commit of all the branches. That can then be pushed to the new Gerrit repo. And blam all the histories are kept and merge :] I am more than willing to assist doing that.

\o/

We decided to merge the repositories into our main reportupdater-queries repository. We will use that going forward in lieu of separate repos. Some of the old repos still have other team-specific logic that we won't be merging and we'll be leaving it up to those folks to figure out what they want to do with it. This is not high priority for us, however, so do ping us if it needs to be.

hashar closed this task as Resolved.May 3 2019, 10:42 AM

Thank you Dan! I just thought all those repositories got forgotten and did not serve any purposes nowadays. I am happy to see Analytics still care of those and have a plan to eventually unify them in a single repository. So that fulfill this task inquiry :]

Whenever the repositories are no more needed, we can just get a new task against Cleanup and we will be able to archive them in Gerrit, delete them from Github etc.

Thank you @Milimetric !

Change 517084 had a related patch set uploaded (by Fdans; owner: Fdans):
[analytics/reportupdater-queries@master] Consolidate limn* repo queries used by report updater jobs

https://gerrit.wikimedia.org/r/517084

Change 517085 had a related patch set uploaded (by Fdans; owner: Fdans):
[operations/puppet@production] ReportUpdater: change repo of all queries to reportupdater-queries

https://gerrit.wikimedia.org/r/517085

@fdans: I think we need to communicate to repo owners this move, would you please take care of doing that? Also, once executed (which I think will require manual intervention as puppet needs to reclone all depos) let's please remember to archive the depos no longer used.

Change 517084 merged by Mforns:
[analytics/reportupdater-queries@master] Consolidate limn* repo queries used by report updater jobs

https://gerrit.wikimedia.org/r/517084

To get the Gerrit repositories archived one can fill a task for all of them against Cleanup . They will then be emptied up, marked read-only in Gerrit and the github mirrors removed.