Missing Git repositories in the list of Gerrit projects (some Git repos are out of the current review process)

I would remove this problem from the scope of this task. Let's count Gerrit repos in the time being.

Hi,

I've started to work on this. In first place I've modified the code of Automator to accept external config files with lists of repositories [1] and later I've added a new temporary repository to store Mediawiki lists [2].

I said temporary because I still do not know if it's a good idea to have this repo in GitHub or maybe it's better to store that in the Wikimedia infrastructure. But let's start with this :).

The good point is that we can all check the evolution of such file and easily keep track of its changes.

[1] https://github.com/MetricsGrimoire/Automator/commit/8646b6587cbdd6a56588047305c01031ba52102b
[2] https://github.com/Bitergia/mediawiki-repositories

Discussed in our meeting:

• Bitergia provides the automated process to gather data from Git/Gerrit and have a list updated every month.
• Wikimedia is responsible to maintain the manual list of repositories to filter out upstream repositories etc.

Some tasks done within this task:

• Octopus, the tool to automatically retrieve data sources information already supports Gerrit.
• Automator, the tool to run the whole machinery already supports that Octopus option.
• Octopus now can export repositories that will feed Automator when retrieving info.

Still in progress:

• Update Korma to use the new versions of Octopus and Automator.
• Let Automator update the list of repositories at https://github.com/Bitergia/mediawiki-repositories

Once this is done, we are supposed to easily change that file with the full list of repositories. I'm starting only with Gerrit since this seems to be the most important data source.

Some updates: there's a new tool in Metrics Grimoire named as 'rremoval' [1] (Repository Removal Tool). This will help to remove those repositories that are not interesting in the analysis.

We still have to integrate the several tools in Automator to automate the whole process.

[1] https://github.com/MetricsGrimoire/rremoval

Some extra steps:

• Octopus is now reliable retrieving information from Gerrit
• Automator is able to deal with information coming from Octopus, removing projects from the blacklist.
• Rremoval now removes projects that are in the blacklist and those that are found in the database but not in the original list of projects (deprecated).

Work to be done:

• Automator should 'git push' each time the file provided when querying gerrit and 'git pull' your changes on the blacklist.

And finally, Automator is now able to fetch and push changes to the git repository. So, as commented, this is just a matter of updating the blacklist to your specific needs, the rest of the info should be automatically updated by Automator (removal of blacklisted projects + deprecated ones).

Closing this task :).

As an example of this behaviour, the last commit by Automator with this respect is available at [1]. There you'll see that three new repositories were added from the previous version found there.

[1] https://github.com/Bitergia/mediawiki-repositories/commit/6201c98564e567603f8b65fe51a153790aad666a