Any news when to see https://wikimedia.biterg.io/app/kibana#/dashboard/C_Gerrit_Demo deployed 'by default'?
And way more important (as this blocks T167085), is it possible to update the data? Currently the latest "new authors" entry is from July 13th and we need recent names...

In T151161#3176245, @Aklapper wrote:

we are one single boring project. :)

In T151161#3117599, @Aklapper wrote:

In T151161#3115662, @Dicortazar wrote:

1. Add other data sources as Gerrit. You may discuss this with @Lcanasdiaz and give some priority if this is important to your needs.

Thing is: We want to be able to support new contributors already before they finally manage to get their changeset merged. For this, we need to know which Gerrit accounts are new and have provided a first changeset (regardless of the changeset status).

I guess we better close this task. Now, data retrieval for MediaWiki is done with a Perceval backend, which probably gets all the needed information.

Hi. Sorry for the delay in answering.

We would like to follow your plans to migrate to Differential as much as possible. During the last monthly meeting, @Aklapper mentioned (if I'm not wrong) that this was very unlikely to happen during the next three or four months. Given this schedule, it is likely that from the upstream point of view, we support this with the new "version" of our tools, which is starting to be published during the next weeks.

I finally ranked my entry. Sorry for the delay.

Just a minor note. From the point of view of the historical information, what is needed is that "closing transaction" (in Bugzilla, a change of state to closed) with the date of the closing action. That allows for calculating the backlog at any snapshot time in the past, for example.

Finally I didn't have good candidates for the other project, so I resigned and applied to a be a mentor for Wikimedia. Just waiting for approval.

In T116589#1771547, @jgbarah wrote:

It seems I cannot apply for being a mentor for WikiMedia, because I already applied as a mentor for Xen (I'm participating in their program as well). Anybody know to whom this can be reported?

It seems I cannot apply for being a mentor for WikiMedia, because I already applied as a mentor for Xen (I'm participating in their program as well). Anybody know to whom this can be reported?

In T60585#1734314, @01tonythomas wrote:

I am shifting this to Outreachy-Round-11 as the project description has atleast two mentors, micro-tasks and looks ready for the 11th edition of Outreachy ( Dec 2015 - Mar 2016 ) . Potential candidates should start by submitting their proposals as a blocker for this task, by November 02.
Feel free to revert it back, if this task has some relevant issues which might block its completion in this term of Outreachy.

In T89135#1734305, @01tonythomas wrote:

I am shifting this to Outreachy-Round-11 as the project description has two mentors, micro-tasks and looks ready for the 11th edition of Outreachy ( Dec 2015 - Mar 2016 ) . Potential candidates should start by submitting their proposals as a blocker for this task, by November 02.
Feel free to revert it back, if this task has some relevant issues which might block its completion in this term of Outreachy.

In T114437#1732753, @Anmolkalia wrote:

So, I was able to correct the code. This is working fine now.

mediawiki_analysis.py11 KBDownload

,

database.py1 KBDownload

I have solved all the bugs I came across.
I hope this works fine. Do let me know if something else is required.

In T89135#1714299, @Fhocutt wrote:

This looks interesting, and I can help with exploring and working with the MediaWiki API.

In T114437#1728837, @Anmolkalia wrote:

Hi, @jgbarah, I am getting that error if I run the code again, basically when the database already contains the values that are being retrieved. Meaning, the print statements in the except part of insert_page function is giving this error. I'll go through what you sent and get back to you. Thank you for the help.

Yes, if I remove print (pageid+" "+ namespace + " " + title+" was already in the db") and replace it with print (pageid+" was already in the db"), I don't get this problem. The problem is with the encoding of the title, I suppose.

In T89135#1720801, @Aklapper wrote:

In T89135#1720707, @ashitaprasad wrote:

Can you please guide me to gain some traction and contribute to this project.

Hi @ashitaprasad. Thanks for your interest! Please see https://www.mediawiki.org/wiki/Outreach_programs/Life_of_a_successful_project#Answering_your_questions for information on best audiences and successful questions. Thank you! :)

BTW, to easy the process of reviewing the code, maybe you can fork the MediaWikiAnalysis GitHub repository, point me to your fork, and I just clone it. That way I can follow your changes more easily.

In T114437#1717889, @Anmolkalia wrote:

There seems to be a problem with the print statement in the insert_page function, line 101. It seems to be working if I am using only pageid in the print statement.

In fact, after a while, I get an error with your code:

I've run your code, and I don't see that unicode error you see:

I guess we've worked on that in the past. It seems and I think it is already fixed. Checking the code in GrimoireLib:

In T114437#1707946, @Anmolkalia wrote:

Hi, @jgbarah, I am facing a problem in running the original mediawiki_analysis.py. I am writing this in the terminal "./mediawiki_analysis.py --database mwdb --db-user root --url https://en.wikipedia.org/w" and getting "./mediawiki_analysis.py:141: Warning: Out of range value for column 'date' at row 1". What should I do about it. The values in the table seem to be getting updated though

In T114437#1704738, @Anmolkalia wrote:

Hi, @jgbarah, this is the what the mapping file looks like. Let me know if this is fine.

db.py866 BDownload

In T114437#1704569, @Anmolkalia wrote:

Hi, I have a doubt. The dbms we will be using in the backend is till MySQL, right? So, I should continue using MySQL datatypes? Sibyl seems to be using mysql.,

In T60585#1677452, @Simmimourya3107 wrote:

Is there any chance that this project gets proposed for Outreachy-Round-11. I'd like to contribute. I've worked with PHP, Javascript, HTML5, CSS3 and I am quite familiar with Angular framework as well. I've made an app with python flask framework as well. Here's the link to my Github Repository https://github.com/simmimourya1

In T60585#1706581, @Oakshweta11 wrote:

@jgbarah could you suggest what I should do next?

Proposal for GSoC 2015, by Sarvesh Gupta, moved to this comment, to make the description more clear, while preserving the interesting analysis and planning, that maybe somebody can use as inspiration.

In T60585#1706581, @Oakshweta11 wrote:

Hi! I would like to this project for Outreachy December 2015-March 2016
I am familiar with data analytics and machine learning. I have worked on a project to detect skin diseases by analysing images.
I am also familiar with Django and Python. I am developing a website for integration of APIs using Django.
@jgbarah could you suggest what I should do next?

In T60585#1677452, @Simmimourya3107 wrote:

Is there any chance that this project gets proposed for Outreachy-Round-11. I'd like to contribute. I've worked with PHP, Javascript, HTML5, CSS3 and I am quite familiar with Angular framework as well. I've made an app with python flask framework as well. Here's the link to my Github Repository https://github.com/simmimourya1

In T114437#1700543, @Anmolkalia wrote:

In T114437#1699222, @Anmolkalia wrote:

@jgbarah, I'll start with this one. Sounds engaging and I will get to learn plenty from it. So my task here would be to change the code in mediawiki_analysis.py to replace the usage of MySQLdb with SQLAlchemy?

This one needs some knowledge about Python, and will require you to learn about SQLAlchemy. Fortunately, there is plenty of documentation about SQLAlchemy, and the tasks needs little analysis: it is doing the same as the tool does now, but using SQLAlchemy.

In T89135#1696177, @Anmolkalia wrote:

@jgbarah, I went through the microtasks. They sound pretty good to me and sum up all of what we aim to do. So let us start with the one which can be completed in a month's time, because in order to be eligible for Outreachy, I need to finish atleast one microtask by 2nd November.

In T89135#1696592, @Qgil wrote:

@jgbarah, a co-mentor is required. Are Alvaro del Castillo, Daniel Izquierdo (mentioned in the description) on board?

In T89135#1694071, @Anmolkalia wrote:

Also, as for organizing the data in a database, I noticed that the code creates a relational database containing three tables, one for wiki_pages_revs, another for wiki_pages and one for people. What schema we finally choose for our database should depend on what information we think is useful and finally extract. So that will also follow the above suggested documentation process.

In T89135#1689285, @Qgil wrote:

Thank you @jgbarah! Please update the description and propose some microtasks. When this is done, we will promote this project idea to Outreachy-Round-11 candidates.

In T89135#1684438, @Anmolkalia wrote:

Hi @Aklapper. That sounds very encouraging :) I recently made a small contribution to the android Wikipedia App, so I do have some idea of how to start with the development work. As far as this project is concerned, I am especially interested in this because I am very interest in working on Information Retrieval. But I am not able to find documentation pertaining to the tools in question so I am not sure where to begin. Please let me know if you have some pointers on how I should start. Thanks.

In T89135#1684292, @Qgil wrote:

@Acs, @Dicortazar, @jgbarah, do you still want to propose this project? Are you able to mentor it?

In T110947#1655634, @Qgil wrote:

We mean open changesets created in the last three months. We want to look at how well we are reacting to new changesets versus the old ones.

In T110545#1592408, @Aklapper wrote:

In T110545#1592030, @jgbarah wrote:

I couldn't find @Jdlrobson+frankie@gmail.com in our database. Maybe it didn't act?

Probably, might have been just an experiment.

From the list in Group Non-Interactive Users, I tagged gerritpatchuploader@gmail.com, yuvipanda+suchabot@gmail.com, and wikidata-services@wikimedia.de as bots. I couldn't find jdlrobson+frankie@gmail.com in our database. Maybe it didn't act?

Added Contributors | Bots subsection at the Community Metrics wiki with this info.

The list of bots is currently maintained in the Sorting Hat (identities) database, in table profiles. If the field "is_bot" is 1, the identity is considered a bot. Otherwise, that field is 0.

I see three issues here:

In T100189#1581570, @Aklapper wrote:

In T100189#1580920, @jgbarah wrote:

In your case, it seems that you only want to have some organizations "as such" ("Wikimedia Foundation", "Wikimedia Deutschland", "WikiWorks", "Wikia", maybe some others), and the rest to be considered as "Individual" (or "Independent"). Is this right?

I'm not entirely sure what "as such" means, but I'd say that's right.

We've removed a bunch of organizations (see below), but I'm going to explain the situation, just to try to be on the same page.

After looking at the available information, the current list of identities labeled as "bot" is:

Dani is on vacation these days, and he is the most knowledgeable about those PKIs. We are already working on this, anyway. In particular, we're trying to see what's happening with T103292. We'll keep you informed.

In fact, the main reason would be simplifying (and making it more accurate) the retrieving of historical stats of almost any kind. For example, if at some point you're interested in studying the historical evolution of time-to-close, those closing records will be needed.

OK, waiting for @chasemp comments, if any.