@jhsoby this is a nice idea, and I am interested to collaborate on this. I will try to think some ideas of how we can approach this before the hackathon begins.

@KCVelaga Awesome! I see you list SQL as one of your skills, and that's perfect, because that's a big gaping hole in my skills, and definitely needed for this. 😊

@jhsoby as I am trying to do some initial data exploration. I have some questions:

1. When a new wiki is created, do the pages and edits related to that wiki while on incubator get deleted or are they kept? This will be helpful filter out edits from the revision table by rev_deleted parameter. From what I understand, we will only need the dashboard for wikis are currently incubating, but not all historically. We will need some initial way of filtering down to edits belonging to current wikis rather than loading the entire edits data. We possibly do with REGEX from page-title, just wondering if there is an easier way.
2. Any thoughts on how we can remove incubator related edits, such as edits related to actual incubator related pages such as main page, FAQ etc.
3. About stats to be collected: how about adding, number of pages as well?
4. do you have specific endpoints/frameworks in mind for the dashboard?

Also, about live/monthly, depending on how big of a dataset we end up, we can setup a weekly, bi-weekly or monthly update.

1. They are deleted. See I:SCL for the list of "ex-test-wikis".

2. Those are all in the Project or Help namespaces. Vice versa, test-wiki content is only found in the main, template, category and module namespaces (and their talk namespaces).

Maybe this query: https://quarry.wmcloud.org/query/61436 (counting raw number of actions) is a helpful reference.

In T307147#7900300, @KCVelaga wrote:

@jhsoby as I am trying to do some initial data exploration. I have some questions:

1. When a new wiki is created, do the pages and edits related to that wiki while on incubator get deleted or are they kept? This will be helpful filter out edits from the revision table by rev_deleted parameter. From what I understand, we will only need the dashboard for wikis are currently incubating, but not all historically. We will need some initial way of filtering down to edits belonging to current wikis rather than loading the entire edits data. We possibly do with REGEX from page-title, just wondering if there is an easier way.
2. Any thoughts on how we can remove incubator related edits, such as edits related to actual incubator related pages such as main page, FAQ etc.
3. About stats to be collected: how about adding, number of pages as well?
4. do you have specific endpoints/frameworks in mind for the dashboard?

Also, about live/monthly, depending on how big of a dataset we end up, we can setup a weekly, bi-weekly or monthly update.

Wonderful!

1. Deleted, as MF-Warburg said, so we could safely disregard deleted edits.
2. What MF-Warburg said.
3. Yeah, number of pages would be good to have!
4. Nothing specific, no, I am unfortunately not familiar with any specific frameworks for this – if I'd been all alone in this, I would have made something (most likely substandard, hehe) from scratch, so anything other than that is probably better. ;-)

Also, I think monthly updates would be fine, since we only really look at monthly stats anyways.

@MF-Warburg and @jhsoby; somehow I missed both your messages. Thank you for the information, that is helpful.

I am an attending the local meetup in India. A couple of the participants are interested, along with me.

I am thinking of the following steps broadly

• Build the required query
• Build required charts and outputs
• Setting up on Toolforge (cron job, json storage etc.)
• Buidling and deploying the dashboard. I am thinking we can use https://dash.plotly.com/

Let me know what you think, and if you have any suggestions.

@KCVelaga Can we meet in the Workadventure space to discuss? :-) I'll be in the "Localization and small wikis" room when I'm online (like now).

Here is a query that gives output of the required metrics for all incubating wikis: https://quarry.wmcloud.org/query/64738