Page MenuHomePhabricator

Fix TreeViews to provide pageviews statistics for all articles of any wikiproject etc.
Open, NormalPublic

Description

James Heilman is the author of the original report, pasted here from https://www.mediawiki.org/wiki/Mentorship_programs/Possible_projects#Ranking_articles_by_Pageviews_for_Wikiprojects_and_Task_Forces_in_Languages_other_than_English

https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Medicine/Popular_pages

Currently we have an amazing tool which every month determine what pages are most viewed for a Wikiproject and then provides a sum of the pageviews for all articles within that project. An example of the output for WikiProject Medicine in English.

The problems is that this tool only exists in English and is running on toolserver rather than Wikimedia Labs. So while we know what people are looking at in English, and this helps editors determine what articles to work on, other languages do not have this ability.

Additionally we are do not know if the topics people look up in English are the same as those they look up in other languages. In the subject area of medicine this could be the basis of a great academic paper and I would be happy to share authorship with those who help to build these tools.

A couple of steps are needed to solve this problem:

For each article within a Wikiproject in English, take the interlanguage links stored at wikidata, and tag the corresponding article in the target language
Figure out how to get Mr. Z's tool to work in other languages https://en.wikipedia.org/wiki/User_talk:Mr.Z-man#Popular_pages_-_please_can_you_make_an_exception.3F . He supposedly is working on it and I am not entire clear if he is willing to have help. Another tool that could potentially be adapted to generate the data is already on Labs http://tools.wmflabs.org/glamtools/treeviews/

Code for existing English Wikipedia bot: https://github.com/alexz-enwp/popularpages


Version: unspecified
Severity: enhancement

Details

Reference
bz54184

Event Timeline

bzimport raised the priority of this task from to Normal.
bzimport set Reference to bz54184.
bzimport added a subscriber: Unknown Object (MLST).
Qgil created this task.Sep 16 2013, 8:32 PM
drdee added a comment.Sep 16 2013, 8:41 PM

Prioritization and scheduling of this bug is tracked on Mingle card https://mingle.corp.wikimedia.org/projects/analytics/cards/1148

I don't know how broad the General/Unknown component of this product is, but this looks out of scope for the analytics team? This depends on bug 42259 (or rather is a duplicate).

drdee added a comment.Sep 19 2013, 2:16 AM

I am also not quite sure if this falls within the scope of the Analytics team. I will ask Sumanah to have a look as well.

Qgil added a comment.Sep 19 2013, 4:24 AM

I didn't file it here because I thought it was a task for the Analytics team, but because it is a feature request related with the area of Wikimedia analytics data. If there is a better place I'm happy to learn about it.

CCing Andre.

Well, http://tools.wmflabs.org/glamtools/treeviews/ mostly does this, but it's broken by the blocker bug. Requests for tools or bots are rarely managed in bugzilla but when they are they can be placed on Wikimedia > General I guess.

In case you refer to existing or yet-to-write tools on Tool Labs there is a "Tool Labs tools" product in Bugzilla.

sumanah wrote:

I think this *could* be something the WMF Analytics team works on and is not obviously out of scope. But the Analytics Product Manager would be the one to decide that, in consultation with their team. If it's not in scope or not in the current TODO list for that team, then James should tell the public https://lists.wikimedia.org/mailman/listinfo/analytics list about the request and ask for a volunteer to help. That is my two cents. :)

Qgil added a comment.Sep 19 2013, 4:38 PM

Just to clarify: my only reason to file this bug is https://en.wikipedia.org/wiki/User_talk:Mr.Z-man#Popular_pages_-_please_can_you_make_an_exception.3F

I have no expertise to decide what technology or team should be used to fix it. :)

Looks like a great candidate for an IEG: https://meta.wikimedia.org/wiki/Grants:IEG

I talked to James about this at Wikimania. I think it's a great idea and I'd be working on it myself if I had time. If this goes forward, I'll help any way I can.

One consideration: How many non-English Wikipedias have active WikiProjects, and active Medicine WikiProjects? Probably fewer than you think. I know that pt.wiki has an INactive Medicine WikiProject, and that's one of our bigger Wikipedias. I think that question needs to be asked and answered in parallel with implementation questions: no use making the tool if no one's going to use it.

That said, I'm not convinced that this is within the scope of ANY Wikimedia teams. This project involves significant data analysis and some intensive scripting (tho probably nothing that requires actual Mediawiki development resources). As well as cross-wiki community organizing. Now, WMF doesn't have any WikiProject support tools in the pipeline, or any concrete plans to help WikiProjects in the near future, or any concrete plans to specifically support collaboration on non-English Wikipedias. As much as I wish that were different...

We have 8 active medicine Wikiprojects. However this data would still be very useful for our translation project were we are translating medical content into about 50 languages. https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Medicine/Translation_task_force/RTT

Will also be useful data to help convince governments to get on board with improving medical content.

James Heilman

Qgil added a comment.Oct 30 2013, 8:35 PM

Is there an interest in proposing this project for Outreach Program for Women?
If so, and if there at least one mentor for it. please move it to the "Featured
projects" section. This way it will be automatically transcluded in
https://www.mediawiki.org/wiki/Outreach_Program_for_Women/Round_7

Thank you!

That would be excellent.

James

Qgil added a comment.Nov 2 2013, 6:57 PM

A technical mentor must step in, though.

vladjohn2013 wrote:

Hi, this project is still listed at https://www.mediawiki.org/wiki/Mentorship_programs/Possible_projects#Ranking_articles_by_Pageviews_for_Wikiprojects_and_Task_Forces_in_Languages_other_than_English

Should this project be still listed in that page? If not, please remove it. If it still makes sense, then it could be moved to the "Featured projects" section if it has community support and mentors.

This project has still not been completed.

Qgil added a comment.Feb 11 2015, 1:44 PM

Wikimedia will apply to Google Summer of Code and Outreachy on Tuesday, February 17. If you want this task to become a featured project idea, please follow these instructions.

@kevinator and others, do you think this task has the volume of work and complexity suitable for GSoC / Outreachy?

(I have pinged James Heilman to join Phabricator and comment here)

I would still love to see this project up and running. I think it has potential not only to increase participation but help direct work towards more highly viewed subjects.

Nemo_bis removed a subscriber: Nemo_bis.Feb 19 2015, 8:29 AM

@Capt_Swing ping. Do you think this task has the volume of work and complexity suitable for a 3-month GSoC/Outreachy project? If yes, would you be willing to mentor?

By the way analysis by Andrew West per this publication has determined that what medical content people look at varies a great deal by language. It was a very unexpected finding.

Paper is open access http://www.jmir.org/2015/3/e62/

Also would love to see mobile added to the popular page tool. Currently it is only desktop views. Mobile now is often over 50% for articles. And mobile data has been available for some time now.

Qgil added a comment.Sep 23 2015, 9:07 AM

This is a message posted to all tasks under "Need Discussion" at Possible-Tech-Projects. Outreachy-Round-11 is around the corner. If you want to propose this task as a featured project idea, we need a clear plan with community support, and two mentors willing to support it.

Qgil added a comment.Sep 23 2015, 9:36 AM

This is a message sent to all Possible-Tech-Projects. The new round of Wikimedia Individual Engagement Grants is open until 29 Sep. For the first time, technical projects are within scope, thanks to the feedback received at Wikimania 2015, before, and after (T105414). If someone is interested in obtaining funds to push this task, this might be a good way.

Also note that Mr.Z-Man, the author of the bot that updates the reports on en.wiki seems to have retired from Wikipedia, so it may be necessary to create a new solution for English Wikipedia as well at some point.

kaldari updated the task description. (Show Details)Oct 9 2015, 10:50 PM
kaldari set Security to None.
kaldari updated the task description. (Show Details)
Nemo_bis renamed this task from Pageviews for Wikiprojects and Task Forces in Languages other than English to Fix TreeViews to provide pageviews statistics for all articles of any wikiproject etc..Jan 19 2016, 8:55 AM
Nemo_bis edited projects, added Tools; removed Analytics-General-or-Unknown.

I've clarified the summary so that it's clear what pages we're talking about, what languages (all) and what output (outside the wikis is fine, so it's not a duplicate of T43327: Add page views graph(s) to MediaWiki's info action for Wikimedia wikis).

Now that the blocking bug is fixed and baglama was updated, it's rather trivial to change treeviews code so that it can list such pages and use the pageviews API.

Kelson added a subscriber: Kelson.Jan 31 2016, 5:26 PM

Having this would be really helpful for the WP1/Kiwix selections. Now that we have more or less achieved to make periodically ZIM files of all Wikimedia projects, we (newly) focus on selections. Kiwix is as a consequence part of a relaunching effort of EN WP1 selection project. Until now we have achieved to get these lists by parsing http://stats.grok.se/ files but this is really too long and consume a lot of hardware/bandwidth resources. It's really not easy to manage. That's why I strongly support this feature request.

Thanks Kelson & All for your work here! We do need this as we build up educational/offline extracts of Wikipedia for schools. In particular the ~4 billion people who cannot possibly afford Internet, and yet have excellent client devices like OLPC's, cheapo Androids, PC's etc. Thankfully 32G and 64G SD cards are really beginning to save the day here...

Bzg added a subscriber: Bzg.Feb 6 2016, 8:29 PM
Bzg added a comment.Feb 6 2016, 8:32 PM

Anything that helps Kiwix helps several education organizations I know.

Sumit added a subscriber: Sumit.Mar 1 2016, 5:37 PM
IMPORTANT: This is a message posted to all tasks under "Need Discussion" at Possible-Tech-Projects. Wikimedia has been accepted as a mentor organization for GSoC '16. If you want to propose this task as a featured project idea, we need a clear plan with community support, and two mentors willing to support it.
Qgil removed a subscriber: Qgil.Mar 2 2016, 10:22 AM
Niharika removed a subscriber: Niharika.Mar 2 2016, 7:00 PM
Sumit added a comment.EditedSep 11 2016, 4:23 PM

Porting this to labs looks like an interesting thing! Can we have this project for the current round of Outreachy-13 3-month internship?

Looking forwards to seeing this solved :-)

srishakatux added a subscriber: srishakatux.

Adding Outreach-Programs-Projects and removing Possible-Tech-Projects as we are planning on killing the later soon!

Aklapper removed Upeksha1996 as the assignee of this task.Nov 18 2018, 2:46 PM

@Upeksha1996: I am resetting the assignee of this task because there has not been progress lately (please correct me if I am wrong!).
Resetting the assignee avoids the impression that somebody is already working on this task. It also allows others to potentially work towards fixing this task.
Please claim this task again when you plan to work on it (via Add Action...Assign / Claim in the dropdown menu) - it would be welcome! Thanks for your understanding!