Page MenuHomePhabricator

Implement support for Flagged Revisions in Pywikibot
Open, MediumPublicFeature

Description

MediaWiki-extensions-FlaggedRevs metadata is not available in Pywikibot.

It would be useful to identify the edition of a Page which has been reviewed, and if a Page has any stable edition.

This project will involve working closely with one Wiki community that has FlaggedRevs implemented, to ensure Pywikibot works correctly on that wiki.

Mentors (two required):

  • Pywikibot: @jayvdb
  • FlaggedRevs community: @Tgr (Hungarian Wikipedia)

https://sourceforge.net/p/pywikipediabot/feature-requests/211 - anonymous - 2009-07-09 20:49:29
https://sourceforge.net/p/pywikipediabot/feature-requests/275 - @Mjbmr

Details

Reference
bz55081

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

@01tonythomas , maybe you know who to contact in WMDE to get some traction on this. c.f. T57081#1737726

For clarity, I am not related to WMDE by anything at this point :D, but @djff I can point you to https://www.mediawiki.org/wiki/Outreach_programs/Possible_mentors where you might be able to find someone you are looking for.

Thanks a lot, I hope to find a mentor for this.

Would there be any interest in mentoring this project for Outreachy Round 15? Application period opens in two days. See a few other ideas for projects that we are promoting for the upcoming round https://www.mediawiki.org/wiki/Outreachy/Round_15

I am interested in mentoring this, but I need to co-mentor it with someone who is highly active in a community that uses FlaggedRevs. As there hasnt been any response from German Wikipedians, I've asked for a co-mentor from the Indonesian Wikipedia community, which I know struggles with their use of this extension. Will let you know if I find anyone.

@jayvdb Hola! Would you be interested in mentoring this project for Google-Summer-of-Code (2018)? If so, do you have someone in mind who would be willing to co-mentor this project along with you? Because in an ideal scenario there be two mentors for a project. If not, should I include this project in my email to Wikitech-I (sometime in the next few days or so) and mention that we are looking for mentors?

I am always interested in mentoring it, but refuse to do it without the support of a real community using FlaggedRevs, via a knowledgeable community member as co-mentor.
We've tried to find one , and failed. (see history of this task) I'll be very happy if you succeed where I have failed.
I've especially tried to solicit Wikimedia Indonesia to provided a co-mentor from the Indonesian community which uses FlaggedRevs , but that hasnt worked in the past either. I will try that again this year, and will also look more broadly in the Indonesian community.

I am always interested in mentoring it, but refuse to do it without the support of a real community using FlaggedRevs, via a knowledgeable community member as co-mentor.

The Hungarian Wikipedia uses FlaggedRevs. I helped introduce it, wrote most of the documentation for it and understand it fairly well on a user level. (Less well on a development level but it's not my first time seeing the code, either.) If you are still interested in this, I would be happy to help. I can't promise spending a lot of time on mentoring but I can answer questions about how things work.

(The use case is that after T121995 we might be interested in using a bot to manage flagged state, e.g. protect BLPs. Also ORES is soon going to be enabled on huwiki which offers some interesting options on automatically deciding which version to show.)

Thanks @Tgr , that use case sounds perfect. I've added you as a mentor, with the expectation you'll mostly be involved in familiarisation at the beginning, and validation of the use cases and checking the final code meets the communities needs.

How do we get started here? What are microtasks?

Left a note on the huwiki bot noticeboard about this.

This message is for students interested in working on this project for Google-Summer-of-Code (2018)

  • Student application deadline is March 27 16:00 UTC.
  • If you have questions about eligibility, please read the GSoC rules thoroughly here https://summerofcode.withgoogle.com/rules/. Wikimedia will not be responsible for verifying your eligibility and also not be able to make any decisions on this. For any clarifying questions, please email gsoc-support@google.com
  • Ensure that by now you have already discussed your implementation approach with your mentors, completed a few bugs/microtasks and made a plan to move forward with the proposal
  • I encourage you to start creating your proposals on Phabricator now to receive timely feedback on them from mentors. Do not wait until the last minute. Give your mentors at least a week's time to review your proposal, so that you could then incorporate any suggestions for changes. Learn how to submit a proposal in our participant's guide: https://www.mediawiki.org/wiki/Google_Summer_of_Code/Participants (Step 9)
  • Proposals that contain links to successfully merged patches before the application period and submitted on both Phabricator and GSoC portal will only be considered for the review process. So, between now and the application deadline, you could consider working on this task.
  • If you would like to chat with me more about the process or have questions, come and talk to me in the Zulip chat: https://wikimedia.zulipchat.com/

Hey @jayvdb @Tgr

I am interested in working on this project.
https://phabricator.wikimedia.org/T184361 : Is this the microtask I need to work on?

Hello I am interest to work on this project. I am Sandip Kumar Bhuyan from India. I was a GSoC 2018 intern for FOSSi Foundation. You can find my work here. I want to contribute to wikimedia organisation this year. So need some stater issue to work to know the code base.

I need the source Repo link to start my work.

@Iamsandipbhuyan Hello, see this search results for good first bugs.

First you should visit the https://www.mediawiki.org/wiki/Manual:Pywikibot/Development basics page. Once ready, Look into the https://www.mediawiki.org/wiki/Manual:Pywikibot/Gerrit Gerrit manual. The detailed descriptions of the steps are also available in https://www.mediawiki.org/wiki/Gerrit/Tutorial. Working with Gerrit repository is quite different than working with GitHub/GitLab repos.

Hello I am Subhang and I would like to contribute under Wikimedia organization . Can I get some assistance as to where to start working on for this idea?

Hello, first of all you should familiarize with our code review system. You could e.g. fix some of the good first bug. Visit the https://www.mediawiki.org/wiki/Manual:Pywikibot/Development basics page. Once ready, Look into the https://www.mediawiki.org/wiki/Manual:Pywikibot/Gerrit Gerrit manual. The detailed descriptions of the steps are also available in https://www.mediawiki.org/wiki/Gerrit/Tutorial. Working with Gerrit repository is quite different than working with GitHub/GitLab repos.

Once ready, you should also read something about Flagged Revisions, especially their API, and also about our API methods.

Xqt changed the subtype of this task from "Task" to "Feature Request".Mar 19 2019, 3:51 PM

Hi @jayvdb !
I'm Abhishek Singh Dhadwal, a second year undergraduate student from Visvesvaraya National Institute of Technology, Nagpur, and I would be delighted to have the opportunity to work on this project as part of GSOC 2019.
How do I start ? What all details should I draft to my proposal ?
And, where can I read more about the subject in hand ?

Hi @Tgr,
I am Jiaxing Wu and I am currently a second year undergraduate from Illinois Institute of Technology. I am interested in working for Wikimedia because I love the idea of promoting free knowledge to everyone in the world (especially since I'm a broke college student). I am particularly interested this project because I have seen a video on youtube where someone tried to edit a wiki page but it was automatically changed back(I am guessing that this was the function for the flaggedrev where the revision, that was highly rated, is set to the default wiki page). As of now, I am researching about flaggedrevs, CMS, and pywikibot.

Could you verify some of the following assumptions that I have about this project to help me clarify what I would need to do for this project:

  • I am mainly working on adding a flagged revision feature in the pywikibot library
  • I would need to contact wiki communities that has already implemented flagged revision to see how they did it.
  • Since the pywikibot lib would be used for many wiki foundation projects, my main contribution would be making it easier for all wiki foundation projects to implement flagged revisions since it will be added into the pywikibot API by me.

-I need to learn how to get the metadata of each revision.

If any of my assumptions are wrong or if there are addition responsibilities, I would be happy to receive some feedback.

https://www.mediawiki.org/wiki/Extension:FlaggedRevs
https://en.wikipedia.org/wiki/Content_management_system
https://www.mediawiki.org/wiki/Manual:Pywikibot/Overview

Hi @Jwu96, I was only a helper on this project in the past (see T57081#3995251) and I probably won't be able to do that this year as I have less free time, I'm trying to run another outreach project, (T218277: Build statistics toolset to support WM-HU editor retention grant) and FlaggedRevs is sort-of-disabled on huwiki now (see T121995: Switch FlaggedRevs on Hungarian Wikipedia to a "flagged protection" mode, although that might get reverted soon, and not sure how much it affects the planned Pywikipediabot API). That doesn't mean much, I'm easy to replace; @jayvdb is the main mentor so you should discuss with him what the plans are for this project (AIUI it was grandfathered in to the task list from some previous year so there's no guarantee the people who planned to mentor it then are still planning to do so).

@AbhishekSinghDhadwal @Jwu96 Hi, just quick note, although I think last year @jayvdb was still helping novices in GCoC and GCI, this year I have seen him online maybe once so far. Unfortunately I also am quite busy this year. You will get some support mainly from me, @Xqt and @D3r1ck01, but don't expect too much. This year Pywikibot is more a project for self-reliant, self-depended and self-governing people, that can work independently most of the time.

To your questions:

  • I am mainly working on adding a flagged revision feature in the pywikibot library: Yes!
  • I would need to contact wiki communities that has already implemented flagged revision to see how they did it: Perhaps more like to see, how it works there, how they use it. The implementation should be similar (if not the same) between wikis.
  • Since the pywikibot lib would be used for many wiki foundation projects, my main contribution would be making it easier for all wiki foundation projects to implement flagged revisions since it will be added into the pywikibot API by me: I would not say this will help wikis to implement FlaggedRevisions, but it would definitely help them to use it after it was implemented. Also note that Pywikibot is not the same thing as MediaWiki API. Pywikibot is just one of the libraries, that works with the MediaWiki API. It can handle many queries nicely, but some, like FlaggedRevisions, are still missing in Pywikibot library. The main contribution is to make FlaggedRevisions easier to work with, after they were implemented I think. I'm not much experienced with FlaggedRevisions, but I know how API and Pywikibot work (and how they work together), perhaps @Tgr could just clarify if I describe FlaggedRevisions correctly. Anyway, please feel free to ask, but also be prepared to do some research on your own to get the answers.

First you can start by setting a bot account on FlaggedRevisions-enabled project. You can try to use Pywikibot on-wiki to solve some typos and easy grammar stuff in wiki articles, maybe try to look into its code a little. Also you should try to understand, how FlaggedRevisions are used on-wiki. After that you could just try to set Pywikibot development repository on your computer and perhaps solve some of the Pywikibot's good first task Once you'll be ready look into the Pywikibot API communication module code and make yourself familiar with it. Learn how the FlaggedRevisions API queries work and then the main task, connect Pywikibot to FlaggedRevisions MediaWiki API. That will be your goal.

I am trying to setting a bot account on FlaggedRevisions-enabled project but I'm not sure how I would get started on that. Is there a guide for this?

Currently, I am just trying to follow the installation process from the project page https://www.mediawiki.org/wiki/Manual:Pywikibot/Overview

You will need to find project, that has FlaggedRevisions enabled. Perhaps it could be some beta project.

Second you will need to create an account on it.

Third you will need to search in its Help or Project pages on how to apply for bot rights (if needed, on some beta projects you don't have to have the rights, perhaps you can ask some admin of the project). Every wiki have different process for this, so pick one wiki and make sure you know, how the bot rights application works there.

Once your user account is ready and you installed successfully Pywikibot, you will need to generate user files (there is a script for it in Pywikibot) and perhaps also generate family file (also a script, some wikis are pregenerated in families folder, some aren't). Then you can try to run some script from scripts, just to try how it works (or shell to test the API).

Then you should be ready.

Fiwiki has FR enabled and it's Flagged revs test pages can be used as for testing. If testing is limited to those those pages I think that it can be done without bot flag and you can ask Flagged Revs permissions for reviewing/unreviewing pages from me (Zache) or if you need stabilization permissions then you can ask those from byrocrat @Ejs-80 . Just reading of review status of the page doesn't need any special permission.

If bot permissions are needed then those can be asked in Fiwiki from

@Dvorapa
When you said "bot rights", did you mean rights to operate a bot like this approval I found?
https://en.wikipedia.org/wiki/Wikipedia:Bots/Requests_for_approval

sure

(PS: I like that you have some testing pages for FlaggedRevisions on Fiwiki @Zache :) )

Yes! I got the pywikibot installed and working. Now I just need the bot approval from en wikipedia :D

Do they on en wikipedia have FlaggedRevisions enabled? (I don't think so, I know only about Finnish and German Wikipedias and some Beta WIkipedias)

Nvm, I checked again and I dont think en has it. I'm going to try with the wikis on listed in this page.
https://en.wikipedia.org/wiki/Wikipedia:Flagged_revisions

Btw, I remembered finding this on some page about the en wiki.

Screen Shot 2019-04-03 at 3.39.50 PM.png (84×1 px, 24 KB)

Per https://en.wikipedia.org/wiki/Wikipedia:Flagged_revisions English Wikipedia made FlaggedRevisions available only for some articles, where Administrator turned FlaggedRevisions on. Therefore I think it may work for some articles on enwiki, but only as an anti-vandalism lock.

Per https://en.wikipedia.org/wiki/Wikipedia:Flagged_revisions English Wikipedia made FlaggedRevisions available only for some articles, where Administrator turned FlaggedRevisions on. Therefore I think it may work for some articles on enwiki, but only as an anti-vandalism lock.

Yes, the English Wikipedia only uses the FlaggedRevs extension for pending changes protection.

There is three basic setups. First two are traditional show latest / stable version by default which means that if revision needs to be reviewed before it is show to anonymous reader by default. This default setting is for whole namespace. Reader can always choose to read latest revision and administrators can toggle between latest / stable case by case. If latest version is shown as default then Flagged revs is mainly used as review manager AND for protecting pages.

Third setup is "protection" mode where flagged revision is used only for small subset of the pages and its user interface is little bit different. There is also some settings which you can only set in protection mode and some only in traditional mode.

Default article revision to show: stable

  • German Wikipedia, Polish Wikipedia ...

Default article revision to show: latest

  • Russian Wikipedia, Finnish Wikipedia ...

Protection mode

  • English Wikipedia ...

Also just to be clear: Deferred changes has not been merged to master branch of Flagged revisions and it is not enabled in enwiki. It is just testwiki stuff currently.

Hi @Tgr,
I am starting to construct my proposal as of now and I am not sure I would go about creating my timeline on deadlines. I have found a page for some components of the FlaggedRev feature. Should I create my deadlines around each of those components?

The project also mentioned about metadata not being available in the pywiki bot so I think I should learn about how metadata works in order to have it available in pywiki bot.

https://en.wikipedia.org/wiki/Wikipedia:Flagged_revisions/fact_sheet

@JJMC89
Is this your bot? :)

Screen Shot 2019-04-05 at 12.34.42 PM.png (374×320 px, 34 KB)

@JJMC89 Is this your bot? :)

Yes, but I don't know what wiki that is.

Language seems Hungarian? Or Turkish? (guess)

I found this as a component for flagged revisions.

Screen Shot 2019-04-06 at 1.04.25 PM.png (758×2 px, 268 KB)

Is the feature that requests for all the user groups' rating already made for pywiki bot?

If it is, I would just have to create the other components for Pywiki bot:
-Special:Stableversions: Lists all revisions of a page that have been marked as stable
-Special:StablePages: List of pages manually configured to show the stable version as the default page to unregistered users (though this can be set as the default)
-Special:Stabilization: Used to change the version of pages that unregistered users see (stable or current)
-Special:Unreviewedpages: List of articles that do not have a stable revision set, or that have revisions that need review
-Special:Reviewedpages: List of articles that have a revision marked as stable
-Special:Log/userrights: The log of all manual rights changes, which would include changes to "editor" and "reviewer" rights
-Special:Log/review: Log of all revision flagging
-Special:Log/stable: Log of all changes to what version of pages is seen by unregistered users

I am assuming that I only need to get my hands on the metadata for the reviews to get started on creating those functions in pywiki bot.

Please let me know if I should be adding that much functions or not. I don't want to overload myself with work that does not help with the problem that this project is trying to solve.

https://en.wikipedia.org/wiki/Wikipedia:Flagged_revisions/fact_sheet

First you should find out, what are the API queries to get these lists and if Pywikibot can handle them or not. I don't think any part of Flagged Revisions is supported by Pywikibot now, but perhaps some may be supported already by coincidence?

@Dvorapa @Xqt please see my message in the Phabricator chat :)

A related discussion took place recently on fawiki. A user suggested that bots should avoid editing pages with an unreviewed revision because this makes the review process more complicated (on fawiki, some pages remain on reviewed for weeks, sometimes months).

I wish we had a clearer plan for working on this.

Due to a lack of interest in mentoring this project, I am removing the OPP tag for now. Folks interested in contributing to the Pywikibot project, check out the recommended tasks here.

Xqt raised the priority of this task from Lowest to Medium.Oct 17 2022, 7:36 PM

increase prio per request

I did some coding.
https://hu.wikipedia.org/wiki/Szerkeszt%C5%91:BinBot/huwiki/flaggedrevs.py is a module for handling FlaggedRevs in Hungarian Wikipedia
Features for end users:
flagged_state(): returns a state code (see class Flags) for a page.
cosmeticable(): tells if a page is suitable for cosmetic changes
CosmeticableFilterPageGenerator(): a wrapper page generator that filters cosmeticable pages. It has two variations: either it handles pages one by one, or takes all the pages from the inner generator, and quries them from API en masse. Perhaps this may be faster.

Because the current soultion does not reflect the API call clearly, I uploaded an erarlier version:
https://hu.wikipedia.org/w/index.php?title=Szerkeszt%C5%91:BinBot/huwiki/flaggedrevs.py&oldid=25979018
If you search for def flagged_state( in this code, you will see the API query.
Special thanks to @Dalba for mining the query (https://www.mediawiki.org/w/index.php?title=Extension%3AFlaggedRevs&diff=3274706&oldid=3214539&diffmode=source).

However please keep in mind that FlaggedRevs may vary from wiki to wiki. Huwiki uses the so called "German approach", see
https://en.wikipedia.org/wiki/Wikipedia:Flagged_revisions#Wikipedia
That means we have flagged namespaces, and once a page in these namespaces gets reviewed for the first time, it will be flagged forever unless we explicitely declare it unreviewed. So my code may not be suitable as is for every wiki. Coding a general solution may be a bit more complicated.

Currently my next problem is how to build this pagegenerator into cosmetic changes, because it does not have a general command line parameter.

First you should find out, what are the API queries to get these lists and if Pywikibot can handle them or not. I don't think any part of Flagged Revisions is supported by Pywikibot now, but perhaps some may be supported already by coincidence?

See my previous comment. I coded querying the existing flags.
API offers other actions, see https://www.mediawiki.org/wiki/Extension:FlaggedRevs#API. They are not included in my module.
Would be nice to see these in _apisite.py, then have a global parameter handled by pwb.