It'd be great if there is a work-around that we can give to users that want to show maps - or to let them know how to use maps when they're using flagged revisions extensions.

Either way, we should have a conversation with RU and DE wikipedias to get their feedback.

I tried searching for it, but do we know which projects are using the Flagged Revisions extension? I'm assuming this will impact which communities we reach out to (and I'd hate to accidentally reach out to one using the extension by accident).

The full list: https://github.com/wikimedia/operations-mediawiki-config/blob/master/dblists/flaggedrevs.dblist

Note that depending on settings and local usage practice, pages can be not under FR even if it's enabled. Use Special:StablePages to find out, e.g. https://en.wikipedia.org/wiki/Special:StablePages

Clarification: 22 wikis with FlaggedRevs enabled that we haven't launched yet:

alswiki, arwiki, bewiki, bnwiki, bswiki, cewiki, ckbwiki, dewiki, enwiki, eowiki, fawiki, hiwiki, huwiki, iawiki, idwiki, kawiki, plwiki, ptwiki, sqwiki, trwiki, ukwiki, vecwiki

The Collaboration team doesn't normally work on FlaggedRevs, but we are doing some code review for it right now, and @Catrope is working with @Cenarium who works on it.

Does this mean that <mapframe/link> does not work at all on old revisions, or that it works but requires a re-parse of the content ?

Hey @MaxSem, @Yurik, as Kartographer would solve one of our tasks from the German-Community-Wishlist, we would like to offer support with this task. Is there anything specific our team could do?

The problem is with map data that's not shown by default:

• For current revision, it gets stored in page_props.
• The problem is with old revisions that have data different from latest. These hashes are nowhere to be found. FlaggedRevs is one of the most popular ways people get exposed to this problem.
• The solution is T119043: Graph/Graphoid/Kartographer - data storage architecture -however there are just too many icky things about this to implement it straightforwardly.

This appears to also be an issue for the graph extension then.
A revisioned page_props would not work, since transcluded content can change (which FlaggedRevs can handle in different ways, see $wgFlaggedRevsHandleIncludes).

And we had a similar issue with lazy loading of references. Using page_props to store references data was explored but it turned out to have the same limitations.
Another idea was to store the references data in the parser cache (or more precisely, a dedicated parser cache, regenerated during parses), since FlaggedRevs has its own stable parser cache, and have the API get the data from the parser cache.

Why can't we use the parser cache as storage? Is the data not regenerated at each parse?

There's no good way to correlate between a certain mapdata hash and a cached parser output. Furthermore, "parser cache" means that there's no guarantee that the needed parser output will even be here.

The parser cache might not be there but then it can fall back to the persistent storage, page_props. As for correlating between a hash and a parser cache, I don't understand because there's only one parser cache for the current revision, and one for the stable revision.

Moving off the sprint board - the Discovery team won't be able to do this work at this time.

For the record, can someone describe what the unacceptable behavior would be if someone put a mapframe on an article in a Flagged Rev wiki? What about maplink? What would users see? (I just put a map in my discussion page in ru.wiki, but I suppose those are not subject to Flagged Revs?)

In T151665#3968226, @jmatazzoni wrote:

For the record, can someone describe what the unacceptable behavior would be if someone put a mapframe on an article in a Flagged Rev wiki? What about maplink? What would users see? (I just put a map in my discussion page in ru.wiki, but I suppose those are not subject to Flagged Revs?)

No, they are not. (Frankly, I don’t really understand what’s the supposed problem here, is Maps extension showing different information based on the revision?)

@MaxSem explained to me what the problem is here.

The GeoJSON for things to paint onto the map is not present in the HTML output by the parser. Instead, it's stored in the page_props table, and the parser HTML is something like <a class="mw-kartographer-map mw-kartographer-link" data-overlays="[&quot;_520367c8201e0dc8ebf9c16addd56b28115f8845&quot;]" style="background-image: url(https://maps.wikimedia.org/img/osm-intl,6,37.7533,-122.4248,350x300.png?domain=sv.wikipedia.org&amp;title=Foo&amp;groups=_520367c8201e0dc8ebf9c16addd56b28115f8845); height: 300px;" ...>...</a>. The long hash is called a group ID. Kartotherian receives this group ID, and calls back into Kartographer (using the MW API) to ask for the GeoJSON for that group ID. Kartographer then retrieves that GeoJSON from the page_props table.

If the page is then edited and the GeoJSON is changed, the page_props table will be updated and the group ID will change. The old GeoJSON will no longer be in the table, and looking up the old group ID will fail. You won't easily be able to reproduce a bug here, though, because Varnish will cache Kartotherian's response to the maps.wikimedia.org URL with the old group ID. However, if you open that URL in a new tab and hard-refresh, you'll see an error ("GroupId not available").

This is a general issue when viewing a historical version (i.e. any version other than the latest version) of any page on any wiki, but on non-FlaggedRevs wikis that's not something that people do very often. On FlaggedRevs wikis it happens very frequently: users who don't have review rights are not shown the latest version by default, but instead get the reviewed/approved version of the page. This may be the latest version, but if there are unapproved edits it won't be: the latest version will be unapproved and the approved version will be historical. (Any user can view the latest unapproved version, but non-reviewers have to click a link to get there, they're initially taken to the approved version.)

@MaxSem has an abandoned WIP patch to store all historical versions of GeoJSON: https://gerrit.wikimedia.org/r/#/c/291300/

Thanks for the technical desription @Catrope. Can you describe what the user experience is? E.g, if there is a mapframe map on a page, and a user without the requisite rights to be pre-approved saves a change:

• To the page with the map on it.
• To the mapframe map.

What do subsequent viewers of that page see?

In T151665#4045154, @jmatazzoni wrote:

Thanks for the technical desription @Catrope. Can you describe what the user experience is? E.g, if there is a mapframe map on a page, and a user without the requisite rights to be pre-approved saves a change:

• To the page with the map on it.
• To the mapframe map.

What do subsequent viewers of that page see?

• If the map was not changed between the approved (older) version and the latest (unapproved) version
  • Everything should be fine
• If the map was changed
  • When a reader views the approved version (this is what most readers see by default):
    • The map will probably break; if/when if does, the user will see a grey rectangle instead of the map
    • The breakage won't happen immediately, it will probably take up to an hour
  • When a reader views the latest version (normal readers have to click through to find this; privileged readers see this by default)
    • The map will work

In T151665#2924526, @Yurik wrote:

Clarification: 22 wikis with FlaggedRevs enabled that we haven't launched yet:

alswiki, arwiki, bewiki, bnwiki, bswiki, cewiki, ckbwiki, dewiki, enwiki, eowiki, fawiki, hiwiki, huwiki, iawiki, idwiki, kawiki, plwiki, ptwiki, sqwiki, trwiki, ukwiki, vecwiki

BUT...

In T151665#2882127, @MaxSem posted a github list of 52 wikis with Flagged Revs.

I'm writing a consultation to tell the NON Flagged-Rev wikis for whom we plan to implement mapframe that this is going to happen. I want to link to a list of the wikis that WILL NOT be getting mapframe.

So what's the difference between Max's list of 52 and Yurik's list of 22? Is it that Max's includes projects besides Wikipedias? Or is one of these more up to date? Bottom line, if I want to say that we're going to bring mapframe to all but the X wikipedias which have Flagged Revisions, where can I get that list of Flagged Rev wikipedias?

In T151665#4105199, @jmatazzoni wrote:

if I want to say that we're going to bring mapframe to all but the X wikipedias which have Flagged Revisions, where can I get that list of Flagged Rev wikipedias?

CommonSettings.php has a section "Per-wiki config for Flagged Revisions" which implies that the list of wikis is at flaggedrevs.php.

@jmatazzoni both lists are correct. I listed just the Wikipedias, whereas @MaxSem listed all other projects too. Note that mapframe is already enabled in all those other projects, and, to my knowledge, noone had any issues. Also note that the <graph> tag uses identical system of data storage to maps, and it has been enabled on all wikis for many years, and noone has raised any major issues either. So while flagged rev issue is real, in reality it seems to affect almost noone. For example, flaggedrev is enabled on enwiki, but there are very limited number of actual articles that enable it.

Update: more specifically, on English Wikipedia, flagged revs is mostly enabled on biographies of living people (not very map related) - see https://en.wikipedia.org/wiki/Wikipedia:Flagged_revisions -- which means English wiki doesn't have to be prevented from mapframe on the account of flaggedrevs.

Probably for the better to link (and update if necessary) to page on Meta and make a clarification that if mapframe is already enabled, nothing will happen to those projects:
https://meta.wikimedia.org/wiki/Flagged_Revisions#Flagged_Revisions_on_Wikimedia_projects

In T151665#4105541, @Yurik wrote:

Update: more specifically, on English Wikipedia, flagged revs is mostly enabled on biographies of living people (not very map related) - see https://en.wikipedia.org/wiki/Wikipedia:Flagged_revisions -- which means English wiki doesn't have to be prevented from mapframe on the account of flaggedrevs.

No worries on that score Yurik. We've already announced plans to release mapframe to en.wiki in April. Our plan is to wait until after we put the internationalization fix in place.

And thanks for clarifying about the lists. I'll link to your list in my consultation with the rest of the Wikipedias.

Thanks @stjn for showing me where this is cataloged on meta. I compared your list to @Yurik's and found these discrepancies. Does anyone know how I can get an accurate list?

On the meta list but not on Yurik's

• ru—Russian
• mk-Macedonian
• zh—Classical Chinese
• fi—Finnish

If Yurik's list is more up to date, I can just update this page. Can anyone confirm?

If they aren’t on Yurik’s list, it just means that in these language editions mapframe is already available, e. g. Russian Wikipedia:
https://ru.wikipedia.org/w/index.php?title=Служебная:Поиск&search=insource%3Amapframe

Thanks @stjn. I think I understand now. What I'm interested is...

Wikis with Flagged Revs that don't already have Mapframe

1. Albanian 
2. Arabic
3. Alemannic
4. Bosnian
5. Belarusian
6. Bengali*
7. Chechen
8. English*
9. Esperanto
10. Persian*
11. Georgian
12. German
13. Hindi*
14. Hungarian
15. Indonesian
16. Interlingua
17. Central Kurdish
18. Polish
19. Portuguese*
20. Turkish
21. Ukrainian
22. Venetian

*Russian-style Flagged Revs wikis (that don't already have mapframe)

In thinking it through, it occurs that we might well wish to offer mapframe to the four wikis (in addition to English) that have Flagged Revs but have implemented it in the style of Russian Wikipedia, which is to say that a user putting the a map on a page will actually see the map ("Flagged Rev Override - false"). The list is below. Do you guys think that makes sense?

1. Bengali
2. English
3. Persian
4. Hindi
5. Portuguese

I've started a new ticket, T191583, and am moving the discussion there as to who will and who won't get mapframe in our wide release, mostly to non-flagged Revs wikis. Thanks to @Yurik and @stjn for helping me understand which wikis are Flagged Revs and which are what type of Flagged Rev. I think I'm clear on the issue now, and you'll see the plan as I now understand it in T191583.

@jmatazzoni, I can confirm that in fiwiki which have been using the flagged revs and mapframe some time and combination seems to work fine.

About Flagged Revs configurations.

; Protection mode

• Flagged revs protection mode setups is setup where flagged revs is used to protect small number of articles so that only edits that have been reviewed will be visible to readers by default.
• https://www.mediawiki.org/wiki/Help:Pending_changes
• Configuration setting: wgFlaggedRevsProtection=true
• Examples: ptwiki, enwiki, hiwiki, fawiki

; Normal mode, latest version by default

• Flagged revs protection mode setups is setup where flagged revs is used with all articles, but the latest version is visible by default. Eg. flagged revisions extension is used for reviewing the edits. A small number of pages can be protected like in protection setup.
• Configuration settings: wgFlaggedRevsProtection=false ; $wgFlaggedRevsOverride = false;
• Examples: ruwiki, fiwiki ...

; Normal mode, stable version by default

• Flagged revs is used with all articles and edits needs to be reviewed first before they are visible to readers by default. This is traditional dewiki setup
• Configuration settings: wgFlaggedRevsProtection=false ; $wgFlaggedRevsOverride = true;
• examples: huwiki, dewiki, plwiki, arwiki, sqwiki, idwiki, trwiki

• https://noc.wikimedia.org/conf/highlight.php?file=flaggedrevs.php

Most likely problems will come with the dewiki style setups where all articles are included to flagged revs workflow and edits needs to be reviewed first before they are visible to the readers.

Thanks for the feedback @Zache. Have a look at T191583 and see if that looks right to you. The sites who will not be getting mapframe in this go are those which have the "Normal mode," as you term it, such as de.wiki.

In T151665#4113593, @jmatazzoni wrote:

Thanks for the feedback @Zache. Have a look at T191583 and see if that looks right to you.

OK, when I wrote that, I was wrong. NOW you can please look at that ticket, and we think we have it right. Your comment was most helpful. As I think you were trying to suggest, it's not whether the wiki uses Flagged revs on all pages that matters (wgFlaggedRevsProtection=false).

The wikis where Flagged Revs is a problem are those that hide the latest version by default ($wgFlaggedRevsOverride = true). Right? See if you agree with the updated breakdown now in T191583. Thanks!

In T151665#4113725, @jmatazzoni wrote:

The wikis where Flagged Revs is a problem are those that hide the latest version by default ($wgFlaggedRevsOverride = true). Right? See if you agree with the updated breakdown now in T191583. Thanks!

Yes and the updated breakdown seems to be ok to me.