Page MenuHomePhabricator

Add OpenHistoricalMap to translatewiki.net
Closed, ResolvedPublic

Description

Project information

Name: OpenHistoricalMap
Homepage: https://openhistoricalmap.org
Project link: https://openhistoricalmap.org
Code repository: https://github.com/OpenHistoricalMap/ohm-website

OS License: https://github.com/OpenHistoricalMap/ohm-website/blob/staging/LICENSE
Issue Tracker: https://github.com/OpenHistoricalMap/issues/issues/
Project contact: https://translatewiki.net/wiki/User:Danrademacher and dan@greeninfo.org

Logo:

Project description:
OpenHistoricalMap is about historical data. Things that used to be there. From a variety of sources, contributed by mappers and historians across the world. OpenHistoricalMap's community is diverse, passionate, and growing every day. Our contributors include enthusiast mappers, academics, digital historians, historical societies, and many more. To learn more about the community, see the user diaries etc. OpenHistoricalMap is open data: you are free to use it for any purpose under the terms of the data you are using. OpenHistoricalMap identifies license and citation requirements at the object level (if any). Credit or citation to OpenHistoricalMap for consolidating this data is appreciated, but not required. If you alter or build upon the data in certain ways, you may distribute the result only under the same license.

Note that we are operating a fork of the OpenStreetMap website, so the locale files with keys and values should be well formed, especially for English and EN-GB. We also inherited all the existing OSM locale files from our upstream code. These are maybe 90% unchanged, but with important changes to the name, obviously, but also to license and some other details specific to our modifications of OSM's web code. We're happy to modify the non-English locale files in our repo however is best to set us up as a separate project here so folks can translate. We do have active interest from data contributors to also help with translation.

NOTE: Section below will be filled by twn staff

Project setup checklist

Project configuration (for translation admins)

Namespace: NS_PROJECTS (New namespace)
Prefix: ohm-
Validators:

  • InsertableRubyVariable
  • MatchSet (For html.dir)
  • UnicodePlural

Insertables:

  • HtmlTagInsertablesSuggester
  • RegexInsertablesSuggester (For HTML entities)

Support: https://github.com/OpenHistoricalMap/issues/issues/new

Concerns

Event Timeline

abi_ triaged this task as Medium priority.

Change 825326 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[translatewiki@master] Add OpenHistoricalMap

https://gerrit.wikimedia.org/r/825326

Change 825326 merged by jenkins-bot:

[translatewiki@master] Add OpenHistoricalMap

https://gerrit.wikimedia.org/r/825326

Exports will happen on Thursday, 02-Aug-2022.

There are a few messages with trailing whitespace, which is not supported in translatewiki.net:

Pattern matched 1 file based message group(s).
Left side is the expected value. Right side is the actual value in wiki.
Projects:Ohm-site.about.open data html/en
'OpenHistoricalMap is <i>open          | 'OpenHistoricalMap is <i>open         
data</i>: you are free to use it for   | data</i>: you are free to use it for  
any purpose                            | any purpose                           
under the terms of the data you are    | under the terms of the data you are   
using. OpenHistoricalMap identifies    | using. OpenHistoricalMap identifies   
license                                | license                               
and citation requirements at the       | and citation requirements at the      
object level (if any). Credit or       | object level (if any). Credit or      
citation to                            | citation to                           
OpenHistoricalMap for consolidating    | OpenHistoricalMap for consolidating   
this data is appreciated, but not      | this data is appreciated, but not     
required.                              | required.                             
If you alter or build upon the data in | If you alter or build upon the data in
certain ways, you may distribute the   | certain ways, you may distribute the  
result only                            | result only                           
under the same licence. See the <a     | under the same licence. See the <a    
href='%{copyright_path}'>Copyright and | href='%{copyright_path}'>Copyright and
                                       |                                       
License page</a> for details. '        | License page</a> for details.'        

Projects:Ohm-site.about.legal 1 html/en
'This site and many other related      | 'This site and many other related     
services are formally operated by the  | services are formally operated by the 
very informal OpenHistoricalMap        | very informal OpenHistoricalMap       
collective                             | collective                            
on behalf of the community. '          | on behalf of the community.'          

Projects:Ohm-site.copyright.legal babble.intro 2 html/en
'You are free to copy, distribute,     | 'You are free to copy, distribute,    
transmit, and adapt our data, as long  | transmit, and adapt our data, as long 
as                                     | as                                    
you abide by the terms of the data you | you abide by the terms of the data you
are copying, distributing, or          | are copying, distributing, or         
adapting.                              | adapting.                             
If you alter or build upon the data on | If you alter or build upon the data on
this site, you may distribute the      | this site, you may distribute the     
result                                 | result                                
only under the terms of the underlying | only under the terms of the underlying
data objects’ licenses. '            | data objects’ licenses.'            

Projects:Ohm-site.copyright.legal babble.contributors intro html/en
'Our contributors are many, many       | 'Our contributors are many, many      
individuals and organizations. We      | individuals and organizations. We     
would                                  | would                                 
not exist without their efforts. '     | not exist without their efforts.'     

Projects:Ohm-site.welcome.whats on the map.off html/en
'What it <em>doesn't</em> include is   | 'What it <em>doesn't</em> include is  
opinionated data like ratings, and     | opinionated data like ratings, and    
data                                   | data                                  
from copyrighted sources. '            | from copyrighted sources.'

These should be fixed to avoid showing up as unsynchronized messages.

I noticed some messages like the copyright has been made to be just .. in English but not in Translations. I would recommend to remove those messages completely from translation until they are sorted out. You are getting bad new translations in, that may not be updated immediately when the content of these messages is updated.

The problem goes further than that. There are numerous references to OSM that have been replaced or updated in English, but not in translations. This means there are numerous imported translations that need updating, but the translators don't know which ones those are.

I think the issue is one I noted in my original summary of the project:

“ We also inherited all the existing OSM locale files from our upstream code. These are maybe 90% unchanged, but with important changes to the name, obviously, but also to license and some other details specific to our modifications of OSM's web code.”

I originally thought that our only two choices might be:

  1. Current situation where it’s hard to tell what translations need revisions
  2. Delete *all* translations and start over, but then there’s a lot of extra work that didn’t need to happen

Maybe what we need to do is

  1. Delete translations only for those items that changed in English, retaining anything that did not change (plus remove the keys you noted as “..”

That third option will still be some extra work translators, for example where the only change was the project name, but better than having to retranslate everything and it will be clear what needs translation

A list of keys whose content was changed before the initial import would be helpful. With them we can either delete or mark the translations as outdated, which means those needing an update are clearly visible to translators.

Anything that is changed now after the initial import will be automatically marked as outdated.

Delete *all* translations and start over, but then there’s a lot of extra work that didn’t need to happen

Not necessarily. Any strings unchanged from OSM would be untranslated, but Translatewiki.net’s translation memory will almost certainly pick up the translation from OSM and present it as a 100% match. If it’s a string that has been customized for OHM in English, then it would still be picked up with a score of less than 100%. So it wouldn’t be starting entirely from scratch. In complex cases, it would also be possible to link to the related OSM message in the /qqq documentation. This can happen at any time following the initial import.

I was comparing the staging (default) branch with the translatewiki branch (see diff) and I see a lot of strings removed. Will investigate what's happening.

I've created a PR to the translatewiki branch. It restores some of the files that were deleted due to incomplete import.

@Danrademacher - How often are we planning to merge the translatewiki branch to the default staging branch and vice versa so that translators get the latest strings?

I merged that PR in today, and I also finally created the diff of our current version of the en.yml against the upstream OSM version to isolate which keys we edited in the en.yml file. That's here https://gist.github.com/danrademacher/e7eb528c31cecb78e2f8c9b9e33af1fd I am not quite sure what the most useful format for this is -- the idea is to use this to help narrow down what actually needs translation compared to OSM. Only the keys shown as modified in this diff should need translation.

As for the frequency of merging, the strings on our site are relatively stable overall, so I would expect we won't add new strings more than a few times a year. But I imagine that at least at first, we would want to merge back more often from translatewiki into our staging (and later production) branches to make sure the work of translators is making it into the live application as quickly as possible.

Realistically, I think we could deploy changes twice a month. Is that a reasonable cadence?

I merged that PR in today, and I also finally created the diff of our current version of the en.yml against the upstream OSM version to isolate which keys we edited in the en.yml file. That's here https://gist.github.com/danrademacher/e7eb528c31cecb78e2f8c9b9e33af1fd I am not quite sure what the most useful format for this is -- the idea is to use this to help narrow down what actually needs translation compared to OSM. Only the keys shown as modified in this diff should need translation.

Would it be possible to do a careful find and replace, to replace occurrences of OpenStreetMap with OpenHistoricalMap in all translations after we've copied them over from OpenStreetMap?

For source strings in English that have changed drastically, the best course of action is to delete them from all the translation files.

Realistically, I think we could deploy changes twice a month. Is that a reasonable cadence?

Yes, that's fine.

I think I figured out the right approach to find/replace here -- I replaced all OpenStreetMap with OpenHistoricalMap in all non-English YML files, and then went back and restored those instances of OpenStreetMap that we have retained in English. This is present at https://github.com/OpenHistoricalMap/ohm-website, live on staging.openhistoricalmap.org where it can be tested, and I hope to deploy to the live site within a day or two

I think I figured out the right approach to find/replace here -- I replaced all OpenStreetMap with OpenHistoricalMap in all non-English YML files, and then went back and restored those instances of OpenStreetMap that we have retained in English. This is present at https://github.com/OpenHistoricalMap/ohm-website, live on staging.openhistoricalmap.org where it can be tested, and I hope to deploy to the live site within a day or two

This commit looks good: https://github.com/OpenHistoricalMap/ohm-website/commit/41e2d40a052a17d79496cae2b793543ca4a7bb3d

Should we be pulling latest strings from the staging branch? The source branch in the description is mentioned as translatewiki.

If we intend to pull to translatewiki.net from staging branch and submit translations to translatewiki branch, then we can create a PR (if an open doesn't already exists) from translatewiki to staging.

Should we be pulling latest strings from the staging branch? The source branch in the description is mentioned as translatewiki.

staging is always the most recent code, so that is where to look for latest strings. If I mentioned a different branch in setup, it would have been just a misunderstanding of how the process works.

FYI, we're also working on this ticket, https://github.com/OpenHistoricalMap/issues/issues/443#issuecomment-1255652919, to make it a lot easier to see that only a relatively small number of keys have had meaningful changes to them and encourage translation of those keys. I hope that really is helpful -- I'll make sure of that before we merge any changes base don that ticket.

staging is always the most recent code, so that is where to look for latest strings. If I mentioned a different branch in setup, it would have been just a misunderstanding of how the process works.

I've updated to pull latest changes from staging and push changes to translatewiki and create a PR to staging.

FYI, we're also working on this ticket, https://github.com/OpenHistoricalMap/issues/issues/443#issuecomment-1255652919, to make it a lot easier to see that only a relatively small number of keys have had meaningful changes to them and encourage translation of those keys. I hope that really is helpful -- I'll make sure of that before we merge any changes base don that ticket.

This will indeed be useful.

Change 834735 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[translatewiki@master] OpenHistoricalMap: Fix source branch and enable PR creation

https://gerrit.wikimedia.org/r/834735

Change 834735 merged by jenkins-bot:

[translatewiki@master] OpenHistoricalMap: Fix source branch and enable PR creation

https://gerrit.wikimedia.org/r/834735