Page MenuHomePhabricator
Create Task
Maniphest T346178

Remove hardcoded message paths in banana-checker configuration
Open, Needs TriagePublic
Actions

Description

Some MediaWiki extensions are configured to load the list of i18n files for banana-checker by pulling them directly from MessagesDirs in extension.json. The code in Gruntfile usually looks like this:

The first lines of Gruntfile.json:

'use strict';
module.exports = function ( grunt ) {
	const conf = grunt.file.readJSON( 'extension.json' );

Then, further down:

banana: conf.MessagesDirs

Some other extensions, however, use a manually-written list of i18n files, something like:

		banana: {
			all: 'i18n/'
		}

Most (or maybe even all) extensions should probably use the former. Developers sometimes add files and forget to add them to the banana configuration, and then these files are left unchecked. Updating to the automatic loading from extension.json will help reduce repetition and avoid bugs of this kind.

Here's a simplistic list of extensions with a hardcoded list. I've generated it by searching for banana: \{ in CodeSearch. Because it's simplistic, it may be not entirely correct, so corrections are welcome.

  • Extension:3D
  • Extension:AJAXPoll
  • Extension:AdminLinks
  • Extension:AdvancedMeta
  • Extension:AdvancedSearch
  • Extension:AkismetKlik
  • Extension:AllTimeZones
  • Extension:AnchorHandler
  • Extension:Arrays
  • Extension:ArticleFeedbackv5
  • Extension:ArticlePlaceholder
  • Extension:Athena
  • Extension:Auth_remoteuser
  • Extension:AuthorProtect
  • Extension:AutoCategoriseUploads
  • Extension:AutoCreateCategoryPages
  • Extension:Awesomeness
  • Extension:BatchUserRights
  • Extension:BlueSpiceCategoryManager
  • Extension:BlueSpiceExtendedSearch
  • Extension:BlueSpicePageVersion
  • Extension:BlueSpiceSMWConnector
  • Extension:BreadCrumbs2
  • Extension:Buggy
  • Extension:CSS
  • Extension:CategorySortHeaders
  • Extension:CategoryTagSorter
  • Extension:CategoryTests
  • Extension:CentralLogging
  • Extension:Checkpoint
  • Extension:CirrusSearch
  • Extension:CloseWikis
  • Extension:CodeEditor
  • Extension:CodeMirror
  • Extension:CollaborationKit
  • Extension:Commentbox
  • Extension:CommonMessages
  • Extension:ConfirmAccount
  • Extension:ConfirmEdit
  • Extension:ContributionScores
  • Extension:Contributors
  • Extension:CreateAPage
  • Extension:CreateUserPage
  • Extension:DPLforum
  • Extension:DateDiff
  • Extension:DebugMode
  • Extension:DebugTemplates
  • Extension:DeleteBatch
  • Extension:DeleteOwn
  • Extension:DeleteUserPages
  • Extension:Description2
  • Extension:Diagnosis
  • Extension:DiscussionThreading
  • Extension:DisplayTitle
  • Extension:DonationInterface
  • Extension:Duplicator
  • Extension:DynamicPageList
  • Extension:EditNotify
  • Extension:EditSubpages
  • Extension:EditUser
  • Extension:Editcount
  • Extension:EmailAuth
  • Extension:EmailDiff
  • Extension:EnhanceContactForm
  • Extension:ErrorHandler
  • Extension:EtherpadLite
  • Extension:EventLogging
  • Extension:ExternalArticles
  • Extension:ExternalData
  • Extension:ExtraLanguageLink
  • Extension:FileAnnotations
  • Extension:FilterListUsers
  • Extension:FixedHeaderTable
  • Extension:FlaggedRevs
  • Extension:ForcePreview
  • Extension:FormWizard
  • Extension:FormatDates
  • Extension:GPGMail
  • Extension:Genealogy
  • Extension:GeoGebra
  • Extension:GlobalContribs
  • Extension:GlobalContributions
  • Extension:GlobalUserGroups
  • Extension:GlobalUserrights
  • Extension:GoogleAnalyticsTopPages
  • Extension:GoogleDocTag
  • Extension:GoogleDocs4MW
  • Extension:GoogleGeocode
  • Extension:GoogleLogin
  • Extension:GooglePlaces
  • Extension:GoogleSiteSearch
  • Extension:GraphViz
  • Extension:GroupsSidebar
  • Extension:GrowthExperiments
  • Extension:GuidedTour
  • Extension:HSTS
  • Extension:HTMLPurifier
  • Extension:HTMLets
  • Extension:HashTables
  • Extension:HelpCommons
  • Extension:HelpPages
  • Extension:HidePrefix
  • Extension:HierarchyBuilder
  • Extension:HostStats
  • Extension:Html2Wiki
  • Extension:I18nTags
  • Extension:IframePage
  • Extension:ImageTweaks
  • Extension:ImportArticles
  • Extension:ImportFreeImages
  • Extension:ImportUsers
  • Extension:InlineCategorizer
  • Extension:InputBox
  • Extension:Interlanguage
  • Extension:InviteSignup
  • Extension:JsonData
  • Extension:LanguageSelector
  • Extension:LanguageTag
  • Extension:LastModified
  • Extension:LastUserLogin
  • Extension:LinkFilter
  • Extension:LinkSuggest
  • Extension:LinkSuggest2
  • Extension:Link_Attributes
  • Extension:LinkedWiki
  • Extension:LockDownEnglishPages
  • Extension:Lockdown
  • Extension:LogEntry
  • Extension:LogoFunctions
  • Extension:LookupUser
  • Extension:LoopFunctions
  • Extension:Loops
  • Extension:MP3MediaHandler
  • Extension:MadLib
  • Extension:MagicNoCache
  • Extension:Mailgun
  • Extension:MaintenanceShell
  • Extension:MassEditRegex
  • Extension:MediaFunctions
  • Extension:MediaWikiChat
  • Extension:MediaWikiFarm
  • Extension:Memento
  • Extension:MetaMaster
  • Extension:MinimumNameLength
  • Extension:MixedNamespaceSearchSuggestions
  • Extension:MobileApp
  • Extension:Model
  • Extension:Mpdf
  • Extension:MsInsert
  • Extension:MsLinks
  • Extension:MsUpload
  • Extension:MsWikiEditor
  • Extension:MultiMaps
  • Extension:MultimediaPlayer
  • Extension:MyVariables
  • Extension:NamespacePaths
  • Extension:NamespacePopups
  • Extension:NamespaceSidebar
  • Extension:NaturalLanguageList
  • Extension:Negref
  • Extension:NetworkAuth
  • Extension:NewSignupPage
  • Extension:NewUserActions
  • Extension:NewUserMessage
  • Extension:NewestPages
  • Extension:NewsBox
  • Extension:NoBogusUserpages
  • Extension:NoTitle
  • Extension:NotebookViewer
  • Extension:NotesLink
  • Extension:NukeDPL
  • Extension:NumberFormat
  • Extension:Numbertext
  • Extension:NumerAlpha
  • Extension:OAuth
  • Extension:OAuthAuthentication
  • Extension:OOJSPlus
  • Extension:OOJsUIAjaxLogin
  • Extension:OdbcDatabase
  • Extension:OnlineStatus
  • Extension:OnlyRecentRecentChanges
  • Extension:OpenBadges
  • Extension:OpenGraphMeta
  • Extension:OpenID
  • Extension:OpenIDConnect
  • Extension:PCRGUIInserts
  • Extension:PGFTikZ
  • Extension:PageDisqus
  • Extension:PageForms
  • Extension:PageInCat
  • Extension:PagePopups
  • Extension:PageTools
  • Extension:PanScroll
  • Extension:ParentPage
  • Extension:Patroller
  • Extension:PerPageLicense
  • Extension:PhabTaskGraph
  • Extension:PhpTagsFunctions
  • Extension:PhpTagsMaps
  • Extension:PhpTagsSMW
  • Extension:PhpTagsSPARQL
  • Extension:PhpTagsStorage
  • Extension:PhpTagsWidgets
  • Extension:PhpTagsWiki
  • Extension:PipeEscape
  • Extension:PluggableAuth
  • Extension:Polyglot
  • Extension:PopcornEditor
  • Extension:Prezi
  • Extension:PronunciationRecording
  • Extension:ProtectSite
  • Extension:PurgeClickThrough
  • Extension:Push
  • Extension:PushAll
  • Extension:QuickResponse
  • Extension:Quiz
  • Extension:QuizGame
  • Extension:ReassignEdits
  • Extension:RefreshSpecial
  • Extension:RegexFun
  • Extension:RegexFunctions
  • Extension:RelatedSites
  • Extension:ReplaceSet
  • Extension:ReportIncident
  • Extension:RightFunctions
  • Extension:SSLClientAuthentication
  • Extension:SacredText
  • Extension:SafeDelete
  • Extension:Sarcasm
  • Extension:Screenplay
  • Extension:SearchStats
  • Extension:SectionDisqus
  • Extension:SecureHTML
  • Extension:SelectCategory
  • Extension:SelectTag
  • Extension:SemanticACL
  • Extension:SemanticBundle
  • Extension:SemanticDependency
  • Extension:SemanticGenealogy
  • Extension:SemanticImageAnnotator
  • Extension:SemanticImageInput
  • Extension:SemanticInternalObjects
  • Extension:SemanticPageSeries
  • Extension:SemanticRating
  • Extension:SemanticSifter
  • Extension:SemanticWebBrowser
  • Extension:SendGrid
  • Extension:ShoutWikiAPI
  • Extension:ShoutWikiAds
  • Extension:SideBarMenu
  • Extension:SidebarDonateBox
  • Extension:SignWritingMediaWikiPlugin
  • Extension:SimpleChanges
  • Extension:SimpleSAMLphp
  • Extension:SimpleSort
  • Extension:SiteSettings
  • Extension:SkinPerNamespace
  • Extension:SlimboxThumbs
  • Extension:SmiteSpam
  • Extension:SocialLogin
  • Extension:SocialProfile
  • Extension:SoundManager2Button
  • Extension:SpamDiffTool
  • Extension:Spark
  • Extension:SparkPost
  • Extension:Special404
  • Extension:SpecialNamespaces
  • Extension:SpellingApi
  • Extension:SpellingDictionary
  • Extension:SplitPrivateWiki
  • Extension:StaffEdits
  • Extension:StaffPowers
  • Extension:StalkerLog
  • Extension:StandardDialogs
  • Extension:StarterWiki
  • Extension:StatCounter
  • Extension:StickToThatLanguage
  • Extension:StopForumSpam
  • Extension:SubpageFun
  • Extension:Sudo
  • Extension:Survey
  • Extension:SwiftMailer
  • Extension:Tabs
  • Extension:TemplateData
  • Extension:TemplateStyles
  • Extension:TestLanguageNameGrammar
  • Extension:Theme
  • Extension:TimedMediaHandler
  • Extension:Tooltip
  • Extension:TopTenPages
  • Extension:TranslateSvg
  • Extension:TranslationNotifications
  • Extension:TrustedXFF
  • Extension:TweetANew
  • Extension:TwitterCards
  • Extension:TwitterLogin
  • Extension:TwnMainPage
  • Extension:UIFeedback
  • Extension:UnusedRedirects
  • Extension:UploadBlacklist
  • Extension:UrlGetParameters
  • Extension:UserExport
  • Extension:UserFunctions
  • Extension:UserOptionStats
  • Extension:UserPageEditProtection
  • Extension:UserPageViewTracker
  • Extension:UserThrottle
  • Extension:VEForAll
  • Extension:Variables
  • Extension:VersionCompare
  • Extension:VipsScaler
  • Extension:VisualEditor (patch)
  • Extension:WantedPagesFromNS
  • Extension:WatchAnalytics
  • Extension:WatchSubpages
  • Extension:WhiteSpace
  • Extension:WhitelistPages
  • Extension:WhoIsWatching
  • Extension:WhosOnline
  • Extension:WikiArticleFeeds
  • Extension:WikiForum
  • Extension:WikiLexicalData
  • Extension:WikiLove
  • Extension:WikiLovesMonuments
  • Extension:Wikibase
  • Extension:WikidataPageBanner
  • Extension:WikimediaBadges
  • Extension:WikimediaEvents
  • Extension:WindowsAzureStorage
  • Extension:YetAnotherKeywords
  • Extension:googleAnalytics
  • Skin:Anisa
  • Skin:BlueSky
  • Skin:Cavendish
  • Skin:GreyStuff
  • Skin:HasSomeColours
  • Skin:Mask
  • Skin:Refreshed
  • Skin:Splash
  • Skin:Tempo
  • Skin:Timeless
  • Skin:WMAU
  • Skin:WoOgLeShades
  • Skin:erudite
  • Skin:mediawiki-strapping
  • Skin:p2wiki

Most of these can probably be safely updated. In some cases, manual lists maybe justified, although I cannot think of any right now; extension maintainers should be able to identify those when reviewing the fixes.

Marking it as a good first task , because the changes are mostly trivial and identical:

  1. Add const conf = grunt.file.readJSON( 'extension.json' ); in the beginning.
  2. Change banana: { all: 'i18n/' } to banana: conf.MessagesDirs.

Perhaps it can even be done by a script bot that runs over all the extension, but that would have to be very careful.

Event Timeline

Amire80 created this task.Sep 12 2023, 6:17 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 12 2023, 6:17 PM
Frostly subscribed.Sep 16 2023, 9:07 PM
Frostly added a comment.Sep 16 2023, 9:48 PM

Some extensions (rough list: https://codesearch.wmcloud.org/search/?q=+var+conf+%5C%3D+grunt%5C.file%5C.readJSON%5C%28+%27extension%5C.json%27+%5C%29%3B+&files=&excludeFiles=&repos=) use var instead of const for declaring the conf variable; this should probably be changed in those as well.

Nox1134 claimed this task.Sep 20 2023, 7:22 PM
Krinkle subscribed.Sep 21 2023, 1:01 AM

Are there more than 10 repos in all of Wikimedia Gerrit that don't simply use i18n/ as the directory for all localisation files? I don't have the data, but it seems likely to me that it would be true. Adding complexity to all 1000+ extensions for that edge case is worth weighing the pros and cons over.

Alternatively, perhaps an even cheaper outcome, if we're going to change all repos, would be to add support for extension.json to banana-checker. For example, rather than setting the i18n/ directory we could set it to '.' like we do with ESLint already.

Banana checker can then be enhanced to perform two heuristics instead of one. Currently, its heuristic is to look for a file called en.json and then assume this is an i18n directory and go from there. It could additionally look for extension.json and consider its`MessageDirs` as providing additional directories to scan.

Something like the following:

banana-checker.js
  const path = require( 'path' );
  const fs = require( 'fs' );

- module.exports = function bananaChecker( dir, options, logErr ) {
+ function checkI18nDir( dir, options, logErr ) {
    …
  }

+ module.exports = function bananaChecker( dir, options, logErr ) {
    options = Object.assign( {
      …
+     extensionMessageDirs: true
    }, options );

+   if ( options.extensionMessageDirs && fs.existsSync( path.resolve( dir, 'extension.json' ) ) ) {
+     return require( path.resolve( dir, 'extension.json' ) ).MessageDirs
+       .every( msgDir => checkI18nDir( msgDir, options, logErr ) );
+  } else {
+    return checkI18nDir( dir, options, logErr );
+  }
+ }

Then in repositories, Gruntfile.js could be simplified as follows:

Gruntfile.js
-   banana: 'i18n/'
+   banana: '.'

Or

Gruntfile.js
- const conf = grunt.file.readJSON( 'extension.json' );
-

-   banana: conf.MessagesDirs
+   banana: '.'

Or, for repos that don't use Grunt, we can invoke it standalone the same way we do with ESLint in several repos already, directly from package.json

https://codesearch.wmcloud.org/search/?q=%22banana-checker&files=package.json&excludeFiles=&repos=

{
  "scripts": {
    …
    "lint:i18n": "banana-checker i18n/"
  }
}
{
  "scripts": {
    …
    "test": "eslint . && banana-checker ."
  }
}
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL