Page MenuHomePhabricator

Collect feedback from module and gadget authors for Developer Productivity & onwiki tooling techconf session
Closed, ResolvedPublic

Description

We’re running a session (T234661: Wikimedia Technical Conference 2019 Session: Developer Productivity & onwiki tooling) at next week’s Wikimedia Technical Conference around the topic: Developer Productivity and onwiki tooling - userscripts, gadgets, templates, modules.

The goal of the conference is to identify changes to tooling and processes to support Wikimedia developers in working more efficiently. One aspect of that is to explore what makes it currently difficult for technical contributors working with templates, modules, userscripts and gadgets, and to discuss what could be improved or done differently.

For this we’re looking for more input from folks who won’t be at the conference. It would be wonderful if you could share your experience and answer the following two questions:

  1. Your background: describe what you do
  2. Describe in what way your productivity as a technical contributor is affected in the context of on-wiki tooling (what slows you down, what makes your life complicated, what helps you, what hinders you …)

To give you two examples (made-up):

  1. I am a volunteer developer and have developed several user scripts for frwiki.
  2. When I’ve developed a userscript, I don’t know how many people are copying the code to use my script. When I make changes to the script, others often still have older versions. When people report bugs, I need to first find out which version they are using which is very time-consuming.
  1. I am a developer of Wikibase extension, WMDE staff member
  2. When I develop a new feature in Wikibase, I am often informed AFTER the feature has been released that Wikidata gadgets had been broken. Then I need to stop my current tasks, go back to my previous work and change the feature. This makes the process of my work on new features longer.

Event Timeline

Tgr created this task.Nov 6 2019, 2:31 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 6 2019, 2:31 AM

@Ocaasi @Edokter @TheDJ @Amalthea @Pharos @Redrose64 @kaldari @MusikAnimal @Ragesoss @Ruud_Koot @AzaToth @Legoktm @Magnus @Krinkle @MSGJ @Happy-melon @Entlinkt @Umherirrender @Raymond @DerHexer @Dr_Brains @Od1n @Gratus @0x010C @Arkanosis @Magister_Mathematicae @Angus @Platonides @Metronomo @Rotpunkt @Pietrodn @Melos @Vituzzu @Bene @Jitrixis @Ebraminio @Fomafix @Ricordisamoa @MrStradivarius @Trappist_the_monk @DePiep @PerfektesChaos @Jobu0101 @Mps @XanonymusX @Zolo @Zebulon84 @ThierryCaro @Hexasoft @Amitie_10g @Agabi10 @Jarould @Moroboshi @Sakretsu @ValterVB @Bultro @Zolo @TomT0m @matej_suchanek @JeanFred @RheingoldRiver @Tacsipacsi @Jackmcbarn @Rschen7754 @Stryn @Ymblanter @Yurik @Tobias1984

Apologies for mass-subscribing you; the name list is a combination of database queries for the most prolific gadget and module editors, and personal recollections about who had good insights about the topic. We would really value your feedback for TechConf 2019 in how the people working on developer productivity could help you with making your gadget or Lua module development work more productive. If you are not interested, I apologize for the spam, feel free to unsubscribe. (In case you are less familiar with Phabricator: the unsubscribe link is in the right-hand sidebar column.) If you are interested, please see the description of this task for the feedback format. (We are very interested in any other feedback too, but would like to have a uniform set of responses that the conference participants can work with.)

Thadguidry added a subscriber: Thadguidry.EditedNov 6 2019, 4:16 AM

No problem @Tgr

What slows me down is not having an efficient quick way to filter properties that don't include " ID" (non-authority properties), in various Property explorer tools such as the excellent https://tools.wmflabs.org/prop-explorer/
Perhaps other Property explorer tools have a quick filter mechanism for this? I tried a simple regex in the Label column filter, but it doesn't work. https://github.com/stevenliuyi/wikidata-prop-explorer/issues/1

Perhaps there could be a universal way for Property explorer tools or any tool using the Wikidata API's to more easily provide this filter mechanism such as enhancing some Wikidata API's with a field of "filterNonAuthorityProps" or something? Basically, in looking deeper into the problem, we might just need to provide an easy way in the API's to filter Wikidata Properties to those of in the set of Wikidata property for an identifier or not.

Jarould removed a subscriber: Jarould.Nov 6 2019, 4:18 AM

Hi,

  1. I'm the lead admin of https://lol.gamepedia.com/League_of_Legends_Esports_Wiki. I do a lot of work with Lua and Cargo.
  1. The #1 thing I'd like to see in modules is a standard class system as part of Scribunto, mw.class alongside mw.html etc. I've looked into using https://github.com/Yonaba/Lua-Class-System but it runs into issues with MediaWiki's garbage collection. Someone helped me make a small change to it that may have fixed it, but also might have broken other things (I can cc him into this thread if you want more specifics, he understands the issues better than I do). Right now I'm using a very simple implementation (https://lol.gamepedia.com/Module:Class) but this has very limited functionality. But even if one of these solutions did work perfectly, it would still be very nice for cross wiki syncing of code if every wiki used the same implementation of classes, and also greatly reduce the barrier to entry of having OOP available.
  1. I am a volunteer developer on dewiki and I created a pile of JavaScript tools and Lua modules applicable to any wiki in any language.
  2. Basically I am happy.
    1. I am about ten years on this drug.
    2. I created all tools I need, and some are running private, some are made available to everything under MediaWiki.
    3. What slows me down are some users in my own community, but there is no remedy on any conference.
  3. What we could need in the long run would be another edition of TechNews; in developer mode, or perhaps focussing on JavaScript, Lua, API, parsing & maintenance categories & LINT, toolforge, VE etc.
    1. The broader and translated TechNews on a weekly base is too much information for me; most things I do know already or I do not need at all. It is too long for me and explaining too much; it is heading towards another auditory and I did not subscribe. Information overflow.
    2. An English only newsletter for developers in a particular field on important occasions like breaking change, deprecating public functionality etc. might improve communication, with short notices for experts (legacy function mw.util.grandfather() deprecating from now, see phab:T1999). Since narrowed to JavaScript, Lua, API those subscribers only will be fed and not bothered by other fields. And if there is nothing worth to broadcast over months it is fine and does not cause work.
    3. The Project News on dewiki are maintained now for a dozen of years, and I can watch this and I am precisely informed, if the editor does not fail to include something which is important for me.
Stryn removed a subscriber: Stryn.Nov 6 2019, 11:07 AM
  1. I’m a volunteer developer working with templates/modules on Hungarian and multilingual projects, as well as developing gadgets and sitewide scripts/styles on Hungarian projects only.
  2. What comes to my mind is two things (more may follow):
    • The inability to deploy gadgets and other JavaScript/CSS files stored in MediaWiki namespace to certain pages. TemplateStyles is a big step forward, but it has some restrictions which could be bypassed by allowing to include MediaWiki-namespace things (which don’t need such restrictions as they can be edited by only interface admins):
      • Some things require JavaScript to work. This can be achieved only using globally enabled gadgets, which may probably function as loaders if a higher amount of JS is needed. Even loaders slow down all pages even they do something useful on only a handful of pages.
      • Some pages need CSS that applies outside of content area (most notably main pages to hide the page title). This is now only possible by hard-coding the page title in MediaWiki:Common.css/Vector.css. This prevents portability and is a huge problem for multilingual wikis, where potentially hundreds of page titles have to be hard-coded. (It also raises encoding issues: .page-Kezdőlap CSS rule breaks the CodeEditor syntax highlighter, and I can only hope it doesn’t do so with some older browsers like IE6.)
    • Insufficient tools to find out where globally loaded things actually do something. When I want to remove for example a CSS rule, I want to know whether it’s in use anywhere, and if it is on a large number of pages, I want to know what template/module/whatever adds it; if it isn’t, I want to make sure it’s not just because no page uses a template with a sufficient parameter set. Currently I can only use insource: search, but that searches in the source text (i.e. before template expansion, so classes added by templates don’t show up), and has no easy-to-use function to limit the results to CSS classes, which is particularly a problem when I want to remove (convert to TemplateStyles) selectors like .navbox and .infobox, which appear as template names much more often than as direct CSS class input.

Hello,

  1. I am a volunteer developer, with home on cswiki. Userscripts, gadgets, templates, and modules are all my fields.
  2. Some ideas:
    • explore T31272: Implement Gadgets 2.0 and T31398: Implement Gadget Manager - many possibly interesting new features, most notably...
      • Since Wikidata is a multilingual project, the gadgets there need localization. This is usually done in the code, so interface admins are required. Also every gadget does localization in different way (eg. using mw.messages.set, which is shared for all gadgets and therefore dangerous).
    • T121470: Central Global Repository for Templates, Lua modules, and Gadgets
    • T120767: A powerful, handy TemplateTiger (I'd like to see what values a template parameter is used with)
    • PerformanceInspector (status?)
    • unix-like diffs for Lua/JS (using wikitext diffs is somewhat strange)
    • {{#if: [[Category:Foo]] | yes | no }} yields yes - this makes developing eg. infoboxes harder because you have to split and duplicate some code and move categorization outside the infobox, otherwise some elements may appear blank
    • the ability to look up all usage of a userscript (locally and globally) is just a hack: T35355#1862685 (once it may be removed)
    • (perhaps already fixed?) when you update TemplateData in the documentation subpage, you need to do a null-edit in the template itself (not everybody knows this, it's confusing)
    • (perhaps off-topic?) Citoid is kind of a black-box, out of our reach - sometimes the results are not ideal and there isn't much we could do on-wiki
    • ...

@Dvorapa @Urbanecm

Od1n added a comment.Nov 6 2019, 4:52 PM
  1. Volunteer developer at frwiki.
  2. For me the main issue is that some scripts are just maintenance nightmares.
    • Examples:
    • The issues are, more precisely:
      • Very few people are able to work on these codes (or are willing to spend dozens of hours to understand the whole script and to investigate the interactions with the other scripts).
      • We are close to a bus factor 1. When the creator of a script leaves the project, there's no one else crazy enough to carry on such script.
      • It's nearly impossible to make any change without breaking something (in the script, or in interacting scripts), also without making the code even more complicated. Making a "clean" change requires a lot of skills and time.
    • Thoughts about improving this:
      • Anything that would improve modularity and decoupling would be good to take. Slicing these monolithic monstruosities into smaller codes that could be worked on separately, sanely.
      • Same with things like unit testing, automated reports, and such. Anything that would help spotting any regressions immediately, or at least before they become difficult to fix.
MSGJ removed a subscriber: MSGJ.Wed, Nov 6, 8:26 PM
  • {{#if: [[Category:Foo]] | yes | no }} yields yes - this makes developing eg. infoboxes harder because you have to split and duplicate some code and move categorization outside the infobox, otherwise some elements may appear blank

I’d prefer T137584: Allow Scribunto code to add a category without changing output (maybe along with a wikitext equivalent) instead—it provides solution for more situations, more elegant, and is less hackish, much easier to implement, while causes fewer things to break (e.g. with your solution, {{#if:{{{1|}}} | <span>{{{1}}}</span>}} with |1=[[Category:Foo]] would not display the category at all).

Arkanosis added a comment.EditedMon, Nov 11, 5:15 PM

Hi!

  1. I'm a volunteer scripts, gadgets, bots and tools developer at frwiki and wikidata, and I do some occasional training for Scribunto beginners, even though I'm not very active in modules development.
  2. I have to say that, having done this stuff since 2009, I'm really happy with how things are improving from a third-party developer's perspective. That being said, there are significant remaining pain points, including:
    • a lack of relevant metrics about who uses what and what depends on what;
    • a lack of attention from upstream to the tools people actually use — which itself is the result of the previous pain point — resulting in these tools being regularly broken with nobody to fix them (this is getting better, but there's room for improvement);
    • the inability to efficiently share and localize code across wikis;
    • the frequent low bus factor or absence of proper maintainership for tools people actually use, which sometimes could be solved by resolving the previous pain point;
    • the frequent diverging forks or competing implementations of highly valuable tools, which could be avoided by solving the same sharing problem;
    • the completely inappropriate tooling for developing tools (scripts, gadgets, templates, modules…) on MediaWiki: it's widely recognized that software development highly benefits from using version control systems with branchs (eg. git), code review (eg. gerrit, pull requests), test suites, proper tagging and release systems, etc., yet most of the development done outside of the core of MediaWiki has none of these. I'm not calling for more custom-tailored tools within MediaWiki, but for more integration with existing tools, so that developers can use their tools of choice (which they are productive with and which improve without any cost for Wikimedia).

Thanks!

Tpt added a subscriber: Tpt.Wed, Nov 13, 3:13 PM
Tgr closed this task as Resolved.Thu, Nov 28, 9:34 PM
Tgr claimed this task.

Thanks to everyone who provided feedback! You can read in T234661: Wikimedia Technical Conference 2019 Session: Developer Productivity & onwiki tooling about how it was used and how the session went.