Page MenuHomePhabricator

Wikimedia Technical Conference 2019 Session: Developer Productivity & onwiki tooling
Open, Needs TriagePublic

Subscribers
Tokens
"Love" token, awarded by Tgr."Love" token, awarded by Theklan."Love" token, awarded by TheDJ."Meh!" token, awarded by Bmueller."Love" token, awarded by Tpt."Love" token, awarded by MusikAnimal."Love" token, awarded by hashar.
Assigned To
Authored By
debt, Fri, Oct 4

Description

Session

  • Track: People and Processes
  • Topic: Developer Productivity & onwiki tooling

Description

Gadgets, modules, templates play a central role in the WM projects for both wiki workflows and for the reader UI (with infobox probably being the most visible example). The lack of a central source code store, testing infrastructure, testing and code review for those components hinders developer productivity on multiple levels and duplicates effort of wiki communities. This session intends to create an overview on current issues in the area of on-wiki tooling and to draft potential solutions.

Questions to answer and discuss

Question:
Significance:

Question:
Significance:

Related Issues

  • ...
  • ...

Pre-reading for all Participants


Notes document(s)

[link to notes document (gdoc and / or etherpad)]

Notes and Facilitation guidance

https://www.mediawiki.org/wiki/Wikimedia_Technical_Conference/2019/NotesandFacilitation


Session Leader(s)

  • [name]
  • [name]

Session Scribes

Session Facilitator

  • [name]

Session Style / Format

  • [what type of format will this session be?]

Session Leaders please:

  • Add more details to this task description.
  • Coordinate any pre-event discussions (here on Phab, IRC, email, hangout, etc).
  • Outline the plan for discussing this topic at the event.
  • Optionally, include what this session will not try to solve.
  • Update this task with summaries of any pre-event discussions.
  • Include ways for people not attending to be involved in discussions before the event and afterwards.

Post-event summary:

  • ...

Post-event action items:

  • ...

Event Timeline

debt created this task.Fri, Oct 4, 3:49 PM
hashar added a subscriber: hashar.Mon, Oct 7, 2:22 PM

Seems to relate to T121470: Central Global Repository for Templates, Lua modules, and Gadgets and a lot of associated tasks. A potential worry is we end up reinventing in MediaWiki systems to conduct code review, conduct test and manage a software life cycle. I would guess the main issue is to make a "business case" for it, have it funded and commit to implement the product.

In the early days of MediaWiki, one would code the feature inside MediaWiki itself then wait for it to be reviewed and deployed (which could take a few months). That also requires knowledge of PHP as well as acquaintance with other developers. Over the years, that has been decentralized:

  • The addition of hooks delegated that responsibility out of MediaWiki core and its maintainer to extension authors and eventually reaching a fixed schedule for deployment (weekly based). The ProofreadPage extension for wikisources wikis or AbuseFilter are probably good examples.
  • MediaWiki templates have delegated a lot of formatting to editors, needs which previously could only be addressed by coding PHP in MediaWiki. That quickly became hard to maintain, infobox are a good example. The Lua modules at least lets one use a proper programming language instead of a lot of curly braces.
  • Gadgets fill a lot of use cases, though I do not know how they are tested they might be risky?

In all cases, there are surely very helpful templates or modules which we might want to "upstream" as MediaWiki extensions or even as core functionalities of MediaWiki itself. That might help resolve the burden of synchronizing them between wikis and ensure the code is properly maintained. I would guess infobox would be a good case for upstreaming.

Anyway I am interested, if there is no conflict for me with another session.

hashar awarded a token.Mon, Oct 7, 2:22 PM
Amire80 added a subscriber: Amire80.EditedMon, Oct 7, 7:31 PM

Modules and templates must be all global, at least optionally. It's not just a necessary change for the MediaWiki platform as it is used by Wikimedia; it's the most necessary change.

As a pet project, I wrote a fairly detailed functional product-level plan for making modules and templates global, if anyone is interested: short version / long version (plug: if anyone has time to translate these pages into French, German, Spanish, Russian, or other languages, it will be very useful). I heard some support for this plan from a bunch of community members from several wikis. I'll be flattered if it's added to the "Pre-reading for all Participants" section.

What I don't have is a technical plan for implementing it. There were some proposals about this; the most notable is probably Shadow namespaces. But AFAIK none is detailed enough to cover the necessary use cases. In addition, it would make a lot of sense to break up this HUGE intimidating task to several manageable stages. This conference should be a good opportunity to do it.

I'll be happy to lead the discussion on making templates and modules global, but there should be commitment of relevant people from Core Platform, Parsing, Visual Editor, and perhaps SRE and Wikibase to participate and contribute.

Gadgets should be global, too, but this project will be completely distinct and probably much easier than templates. In fact, it's already kind of possible, albeit quite hacky, to make Gadgets global. WikidataInfo and HotCat are fairly well-known examples. So making Gadgets global is a valid topic for discussion, but its important not to mix it up with modules and templates.

I've suggested splitting out some aspect of Gadget related stuff to: T234957. Please feel free to comment there, disagree with separating, etc!

I've suggested splitting out some aspect of Gadget related stuff to: T234957. Please feel free to comment there, disagree with separating, etc!

Makes sense to me. My experience from discussing this for several years tells me that gadgets should be discussed separately, if we are actually interested in seeing a solution implemented (I know I am). I commented in more details on that task.

Tpt awarded a token.Thu, Oct 10, 9:16 AM
Tpt added a subscriber: Tpt.

I'm marked as a note taker for Tech Conf, I'm not positive what the schedule is for note takers yet but if it's at all possible I'd like to attend this session either in that capacity or generally.

Tpt added a comment.Fri, Oct 11, 7:31 AM

I agree with Amir that this is a very important topic.
For example, Wikisource workflows rely a lot on templates that are slightly different in each wikis, making very hard hard to roll out change in all wikis. It leads to an increasing technological gap between big and small wikis.
Global templates/modules might decrease the technical gap that exists between wikis.

I am not aware of any volunteer written extension deployed on Wikimedia cluster in the recent years (except maybe CodeEditor?), so the already existing "extension solution" does not seem to solve this need in practice.
However, indeed, completely duplicating the existing development workflow does not seem to be the best solution either.

To summarize, there is a huge room for discussions on how the templates/modules duplication problem should be solve, and the TechConference seems a good place to start drafting what a possible solution might be.

Bmueller added a subscriber: Tgr.Fri, Oct 11, 6:23 PM

I would guess the main issue is to make a "business case" for it, have it funded and commit to implement the product.

Hey @hashar yes, that's the idea behind the session. This is mainly the reason why templates, modules, gadgets, scripts are covered in the same session proposal. As we are probably tight with the schedule, one idea how we could do it (to pick up @Amire80's and @Tgr's and @WMDE-leszek's comments):

Developer Productivity and Onwiki Tooling (2 hours)

  • Intro, general overview, goals of the session (20 min)
  • Breakout groups: a.) T234957; b.) Modules/templates (40 min)
  • Get together in big group, breakout groups present their key results (20 min)
  • Bring the different aspects together + agree on recommendations/way forward (this will need some prep work + good planning, but should be doable) - 40 min

I'm happy to take over the framework part - intro, and the planning for the last 40 min (with input from the breakout group leaders). @Amire80 could run the modules/templates breakout group, and @WMDE-leszek would be responsible for the gadget session? Eventually we could also try to get more time in the schedule, depending on how many participants would be interested in this/how much time other sessions will need. What do you think?

Oh - I confused "@Tgr" and "@Tpt", really sorry ;) - @Tpt what do you think? :-)

I'm marked as a note taker for Tech Conf, I'm not positive what the schedule is for note takers yet but if it's at all possible I'd like to attend this session either in that capacity or generally.

Thanks @WDoranWMF! I'll add you to the task description - it should be possible unless it overlaps with the team practices session ;)

Bmueller updated the task description. (Show Details)Fri, Oct 11, 7:00 PM
Tpt added a comment.Fri, Oct 11, 7:44 PM

@Bmueller Thank you for the proposal! It looks great to me. +1 to split into two groups. Having a significant amount of time like you proposed for the last step with everyone seems very useful to me because the two groups conclusions might overlap and/or conflict significantly.

Quiddity updated the task description. (Show Details)Fri, Oct 11, 11:03 PM
Amire80 updated the task description. (Show Details)Sat, Oct 12, 8:48 AM

I went bold and added https://www.mediawiki.org/wiki/User:Amire80/Global_templates_draft_spec/TLDR to pre-reading. It's a very personal and unofficial proposal, and it explicitly excludes gadgets (even though it's totally find to discuss gadgets, too; I just want to discuss them separately in order to focus better). However, a bunch of editors from several wikis told me they liked this plan and its (much) longer version, so perhaps it can be a useful starting point. If I'm wrong about anything, please revert me :)

TheDJ awarded a token.Sat, Oct 12, 3:08 PM

I think this is pretty important for our movement. I have helped some newly created Wikipedias with their templates and modules and it is extremely lenghty and frustrating. They lack everything, and normally technical people is not among the wikimedians in that language. We should imagine a way in which every functionality is present (as images are in Commons) and you can override locally everything.

Modules and templates must be all global, at least optionally. It's not just a necessary change for the MediaWiki platform as it is used by Wikimedia; it's the most necessary change.
As a pet project, I wrote a fairly detailed functional product-level plan for making modules and templates global, if anyone is interested: short version / long version (plug: if anyone has time to translate these pages into French, German, Spanish, Russian, or other languages, it will be very useful). I heard some support for this plan from a bunch of community members from several wikis. I'll be flattered if it's added to the "Pre-reading for all Participants" section.
What I don't have is a technical plan for implementing it. There were some proposals about this; the most notable is probably Shadow namespaces. But AFAIK none is detailed enough to cover the necessary use cases. In addition, it would make a lot of sense to break up this HUGE intimidating task to several manageable stages. This conference should be a good opportunity to do it.
I'll be happy to lead the discussion on making templates and modules global, but there should be commitment of relevant people from Core Platform, Parsing, Visual Editor, and perhaps SRE and Wikibase to participate and contribute.
Gadgets should be global, too, but this project will be completely distinct and probably much easier than templates. In fact, it's already kind of possible, albeit quite hacky, to make Gadgets global. WikidataInfo and HotCat are fairly well-known examples. So making Gadgets global is a valid topic for discussion, but its important not to mix it up with modules and templates.

As a pet project, I wrote a fairly detailed functional product-level plan for making modules and templates global

@Amire80: That sounds like material for T121470 where I cannot find this link?

As a pet project, I wrote a fairly detailed functional product-level plan for making modules and templates global

@Amire80: That sounds like material for T121470 where I cannot find this link?

True. I added it there now.

Tgr awarded a token.Sun, Oct 13, 8:45 PM
Tgr added a comment.Sun, Oct 13, 9:08 PM

Super important topic that would really deserve its own session:

  • understanding the onwiki tool developer community (metrics, surveys, community maps etc - unlike say MediaWiki developers we know next to nothing about them, especially on the non-English projects, they are not included on the technical community dashboard metrics etc.)
  • developer engagement for onwiki developers (documentation, mentorship/training, capacity matching, UI learning curves, documentation, support channels etc) - our traditional efforts in these areas typically don't cover on-wiki developers well
  • supporting sane development workflows (code review, CI and manual testing, IDE integration) - gadgets and modules/templates are probably not too different in this one
  • globalization/sharing (inside and outside Wikimedia projects), tool discoverability, internationalization and customization (here, gadgets are pretty different)
  • gadgets and security (not strictly a developer productivity topic, but security restrictions necessary in the current regime result in security restrictions that limit productivity)
  • logging and monitoring for gadgets
  • building or providing design and user testing expertise for templates
  • templates and structured data (I wrote an essay on that some time ago)

There's also a (draft) movement strategy recommendation on the topic.