Page MenuHomePhabricator

[Dev Conference] Facilitating WikiProjects: reports, bots, gadgets, templates, wizards, etc.
Closed, ResolvedPublic


Let's figure out what the biggest technical needs are for WikiProjects (not just on English Wikipedia), and start hacking on some solutions at the Developer Conference.

Problem definition

Everything about WikiProjects is a bad hack made out of wikitext and duct tape.

WikiProject X as an IEG project has been working on researching and developing a more structured approach to WikiProjects on-wiki, but our work is all in wikitext, bots, and gadgets (maybe). This is good for prototyping, but we need more down the road for it to actually be sustainable. But such backend support and development does fall under the scope of at least one WMF team, who may be able to pick this up in full later.

The question is, what do we have, and what do we need from a software solution? WikiProject X has limited scope, and may only answer some, not all, related problems, especially with regard to architectural needs.

Expected outcome at the Summit

We need to establish the scope and direction for a sustainable, scalable software backend for WikiProjects, such that they are easy to create, use, and maintain, regardless of project, language, etc.

No more wikitext hacks.

Related links and background information

Event Timeline

kaldari raised the priority of this task from to Needs Triage.
kaldari updated the task description. (Show Details)
kaldari renamed this task from Facilitating WikiProjects: reports, bots, gadgets, templates, wizards, etc. to [Dev Conference] Facilitating WikiProjects: reports, bots, gadgets, templates, wizards, etc..Sep 28 2015, 5:32 PM
kaldari updated the task description. (Show Details)

Please check the expected fields in the description at -- although there is time to do this.

Something more urgent is: who should be invited to participate in this discussion? We are looking for remarkable volunteers that should be involved, and the deadline for requesting travel sponsorship is... October 2. Please invite them to register, or give me their details and we will invite them.

@Harej, your input on this task would be helpful. We're trying to figure out how to figure out what the biggest technical needs are for WikiProjects. Ideas?

I have sent an invitation to several volunteers at {Z205} but the list is based on Gerrit activity, and therefore developers focusing on gadgets, templates, bots, tools are not well covered. If you know developers that should receive that invitation, you can simply add them to that Conpherence.

@Qgil: It says I don't have permission to add people to the Conpherence ("Access Denied: Invitation to Wikimedia-Developer-Summit-2016. You do not have permission to edit this object."). I wanted to add Earwig.

Only members of the conpherence can add other members. In other words, you would need to join first. No problem, I have added Earwig.

I suppose a couple of the main problems we ran into with WikiProject X were the lack of notification support and no good way to do semantic content. Notifications were an issue because as much as we have this entire extension for notifications (Echo), there's just no good way to hook into that on-wiki and send people arbitrary notifications related to a wiki-specific thing, which WikiProjects generally are. And without any sort of semantic content, the only way to make a dynamic project listing what's going on, auto-updating content, suggesting things for people to do, listing and connecting users, notifying people of ongoing discussions, and whatever is bots. Now bots are great, but they're hard to set up and maintain and don't really scale well. They're basically the standard wiki workaround to the impossible.

It would also be great if we could make a more meaningful sign-up process for WikiProjects - currently we use some fancy form gadgets working with a bunch of templates, but if that could be handled server-side as an extension, it would really open up a lot of possibilities for on-boarding and further integration of WikiProjects into users' overall workflows. Ultimately an extension for handling all of this would be a huge process, but really that's where I'd like to see this all go, and if WikiProject X is successful it should lay the groundwork for exactly what we'd need from said extension.

@Isarra: Thanks for the useful ideas. Any chance you're coming to the Dev Conference?

@Isarra: Thanks for the useful ideas. Any chance you're coming to the Dev Conference?

I hope to. This is definitely something we want to discuss and see continue to move forward.

Congratulations! This is one of the 52 proposals that made it through the first deadline of the Wikimedia-Developer-Summit-2016 selection process. Please pay attention to the next one: > By 6 Nov 2015, all Summit proposals must have active discussions and a Summit plan documented in the description. Proposals not reaching this critical mass can continue at their own path out of the Summit.

Expected outcome at the Summit: Establish scope and direction for

This work can start already now, right?

Expected outcome at the Summit: Establish scope and direction for

This work can start already now, right?

Aye, this has been ongoing and will hopefully continue through the summit and after. This is essentially what WikiProject X is doing on a small scale, but our impact and scope are fairly limited; we need product buy-in across teams upstream with how to continue the conversations and move it on into real development, with, uh, more reach and stuff. Yes.

Harej triaged this task as Medium priority.Oct 20 2015, 6:04 AM

In general, WikiProjects needs better project-specific tools (notifications, newsletters, SuggestBot, identifying potential members, organizing members, etc.) and possibly changes in MediaWiki itself to accommodate WikiProjects. One example is we need a more automatic way to do change patrol across the included pages of a WikiProject. "Related changes" doesn't accommodate this without the periodic manual creation of a watchlist page.

If this proposal is a continuation of, are the participants of that project aware of it? Or who do you think should be involved in this discussion?

Yes; I informed people of this task through the WikiProject X newsletter.

This session is scheduled for 11:30 AM Pacific Standard Time in Leach Room 1.

Etherpad copy:

Session name: WikiProjects and Software Development
Meeting goal: Determine best method for providing better support for WikiProjects directly within MediaWiki:
Meeting style: Problem-solving: surveying many possible solutions
Phabricator task link:

Topics for discussion:

What we can do to do away with wikitext hacks (such as inline styling)

Support for custom css/javascript beyond editing common.css/.js

General notes
[james h] WikiProjects have been around since 2002, but there haven't been any WMF resources to support them. WikiProject Arthropods is an example: uses same tech and formats as Wiki articles to design a collaborative portal page. Modern group websites don't look like this. This is very text-heavy. WikiProjects take a lot of effort to maintain, so a lot of them collapse. You need a volunteer coordinator to maintain everything. WikiProject X is intended to provide a general tool that can be used by any WikiProject to set up and maintain their project spaces more easily. WikiProject Hampshire is an example of this new format. It's less text heavy, it doesn't show the full member list, it doesn't show everything on the main page. The idea is that it only shows the most important things. You just have to use our templates. It's about halfway complete. We started as a grant project, and we couldn't write code for MediaWiki, because there was no system in place for reviewing code for grant projects. So we went about this by making workarounds. MediaWiki is very good at accepting workarounds--bot, Lua, gadgets--but it doesn't work very well in practice. Inline styling is hard. It doesn't work on mobile very well. We avoided tables.
<James demo's the mobile site>
[people] This actually looks pretty good on mobile.
[isarra] Even doing this was horribly hard, though.
[james h] We can't deploy JS and CSS the way we need, which means our UX is not optimal. We would have to create gadgets to provide this functionality, but that's not what gadgets are for (not for full page rendering). We could use common.js, but we only need this code on a subset of pages, so we don't want it to load on every page.
[bawolff] It wouldn't be that big of a problem, actually.
[j-mo] what if there were 50 different versions, would it scale?
[volker] it's still not a good idea
[james h] Module:Load WikiProject Modules can be customized a little bit between projects, but we wanted it to be fairly standardized across projects. The question is what can we do to extend MW to support specific user experiences within specific contexts. I have some ideas: what can we do to get away from inline styles, and how can we support custom CSS and JS? One approach would be to have a lightweight WikiProject extension. It woudl create custom CSS classes and JS, conditionally loaded only on WProject pages. The issue is it isn't extensible. The other approach is to create a generalized tool for custom classes to be deployed on different pages under different circumstances. It causes security issues, but it provides flexibility so that people can create custom things as needed.A middle ground would be something for "complex pages" (not just WikiProjects, also portals and other things).
[volker] what were the biggest UI issues that you encountered? what couldn't you accomplish at all?
[james h] Inline styling doesn't allow you to do media queries [...not captured]. I don't know if we have a specific need for JS at this point, but this thing relies on a complex network of templates and modules, and it's complicated.
[isarra] formatting the whole list of wikiprojects, putting all the styles inline for every item in the list was time/resource consuming.
[james h] I had to squash the template size below 200 bytes so that it would even run.
[kaldari] there's usually a tab system in WikiProjects to reduce page length. But there's no standard.
[j-mo] does this need to be on MediaWiki? Could it be a labs-hosted tool?
[james h] we wanted to augment the Wikipedia experience, not send people off Wikipedia.
[yuvipanda] There was something a couple years ago, that let you define style guides for sets of pages. It doesn't address JS, though. There was a lot of discussion, but wasn't approved because of security issues.
[bawolff] we don't need the stylesheet scoped in the browser or a place in the DOM, just scoped to a specific set of pages
[kaldari] several Wikis have a WikiProject namespace
[volker] there was a proposal to keep a custom skin for namespaces. that's a possibility.
[volker] The discussion around SkinPerNamespace can be found at
[yuvipanda] On the RFC, you'll see an overview of the kinds of issues with this solution. It's pretty out of date, though. Have you thought of setting up a special content handler?
[james h] we thought of this for a WikiProject namespace. It's doable, may be a better solution than letting everyone write their own CSS. But it would be very complicated to set up a WikiProject content model.
[kaldari] you'd have to get it right, keep it flexible and ask the users to make sure you're meeting all their needs.
[bawolff] What's the benefit of having a custom content model for WP?
[james h] the question is, what is MORE vs LESS complicated? Custom CSS or a custom content model?
[yuvi] Custom content handler is simpler to get deployed - normal extension, nothing special. In a sane world, custom CSS is the simpler solution, but because it involves parser hacks, it's actually more complicated to implement.
[bawolff] it's a broader solution, but not as clean.
[james h] we're interested in being a new standard, so we didn't want to over-customize at the outset.
[volker] from UI standardization point, don't want another solution but look at one that might fit into your work. MW allows certain customizations with inline styles - people will always override it but that should not be the ideal. Pushing to core is an intense process.
[james h] Even making a new extension on production is hard.
[bawolff] if you did decide to do this as an extension, it probably wouldn't be that hard or controversial.
[james h] going forward, we want to abstract it from WikiProjects. We want a modularized interface with different sections, some are automatically generated by bots, others manually curated. Lots of pieces are subpages.
[kaldari] in this example (WikiProject Hampshire), the page is mostly updated by bots, not people.
[volker] I see a bigger problem. You can come up with an extension for pretty much anything, but getting it deployed is much harder.
[yuvipanda] but this case should be relatively non-controversial, and would be supported by a team that's already in place.
[bawolff] the solution you have is actually pretty good. takes advantage of the flexibility of Wikipedia. For your project, the content handler seems like the best solution.
[james h] custom CSS has usability/security risks, content handler is less flexible, but [may be better].
[JMo] What are the specific problems that the solution you're proposing would address?
[james h] Like Renaming the project (update user profiles, update links, templates), ideally it should be changed once and propagated everywhere.

[J-Mo] What is the real problem? Trying to flesh out the problem.
[james h] Inline styling is the best way to style wikiproject pages atm.
[Kaldari] Creating a wikiproject is hard. Requires a lot of effort.
[yuvipanda] let me show you an example of the Campaigns extension, which uses a content handler. This page has sone statistics, special javascript, an image gallery. If you hit edit, you see a JSON blob. You can update some things by changing the values. This allows us to build somewhat arbitrary pages within the scema. It allows you to solve the problem generally. And it is still under revision control, so you have a history. And to create a new campaign, you can just copy/paste an existing config. [explanation of how bots work with content handler approach] And this was pretty easy... it only took me a few weeks to code.
[james h] are we all in general agreement that content handler is the best way to go? If so, let's work/plan! The general idea is a modular interface. You highlight certain ideas and expand them as you need. You can associate your username with the project.
[yuvipanda] examples of non-WikiProjects that this would support?
[james h] we expect that people will use this for other things. We will design for WikiProjects, but it should be flexible enough to work for other use-cases.
[bawolff] the section about common articles... is that bot-edited?
[james h] yes!
[bawolff] would this be better handled by MediaWiki?
[kaldari] it's complicated because we're querying categories, and things are inconsistent.
[bawolff] is this enwiki only?
[james h] we want this to work for other projects too, where there is growth and need. WikiData is a good example.
[james h] here's an example of our current JSON solution: This is how a bot knows whether to run on a Wikiproject or not.
[kaldari] I'm active on a project that uses WikiProject X, and I didn't even know this file existed!
[yuvipanda] for the extension, we should just convert our current template + Lua module into a content handler. It's easy to explain. Convert the Lua to PHP, get it deployed. No new features yet, but sets you up to extend things more easily going forward.
[kaldari] will this be tied to a namespace? what about projects that aren't in that namespace?
[james h] if we don't, it could be a Flow situation, where it's inconsistent between talkpages in the same namespace.
[yuvi] you can set a content handler per page (right limited to a few people), or per namespace. we don't have to decide yet. Suggest initially doing it per-page and later changing, if needed.
[james h] what about using the content handler to let people join a project?
[yuvi] save that for "step 2". Use the bot until then.
[andyrussg] associating a user with a project was the first thing we worked on with the education extension. A db backend lets you generate notifications with less overhead, gain consent from people being associated, and handle removing people from the project more easily. DB lets you create and delete all sorts of "entities" better (including pages).
[discussion about using Gather, PageAssessments to associate people with articles]
[bawolff] categories are also used to match entities right now
[james h] they're inconsistent, and reading subcategories (for FA vs GA project articles, for instance) queries get expensive
[yuvi] so, to start: a wikipage module. then: artibitrary wikitext extensions for new module types (for member lists, for instance)
[james h] to create new modules, just define the module name and module type. Then set the value to some transclusion of a wikipage.
[kaldari] do we want an overall page architecture to start out? with a grid? like Wordpress?
[james h] at first, we probably want to use the current structure for WikiProject X
[volker] people sometimes want tools before they consider the content they want to serve. I try to design from the content. There are some technical limitations, but I think a grid-based layout is within the realm of possibility.
[yuvipanda] who owns this (technical ownership)? if there's a security issue discovered at 2am, who you gonna call?

Conclusion: We will proceed with an extension that uses contenthandler for modular content presentation and bot configuration. James Hare + Yuvi Panda will work on developing schema and scope of work for extension.