Page MenuHomePhabricator

Wikimedia Technical Conference 2019 Session: Continuous Delivery/Deployment in Wikimedia: The future of the wiki creation process
Closed, ResolvedPublic

Description

Session

  • Track: Deploying and Hosting
  • Topic: Continuous Delivery/Deployment in Wikimedia: The future of the wiki creation process

Description

The wiki creation process is very cumbersome and unstable, as can be seen for example in the comment T158730#5527658 and in some other tickets. This greatly slows down the creation of new projects and wikis in new languages. Improving this process probably needs to be in conjunction with the new Deployment Pipeline (T234641), but this topic deserves its own discussion because it has special considerations for the end users that may get lost in a big technical discussion about deployment.

Questions to answer and discuss

Question: How can the wiki creation process be stabilized and automated as much as possible? (T228745, T158730)
Significance: Without such stabilization and automation, the creation of wikis in new languages is extremely difficult for end users and unstable for DB and Site reliability engineers.

Question: Does the automation of this process have to wait for the completion of the Deployment Pipeline system, or can it be done indepently?
Significance: The completion of the Deployment Pipeline system may take years, and this issue is severely broken now.

Related Issues

  • ...
  • ...

Pre-reading for all Participants


Notes document(s)

[link to notes document (gdoc and / or etherpad)]

Notes and Facilitation guidance

https://www.mediawiki.org/wiki/Wikimedia_Technical_Conference/2019/NotesandFacilitation


Session Leader(s)

Session Scribes

  • [name]
  • [name]

Session Facilitator

  • [name]

Session Style / Format

  • [what type of format will this session be?]

Session Leaders please:

  • Add more details to this task description.
  • Coordinate any pre-event discussions (here on Phab, IRC, email, hangout, etc).
  • Outline the plan for discussing this topic at the event.
  • Optionally, include what this session will not try to solve.
  • Update this task with summaries of any pre-event discussions.
  • Include ways for people not attending to be involved in discussions before the event and afterwards.

Post-event summary:

  • ...

Post-event action items:

  • ...

Event Timeline

Amire80 created this task.Oct 15 2019, 2:49 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 15 2019, 2:49 PM
debt added a comment.Oct 15 2019, 4:15 PM

Hey @Amire80 - after chatting with Greg on this, we think it'd be better as an RFC for TechCom to take a look at (rather than having a session at TechConf).

A really important topic. Many languages are struggling with this, and even some existing languages can't create new projects when they have a couple of volunteers.

I believe the reason this area hasn't really seen much investment is due to the fact that wiki creation is still viewed as something that doesn't happen very often.
I would love this process to be a single click or single command situation, and there is no reason we can't get there, although getting there involves some challenges and cross team work.

Q: Does the automation of this process have to wait for the completion of the Deployment Pipeline system, or can it be done indepently?

I don't personally see why it would have to

I believe the reason this area hasn't really seen much investment is due to the fact that wiki creation is still viewed as something that doesn't happen very often.
I would love this process to be a single click or single command situation, and there is no reason we can't get there, although getting there involves some challenges and cross team work.

Thanks :)

You understood my intention precisely :)

Amire80 added a subscriber: mark.Oct 16 2019, 11:57 AM

Hey @Amire80 - after chatting with Greg on this, we think it'd be better as an RFC for TechCom to take a look at (rather than having a session at TechConf).

But if it is related to the Deployment Pipeline, as @mark says, and there is a session about the Deployment Pipeline, why can't this be a topic, too?

This issue clearly affects end users, some of whom are "just" editors, but some are developers of gadgets, templates, and other tools, and it also affects SRE people, who repeatedly run into issues when creating wikis.

Tgr added a subscriber: Tgr.Oct 16 2019, 1:54 PM

I wrote an essay to a previous dev summit about why I think this is a key strategic issue, which I am boldly recommending to be pre-reading:
https://www.mediawiki.org/wiki/User:Tgr_(WMF)/Sister_project_incubator

Tl;dr: we are trying to capture the sum of human knowledge, and there are many types of knowledge, not just encyclopedic knowledge, but all things from howto materials to 3D models to product information catalogs to learning materials to reviews to scientific publications to trust networks... (see also the list from the 2010 strategy for more). And we do a poor job (or rather no job at all) of experimenting with collecting those.

To illustrate that point, here is the movement's innovation in knowledge types, on a timeline:


(Wikidata took a long time to build but the plans were made in 2006.)

Fixing that would very likely involve rethinking of our (currently totally broken) project creation process, both from a social point of view and from the technical process needed for creating new wikis, cost and risk implications, and where we want those to be.

On technical side of things I can say that process of creating a wiki is extremely fragile and complicated, Improving that would be greatly appreciated. I already mentioned its biggest issues in T158730#5527658

I wrote an essay to a previous dev summit about why I think this is a key strategic issue, which I am boldly recommending to be pre-reading:
https://www.mediawiki.org/wiki/User:Tgr_(WMF)/Sister_project_incubator

I highly recommend looking at these slides.
I also agree with everything else tgr said.

On technical side of things I can say that process of creating a wiki is extremely fragile and complicated, Improving that would be greatly appreciated. I already mentioned its biggest issues in T158730#5527658

Fragile without a way of making it less fragile? Or fragile because of how we do things?
Who / which team really owns the issue of creating new projects?

On technical side of things I can say that process of creating a wiki is extremely fragile and complicated, Improving that would be greatly appreciated. I already mentioned its biggest issues in T158730#5527658

Fragile without a way of making it less fragile? Or fragile because of how we do things?

Of course there is a way. It's just a bunch of buggy scripts, and bugs can be fixed. I, sadly, don't have the skills to do it; I am, however, well aware of the effects of this given my involvement in supporting almost every group of people who has created a new wiki in recent years.

Who / which team really owns the issue of creating new projects?

As far as I know, no-one is designated, and this is part of the problem. It has always been an afterthought. No offense to all the people who do help do it, both staff and volunteers—your work is much appreciated. However, it needs better coordination.

I'd agree with the suggestion above, this topic does not really seem to fit to be a session at the Technical Conference. It seems a good idea to talk about what improvements to creation wikis in WMF universe the Deployment Pipeline could bring in the more informal/unconferencey set up. There will be room provided for such session at the Conference.

@Amire80: Thank you for proposing and/or hosting this session. This open task only has the archived project tag Wikimedia-Technical-Conference-2019.
If there is nothing more to do in this very task, please change the task status to resolved via the Add Action...Change Status dropdown.
If there is more to do, then please either add appropriate non-archived project tags to this task (via the Add Action...Change Project Tags dropdown), or make sure that appropriate follow up tasks have been created and resolve this very task. Thank you for helping clean up!

Thanks for the poke! I'm in COVID-19 mess at home, but I made myself a reminder to go over all of it and make last clean-ups ASAP. If I don't resolve it before the end of March 2020, feel free to close it yourself.

Amire80 closed this task as Resolved.Mar 24 2020, 1:29 PM
Amire80 claimed this task.