(Formerly called "MediaWiki outside Wikimedia" and merged with "Dependency management" -- let's discuss how this session is organized)
There is a range of topics to discuss when it comes to the pain points of using MediaWiki outside of large setups like Wikimedia's. A major concern is still how to best publish the software and extensions. This slot aims to collect, prioritize and coordinate the various efforts:
Topics:
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Mglaser | T86559 "Distribution of MediaWiki" session at MediaWiki Developer Summit 2015 | |||
| Resolved | Qgil | T85632 Pre-schedule all the 2015 MediaWiki Developer Summit sessions in the main room |
I have merged this session with "Dependency management" since both have a lot of overlap. The scope of this session is the distribution of MediaWiki and extensions for third parties, which implies packaging, dependency management, and releases.
Let's discuss how this session is organized. Like other sessions scheduled for the main room, the emphasis should be put in tough problems requiring cross-team collaboration, bringing other subtopics and potential workshops to smaller rooms.
I think that is confusing two different issues. Extensions (and skins!) are not dependencies. Extensions may have dependencies. core has dependencies. Combining and confusing the two just makes it more difficult to deal with. Both have different problems and may have different solutions. Finally, I don't see why we should limit ourselves just to "third-parties". The WMF has its own requirements, but our extension management system needs to scale, just like the rest of MediaWiki.
All this is fair, and I'm definitely not an expert. I just kicked a couple of balls that had many unclarities and need a resolution. Until a couple of hours ago, there was not even a good place to discuss them. Any improvement on the current situation is welcome, if needed splitting this session in two or many.
The urgent question is what goes to the main room. For the rest, the schedule is open with many available slots.
Having slept on this a bit, I think the clear topic for the main room is how to improve the experience for third party MediaWiki users when installing and upgrading MediaWiki and compatible extensions. Over the year we have got several discussions on how to solve this clear problem, and it seems that it would be useful for everybody to have a clear plan.
@Mglaser had created the task and he is a good representative of the 3rd party perspective out of the WMF Release Team. @Legoktm has a good overview of this topic, both from the perspective of third parties and from the WMF. It would be great if you could agree on the scope for this first session in the main room. Just take this task and run with it, as soon as the main scope of the session is clear, we can confirm a slot for it.
Feel free to create tasks for other topics that should be covered as well, specifying whether you want to push them for the main room or not.
Sounds good! I edited the description for the kind of session I was thinking of.
I think the main topic will end up being extension management since that is really the biggest pain point with upgrading, maintaining compatibility, etc and that is where the current momentum is.
I'm not really sure what "New forms of releasing for increasingly complex setups" is about...maybe parsoid packaging or MW packaging? Or...? :)
Parsoid, mathoid, the pdf generator, restbase, the proposed hieroglyphics server, and the potential expansion throughout 2015 and beyond of new services that run outside the web container.
The service oriented architecture track is at least in part about taking MediaWiki in a direction that is not compatible with shared hosting environments where the deployer does not have the ability to run arbitrary daemon processes. Today we already have a large collection of auxiliary services that are really only "packaged" as git repos and Puppet rules buried in operations/puppet.git and possibly mediawiki/vagrant.git. @GWicke has proposed packaging these apps as Debian installer archives as a replacement for the WMF use of Trebuchet and much of the Puppet configuration. I don't know if he has thought much about support for all the non-Debian platforms (or about Windows at all).
Unfortunately there really is no one-size-fits-all cross platform packaging standard so either a full MediaWiki deploy will become a largely manual endeavor or we need to find new ways to think about packaging services and invest in them in some manner. One solution might be to partner with various distributions and help them consume our release versions easier. Another way to go would be to choose a semi-portable container system like LXC or Docker and make things available that way. Yet another approach would be to lean heavily on something like Alien and hope we can package once and deploy to "most" Unix platforms.
@bd808 thanks for this pretty good explanation of what I meant :) I personally think a combination of puppet and docker could be the way to go: Describe the setup in puppet (potentially platform agnostic) and make shipping easy by using a docker container. But I'm getting distracted. Anyhow, what you described is what we should talk about.
@Legoktm, I am happy to run this session together with you. What I originally intended was to actually get the bigger picture. So, for example, we have the dependency management, which somehow ties in with the proposed extension management, which in turn somehow is related to questions of updating complex installations, which could use standarized packages ;o) . And, shouldn't all this be properly configurable using your new configuration system? So I think we should not get lost in the tiny bits and pieces, but try to see how the different topics and approaches tie in. Does this make sense to you?
From the experiments I did to create the MediaWiki-Vagrant Docker provider integration, Puppet and Docker are not friends. I managed to mash them together but I really wouldn't recommend that anyone run a wiki that way even for dev and especially for production. I think that native Docker packaging would be possible for a "full stack" wiki with multiple services but it requires a different way of thinking about the deployment. A Docker container is not a VM and it's really not a full LXC container either. It is its own special thing. The closest other tech I've worked a bit with is Heroku but it is different than that as well.
I thought I had read something about how well they integrate. But hey, practice beats theory ;)
A Docker container is not a VM and it's really not a full LXC container either. It is its own special thing.
This is something that I was wondering about. Most of the enterprise users I work with use virtualized environments. Question is, how well does a docker container work, say, on a vmware virtualized server, as this is basically a double virtualization.
Using puppet to configure a service in a Docker container as the basis for creating an image is probably fine. What is not fine is attempting to use a Docker container as a full VM and provision multiple services inside it using Puppet. I did get it to work to some extent with MediaWiki-Vagrant but it required doing things that the Docker maintainers have said they will not support (running an init process) and has very nasty consequences to the data held inside the container. Just as an example, when using Vagrant to manage Docker adding a new port forward to the container requires completely destroying the current container and then creating a new one that includes the new port forward. In a more Docker centric management scenario I'm not sure if you can just layer another container on the existing one as a base to provide new external ports or not.
To do things the proper Docker way, each executing process should be in its own container. This means you have a database container, a Redis container, a jobrunner container, an HHVM fcgi container and finally a web server container to build a "simple" MediaWiki stack. If you want Parsoid, that goes in yet another container. Same for ElasticSearch or Mathoid or any other long running process that may be needed to enable the stack. The database container probably needs a file system providing container as well to separate the storage of table data from the runtime services provided by MySQL/MariaDB/whatever.
Some people will want the "full" MediaWiki deploy, but it seems short-sighted to me to say that anything less has no value.
Which is to say, I'd like to have sufficient attention given to the scaled down, traditional MediaWiki installation so that it remains viable.
Maybe no one is suggesting that we abandon support for the minimalist (php-only) MediaWiki, but it seems that it is being dismissed here as unworthy. Hobbiests have found MW to be very useful. I have even seen corporate installations make good use of it -- running 1.23 happily with FCKeditor.
I'm not personally dismissing a single process PHP-only version of MediaWiki, but I'm trying to make the potential future dangers to that known so that a proper discussion is had of the pros and cons of this before someone intentionally or unintentionally makes that choice without a larger MediaWiki community discussion of the ramifications.
Illustrated the 'increasingly complex setups' bit in the description by mentioning SOA and possible directions.
It would be interesting to learn about how common running MediaWiki directly on Windows actually is, and whether VMs running on Windows can take care of that. In any case, we need to start somewhere, ideally where most of our current users are.
I didn't find anything relevant on windows usage by browsing the site or a google search. I might have missed it, but it certainly does not seem to be very prominent.
We did get feedback from Windows users running Parsoid before, so I'm sure they do exist. I'm just not sure about their relative quantity.
Opened issue on WikiApiary to track this information: https://github.com/WikiApiary/WikiApiary/issues/223
@GWicke, about 50% of my customers are running MediaWiki on Windows Servers with IIS. These are medium to large companies which rely on a Microsoft infrastructure and have no internal support for Linux. Obviously, though, they should be able to handle complex setups.
I'd like to discuss extension management as a topic on it's own. Yes, it could be discussed along with the bigger picture of dependency management, but currently most extensions are still just PHP(+JS/CSS) and don't have complex dependencies, which is something I don't see drastically changing in the near future.
But if that's different than what most people want to talk about, I can schedule it for its own session.
@Legoktm, so do you think you'll need 45 minutes for extension management? If not, we could split this session: we first focus on extension management, say for 20 minutes, then on deployment, say 15 minutes, and then 10 minutes how these technologies all could tie into one big picture.
I think one topic for 45 minutes in the main room is enough. Think that people will have questions, different opinions, etc.
I agree with Quim that that wouldn't be enough time based on my experience with the config session last year. I think splitting it into a separate session is a better idea.
To close this, tasks need to be filed for any actionable item found (none, AFAICS) and notes need to be posted for further discussion/archived in a proper place (https://www.mediawiki.org/wiki/Talk:Software_bundles seems a suitable talk).