Page MenuHomePhabricator

Huggle development environment - portable virtual box
Closed, ResolvedPublic

Description

I am now preparing a portable virtual box image of Ubuntu, that is completely prepared for Huggle development. Reason is simple: there are too many newbies who struggle to set up their development environment in order to be able to compile Huggle and to start contributing (which really is hard when you can't compile it).

Because both MacOS and Windows are proprietary I decided to use Linux. This image will contain preinstalled Ubuntu, stripped of unneeded stuff, which has all that you need in order to develop Huggle. The target image should be around 1GB of size, maybe more (probably not more than 4GB, the virtual disk has 10GB but it will be compressed).

The problem now is how we distribute this image. I want to make the download available through wikimedia labs, simply upload the image on tool labs. I don't expect that many people to download it, so it should really be a big deal.

There might a little spike during GCI (end of the year) when students might download it in order to work on GCI tasks related to Huggle.

@Andrew wanted me to let you know about this using this task.

Event Timeline

Also before you tell me: why not use vagrant etc. I want this to be newbie friendly, so that even 15 years old students from GCI can work with it. Also I don't know if vagrant is any good with packaging whole development environment.

Also I don't know if vagrant is any good with packaging whole development environment.

I'd be glad to help you add a huggle role to MediaWiki-Vagrant that clones the necessary git repos and installs any needed libraries and tools for development. One advantage of this would be that your Huggle development environment would then be integrated with the same environment that is promoted for developing MediaWiki and other Wikimedia FLOSS projects.

Also I don't know if vagrant is any good with packaging whole development environment.

I'd be glad to help you add a huggle role to MediaWiki-Vagrant that clones the necessary git repos and installs any needed libraries and tools for development. One advantage of this would be that your Huggle development environment would then be integrated with the same environment that is promoted for developing MediaWiki and other Wikimedia FLOSS projects.

OK we might do that (in a separate task), but something tells me that this would take more time and also probably would work that easy, keep in mind some of GCI students are using Windows and have no idea what Vagrant is.

In order to develop Huggle on Windows, you need to download massive amount of 3rd party libraries as well as Visual Studio community edition 2015. Therefore having a VirtualBox with Linux that has everything, including IDE preinstalled and preconfigured, so that student can just download it, and run in VirtualBox and start working on Huggle straight away, is probably easier option for them. But that's just what I think.

I have no issues packaging Huggle within MediaWiki vagrant, but something tells me that @Addshore would be more interested in that, than me :)

Petrb triaged this task as Medium priority.Sep 30 2017, 5:34 PM

I think there may be some misunderstanding here about what MediaWiki-Vagrant and Vagrant are. Your description of "portable virtual box image of Ubuntu" is exactly what MediaWiki-Vagrant provides, with the small difference of Debian Jessie rather than some version of Ubuntu as the base OS image in the provisioned vm.

Vagrant is an application for managing provisioning virtual machines in a repeatable way. By default it works with Virtualbox as the virtual machine provider, but it can also manage other virtual machine/container providers like LXC, Hyper-V, Parallels, etc.

MediaWiki-Vagrant is a collection of configuration for use with Vagrant that provisions a virtual machine running Debian Jessie. Inside that virtual machine it adds Apache, MySQL, PHP, HHVM, Redis, and some basic development tools. It also clones MediaWiki and sets up a wiki farm environment that initially contains a single wiki. This environment has been in use by contributors to Wikimedia technical projects for over 4 years. It is well documented on mediawiki.org and has been tested and used by developers on a range of Linux distributions, OSX, and Windows. It has been the basis for training new contributors at hackathons since the 2014 Zürich hackathon.

A MediaWiki-Vagrant "role" is a Puppet manifest that describes how to add some additional functionality to the basic mw-vagrant environment described above. The majority of current roles add something to the wiki farm environment like CentralAuth, Visual Editor, Flow, etc. There are however a number of roles that add development environments for non-MediaWiki applications. These include fairly simple PHP applications like Wikimedia-Wikimania-Scholarships and
Wikimedia-IEG-grant-review as well as more complex application stacks like the development environment for Striker which provisions an LDAP server, a Phabricator instance, and some OpenStack services.

The development effort of a new MediaWiki-Vagrant role is typically low if the developer is already comfortable with scripting deployments using Puppet. This is why I volunteered to help implement the role. If you can give me a list of the development packages you would want installed, the git repos you want cloned, and any specific configuration you want for the dev tools I should be able to turn that into the start of the role in short order (a couple hours of work at most, likely more like 20 minutes).

I'm not on a crusade to make you use MediaWiki-Vagrant however. I was merely offering to help.

Oh in that case I was slightly confused, I always heard about vagrant in context of containers, like LXC, where various services were put into container for easy deployment on servers mostly, or desktops.

I didn't know you already have some VM preinstalled. In that case this makes sense, but on other hand, Huggle requires full stack of qt5 development packages, which is about 500MB in size, meaning your current image would grow up by ~500MB (maybe less if you use compression). And I am not sure if other MediaWiki devs would be happy to see their beloved MediaWiki dev VM get so much bigger just because of a tiny side project like Huggle, that little amount of people really care about :)

BTW, I finished setting up that VM, it's here: http://tools.wmflabs.org/huggle/files/huggle.ova and it's slightly bigger than expected ~3GB

I don't think many people would download it, so it shouldn't put big load on servers, but if it does, feel free to delete that file.

The QT dependencies would only be downloaded and installed by Puppet if the user ran vagrant roles enable huggle; vagrant provision. No bloat to the base image required.

I will definitely have a look on your current vagrant setup to understand how I could use it for this purpose.

BTW, I finished setting up that VM, it's here: http://tools.wmflabs.org/huggle/files/huggle.ova and it's slightly bigger than expected ~3GB

I don't think many people would download it, so it shouldn't put big load on servers, but if it does, feel free to delete that file.

It would probably be nicer for the servers if you placed the file in ~tools.huggle/www/static and linked to it as https://tools-static.wmflabs.org/huggle/huggle.ova.

Can you give me a full path? The home is /data/project/huggle and in this folder there is no "www"

We need to avoid users downloading this off an NFS filesystem.

Can you give me a full path? The home is /data/project/huggle and in this folder there is no "www"

$ become huggle
$ mkdir -p $HOME/www/static
$ mv $HOME/public_html/files/huggle.ova $HOME/www/static

We need to avoid users downloading this off an NFS filesystem.

In that case we can't really host it anywhere in Toolforge. The only content in tools-static that is not backed by NFS is the /cndjs/ content.

Using a MediaWiki-Vagrant role neatly bypasses the large binary image download problem by instead distributing the server load to apt servers and git servers which are better tuned for bulk file distribution.

Well, do you realize that it's only going to be a few of people who would eventually download this? I am not distributing it to millions :)

In order to keep big files off of NFS, I've created a static download site for things like this. Your file, for example, is now:

http://download.wmflabs.org/huggle.ova

When you have a moment, please delete the file off of NFS. For the moment, when you rev that file you'll need to ask one of the WMCS team directly... if that turns out to be a problem then we'll consider other options.

Petrb claimed this task.

@Petrb I am trying to get familiar with the portable development environment, but I couldn't find the password for the VM anywhere in the developer docs. Can you please help me out?