Recent Posts

Announcing the Scoring Platform team

Written by Halfak on Fri, Jul 21, 4:46 PM in Score all the things.

The Wikimedia Foundation’s new Scoring Platform team, led by Aaron Halfaker, will be working on democratizing access to AI, developing new types of AI predictions, and pushing the state of the art with regards to ethical practice of AI development.


Toolforge Elasticsearch upgraded to 5.3.2

Written by bd808 on Fri, Jul 14, 12:49 AM in Clouds & Unicorns.

The shared Elasticsearch cluster hosted in Toolforge was upgraded from 2.3.5 to 5.3.2 today (T164842). This upgrade comes with a lot of breaking API changes for clients and indexes, and should have been announced in advance. @bd808 apologizes for that oversight.


Status update (July 11th, 2017)

Written by Halfak on Wed, Jul 12, 10:44 PM in Score all the things.

Two outages with documentation. Revscoring 2.0 coming with better model information and "thresholds". New support for Romanian, Albanian, Tamil, Greek, and Bengali. We're officially welcoming @awight to the team!


Official Debian Stretch image now available

Written by Andrew on Jun 20 2017, 4:00 PM in Clouds & Unicorns.

Debian Stretch was officially released on Saturday[1], and I've built a new Stretch base image for VPS use in the WMF cloud. All projects should now see an image type of 'debian-9.0-stretch' available when creating new instances.


The journey to Thumbor, part 1: rationale

Written by Gilles on Jun 20 2017, 3:33 PM in The speed of thought.

We are currently in the final stages of deploying Thumbor to Wikimedia production, where it will generate media thumbnails for all our public wikis. Up until now, MediaWiki was responsible for generating thumbnails.


Watroles returns! (In a different place and with a different name and totally different code.)

Written by Andrew on Jun 20 2017, 3:26 AM in Clouds & Unicorns.

Back in the dark ages of Labs, all instance puppet configuration was handled using the puppet ldap backend. Each instance had a big record in ldap that handled DNS, puppet classes, puppet variables, etc. It was a bit clunky, but this monolithic setup allowed @yuvipanda to throw together a simple but very useful tool, 'watroles'. Watroles answered two questions:


Looking back: improvements to edit save time

Written by Gilles on Jun 12 2017, 9:32 AM in The speed of thought.

The WMF's financial year and its annual plan are coming to an end, and one of the Performance team's goals this past year was to reduce the amount of time it takes to save an edit on a wiki.


Improving time-to-logo performance with preload links

Written by Gilles on Jun 7 2017, 7:38 AM in The speed of thought.

One of the goals of the Wikimedia Performance Team is to improve the performance of MediaWiki and the broader software stack used on Wikimedia wikis. In this article we’ll describe a small performance improvement we’ve implemented for MediaWiki and recently deployed to production for Wikimedia. It highlights some of the unique problems we encounter on Wikimedia sites and how new web standards can be leveraged to improve performance.


#wikimedia-labs irc channel renamed to #wikimedia-cloud

Written by bd808 on Jun 5 2017, 3:11 PM in Clouds & Unicorns.

The first very visible step in the plan to rename things away from the term 'labs' happened around 2017-06-05 15:00Z when IRC admins made the #wikimedia-labs irc channel on Freenode invite-only and setup an automatic redirect to the new #wikimedia-cloud channel.


Status update (June 3rd, 2017)

Written by Halfak on Jun 3 2017, 8:24 PM in Score all the things.

Updates now coming to the phame blog! We made presentations and gathered new collaborators at the Wikimedia Hackathon 2017 in Vienna. ORES is back in api.php. Wikilabels has stats. ORES in CODFW fell over for a while, but it's back.


Join my Reddit AMA about Wikipedia and ethical, transparent AI

Written by Halfak on Jun 3 2017, 6:35 PM in Score all the things.

I wanted to let you know about an upcoming experimental Reddit AMA ("ask me anything") chat we have planned. It will focus on artificial intelligence on Wikipedia and how we're working to counteract vandalism while also making life better for newcomers.


AI Wishlist initialized and a new Phab Tag (January 31st, 2017)

Written by Halfak on Jun 3 2017, 6:21 PM in Score all the things.

I hosted the AI Wishlist session at the Developer Summit(T147710). At that session, we brainstormed a set of AIs that we think would be interesting to implement. Generally I asked people to do their best to follow template that would help us remember why the AI was important, what it would help with, and what resources might help get it implemented. See artificial-intelligence


Deployment of ORES review tool in Englis Wikipedia as a beta feature (August 23rd, 2016)

Written by Halfak on Jun 3 2017, 4:51 PM in Score all the things.

We The Revision Scoring Team
are happy to announce the deployment of the ORES review tool as a beta feature on *English Wikipedia*. Once enabled, ORES highlights edits that are likely to be damaging in Special:RecentChanges, Special:Watchlist and Special:Contributions to help you prioritize your patrolling work. ORES detects damaging edits using a basic prediction model based on past damage.


Investigating a performance improvement

Written by Gilles on Jun 2 2017, 10:02 AM in The speed of thought.

Last week @Jdlrobson pinged me by email about a performance improvement his team noticed for large wiki articles on the mobile site in our synthetic tests run on WebPageTest. The improvement looked like this, a sudden drop in SpeedIndex (where lower is better):


New feature: Embed videos from Commons into Phabricator markup

Written by mmodell on Jun 1 2017, 11:49 PM in Doing the needful.

I just finished deploying an update to Phabricator which includes a simple but rather useful feature:


Updated `webservice` command deployed

Written by bd808 on May 31 2017, 8:18 PM in Clouds & Unicorns.

The v0.37 build of rOSTW operations-software-tools-webservice has been deployed to Toolforge hosts and Tools-Kubernetes Docker images.


Project-wide sudo policies in Horizon

Written by Andrew on May 30 2017, 8:02 PM in Clouds & Unicorns.

When @Ryan_Lane first built OpenStackManager and Wikitech, one of the first features he added was an interface to setup project-wide sudo policies via ldap.


Manage Instance on Horizon (only)

Written by Andrew on May 26 2017, 7:40 PM in Clouds & Unicorns.

For nearly a year, Horizon has supported instance management. It is altogether a better tool than the Special:NovaInstance page on Wikitech -- Horizon provides more useful status information for VMs, and has much better configuration management (for example changing security groups for already-running instances.)


Experimental Debian Stretch image now available

Written by Andrew on May 25 2017, 5:47 PM in Clouds & Unicorns.

I've just installed a new public base image, ' debian-9.0-stretch (experimental)' and made it available for all projects. It should appear in the standard 'Source' UI in Horizon any time you create a new VM.


Labs Openstack upgrade on Tuesday, 2016-08-02, 16:00 UTC

Written by Andrew on Aug 1 2016, 5:01 PM in Clouds & Unicorns.

Andrew will be upgrading our Openstack install from version 'Kilo' to version 'Liberty' on Tuesday the 2nd. The upgrade is scheduled to take up to three hours. Here's what to expect:


Sponsored Phabricator Improvements

Written by mmodell on Jul 27 2016, 10:44 AM in Doing the needful.

In T135327, the WMF Technical Collaboration team collected a list of Phabricator bugs and feature requests from the Wikimedia Developer Community. After identifying the most promising requests from the community, these were presented to Phacility (the organization that builds and maintains Phabricator) for sponsored prioritization.


Labs is auditing and removing inactive projects.

Written by chasemp on Jul 8 2016, 4:20 PM in Clouds & Unicorns.

If you are exclusively a user of tool labs, you can ignore this post. If you use or administer another labs project, this REQUIRES ACTION ON YOUR PART.


Kubernetes Webservice Backend Available for PHP webservices

Written by chasemp on Jul 8 2016, 4:19 PM in Clouds & Unicorns.

The Kubernetes ('k8s') backend for Tool Labs webservices is open to
beta testers from the community as a replacment for Grid Engine


Community Consultation on Labs Terms of Use: Round 1

Written by chasemp on May 26 2016, 6:55 PM in Clouds & Unicorns.

The Wikimedia Legal team is interested in revising, updating, and clarifying the existing Labs Terms of Use governing developers and their projects on labs.


Code Review Office Hours

Written by mmodell on May 9 2016, 9:50 PM in Doing the needful.

Starting Thursday May 12th, 13:00 PDT ( 20:00 GMT ) we will be having the first weekly Code Review office hours on freenode IRC in the #wikimedia-codereview channel.


Horizon is now the best UI for Labs/Tools

Written by chasemp on Apr 4 2016, 9:35 PM in Clouds & Unicorns.

horizon (OpenStack Dashboard) is the canonical implementation of OpenStack’s Dashboard, which provides a web based user interface to OpenStack services.


New bastion at

Written by chasemp on Apr 4 2016, 7:51 PM in Clouds & Unicorns. is on a new bastion host with twice
the RAM and CPU of the old one. This should hopefully provide a better
bandaid against it getting overloaded up. More discussion about a
longer term solution at


Kubernetes to 1.2 on Tuesday, 2016-04-05

Written by chasemp on Apr 1 2016, 5:30 PM in Clouds & Unicorns.

On Tuesday, 2016-04-05, we'll be upgrading Kubernetes to 1.2 and using
a different deployment method as well. While this should have no user
facing impact (ideally!) the following things might be flaky for a
period of time on that day:


What's new: Lots of improvements on

Written by mmodell on Feb 23 2016, 12:23 AM in Doing the needful.

Not a lot has changed for Wikimedia's instance of Phabricator over the past few months. That's because a lot has been happening behind the scenes, as well as upstream at Phacility. Members of the Release-Engineering-Team and Team-Practices group have been working since December 2015 to integrate various upstream changes, however, nothing was released to our production instance because there were so many important features that were in-progress and not yet fully usable. Additionally, we had to figure out exactly how these features would fit with the specific needs of our project and test a lot of functionality to be sure that we would not break anyone's workflows.