Page MenuHomePhabricator

Recent Posts

Visual Studio Code + Neovim

I've started using Vim in 2014. I've read a few books, wrote a few articles and gave a few talks on Vim. I really like editing text with Vim, but I was never happy with it's file management, extensions, things like that.

Read more...

Google Summer of Code, June-August 2020

June and July were pretty busy. I was on vacation the majority of August. Interns and other mentors were busy even then. For more introduction, read my post Blog Post: Google Summer of Code, February-May 2020.

Read more...

CI now updates your deployment-charts

Written by jeena on Thu, Sep 24, 5:34 PM in Doing the needful.

If you're making changes to a service that is deployed to Kubernetes, it sure is annoying to have to update the helm deployment-chart values with the newest image version before you deploy. At least, that's how I felt when developing on our dockerfile-generating service, blubber.

Read more...

Google Summer of Code, February-May 2020

In February 2020 I've noticed an e-mail that Wikimedia is participating in Google-Summer-of-Code. Unfortunately, I've ignored the e-mail and soon forgot about it.

Read more...

Production Excellence #23: July & August 2020

Written by Krinkle on Wed, Sep 23, 6:10 PM in Doing the needful.

How’d we do in our strive for operational excellence last month? Read on to find out!

Read more...

RPKI Origin Validation

Written by ayounsi on Aug 10 2020, 1:02 PM in Routing knowledge.

Since the late 90s, databases named Internet Routing Registries (IRR) have been trying to fulfill that (single) source of truth role. Unfortunately, they are subject to a lot of issues: fragmentation (many existing databases, not all equally well-maintained), security (some databases allow anyone to “claim” a prefix) and complexity (for the network operators). They also contain a lot of inaccurate data that have accumulated over time.

Read more...

Internal anycast

Written by ayounsi on Aug 7 2020, 9:48 AM in Routing knowledge.

This project brought two major changes to our infrastructure. Firstly, servers that used to be fronted by LVS for load balancing are now peering directly with our routers. Secondly, we have started using IP anycast for a highly critical service: recursive DNS.

Read more...

Phabricator Search Backend Changes

Written by mmodell on Jul 29 2020, 5:45 PM in Phabricating Phabricator.

Phabricator upstream has implemented a search engine that does not depend on an external full-text index service. It's been the default in Phabricator for quite some time, however, we have been using the ElasticSearch engine for a few years now due to previous issues with the old engine. Specifically, the old phabricator fulltext engine depended on MySQL's built-in fulltext index functionality. Unfortunately, the fulltext engine in MySQL had inconsistent performance and often returned low quality search results.

Read more...

All code is built

Written by Niedzielski on Jul 28 2020, 1:27 PM in Leave it to the prose.

HEADER CAPTION: The head of the Statue of Liberty on exhibit at the Paris World's Fair, 1878. The statue was built in France ahead of time, shipped overseas in crates, and then assembled in New York. Image by Albert Fernique / public domain.

Read more...

Production Excellence #22: June 2020

Written by Krinkle on Jul 23 2020, 3:25 AM in Doing the needful.

How’d we do in our strive for operational excellence last month? Read on to find out!

Read more...

Fanboying Cypress

Software development is pretty agile in its nature in that things normally tend to move pretty quickly. However, the faster you move, the more things break. As a codebase grows in size, its pieces become more and more complex, with every line adding a potential bug. In Wikimedia Foundation, we keep a handle on this through rigorous amounts of testing. Manual testing requires a lot of effort especially when you have a large core repository with a plethora of plugins and extensions that need to be tested. One of the hot frameworks on the scene is Cypress, a complete end to end testing solution.

Read more...

GSoCpedia: The journey so far

“Imagine a world in which every single human being can freely share in the sum of all knowledge.”
~ Wikimedia Foundation

Read more...

Addressing bug from 2019: information about private, security-related Phab tickets

Written by Dsharpe on Jul 6 2020, 5:04 PM in Wikimedia Security Team.

Today, we are writing to share the discovery and squashing of a bug that occurred earlier this year. This particular bug was also one of the rare instances in which we kept a Phabricator ticket private to address a security issue. To help address questions about when and why we make a security-related ticket private, we’re also sharing some insight into what happens when a private ticket about a security issue is closed.

Read more...

Faster source code fetches thanks to git protocol version 2

Written by hashar on Jul 6 2020, 10:57 AM in Doing the needful.

In 2015 I noticed git fetches from our most active repositories to be unreasonably slow, sometimes up to a minute which hindered fast development and collaboration. You can read some of the debugging details I have conducted at the time on T103990. Gerrit upstream was aware of the issue and a workaround was presented though we never went to implement it.

Read more...

Production Excellence #21: May 2020

Written by Krinkle on Jun 24 2020, 8:00 PM in Doing the needful.

How’d we do in our strive for operational excellence last month? Read on to find out!

Read more...

Celebrating 600,000 commits for Wikimedia

Written by Jdforrester-WMF on May 29 2020, 10:47 PM in Doing the needful.

Earlier today, the 600,000th commit was pushed to Wikimedia's Gerrit server. We thought we'd take this moment to reflect on the developer services we offer and our community of developers, be they Wikimedia staff, third party workers, or volunteers.

Read more...

Production Excellence #20: April 2020

Written by Krinkle on May 14 2020, 4:10 PM in Doing the needful.

How are we doing on that strive for operational excellence during these unprecedented times?

Read more...

Moving Milestones

Written by mmodell on May 1 2020, 10:39 AM in Phabricating Phabricator.

This week a long-standing "bug" has been fixed in Phabricator.

Read more...

Production Excellence #19: February 2020

Written by Krinkle on Mar 24 2020, 9:40 PM in Doing the needful.

How’d we do in our strive for operational excellence last month? Read on to find out!

Read more...

Coming to Terms with Change

Written by Ladsgroup on Mar 23 2020, 3:54 PM in The Sum of All Data.

We are dropping the wb_terms table, you might ask why we are doing it and why it’s such an important endeavour.

Read more...

The best documentation automation can buy

Written by Niedzielski on Mar 22 2020, 2:24 PM in Leave it to the prose.

HEADER CAPTION: Screenshot from Wikimedia's famous Visual Editor. The typo "documenation" has a red squiggly line under it indicating the spell checker has automatically detected a spelling error by the author.

Read more...

Production Excellence #18: January 2020

Written by Krinkle on Feb 28 2020, 7:39 PM in Doing the needful.

How’d we do in our strive for operational excellence last month? Read on to find out!

Read more...

New names for everyone!

Written by Andrew on Feb 18 2020, 9:14 PM in Clouds & Unicorns.

The Cloud Services team is in the process of updating and standardizing the use of DNS names throughout Cloud VPS projects and infrastructure, including the Toolforge project. A lot of this has to do with reducing our reliance on the badly-overloaded term 'Labs' in favor of the 'Cloud' naming scheme. The whole story can be found on this Wikitech proposal page. These changes will be trickling out over the coming weeks or months, but one change you might notice already.

Read more...

Parsoid in PHP, or There and Back Again

Written by cscott on Feb 12 2020, 3:56 PM in Wikitext & Parsing.

Authors: S.Subramanya Sastry, C.Scott Ananian; Parsing Team; Wikimedia Foundation

Read more...

Changes to Security Team Workflow

Written by JBennett on Feb 3 2020, 8:07 PM in Wikimedia Security Team.

In an effort to create a repeatable, streamlined process for consumption of security services the Security Team has been working on changes and improvements to our workflows. Much of this effort is an attempt to consolidate work intake for our team in order to more effectively communicate status, priority and scheduling. This is step 1 and we expect future changes as our tooling, capabilities and processes mature.

Read more...

Why does building a skin require PHP knowledge?

Written by Jdlrobson on Feb 3 2020, 6:37 AM in Leave it to the prose.

One of my longstanding pet peeves is that skin development for MediaWiki is so hard. I propose a radical change to how skins are installed and ask for feedback.

Read more...

14 January 2020 security incident on Phabricator

Written by Dsharpe on Jan 16 2020, 10:20 PM in Wikimedia Security Team.

On 14 January 2020, staff at the Wikimedia Foundation discovered that a data file exported from the Wikimedia Phabricator installation, our engineering task and ticket tracking system, had been made publicly available. The file was leaked accidentally; there was no intrusion. We have no evidence that it was ever viewed or accessed. The Foundation's Security team immediately began investigating the incident and removing the related files. The data dump included limited non-public information such as private tickets, login access tokens, and the second factor of the two-factor authentication keys for Phabricator accounts. Passwords and full login information for Phabricator were not affected -- that information is stored in another, unaffected system.

Read more...

Production Excellence #17: December 2019

Written by Krinkle on Jan 10 2020, 2:51 AM in Doing the needful.

How’d we do in our strive for operational excellence in November and December? Read on to find out!

Read more...

The journey to Prometheus 2

Written by fgiunchedi on Dec 19 2019, 2:18 PM in Observing the observable.

In July 2016 the SRE team at Wikimedia Foundation began investigating Prometheus as a modern metrics framework. The experience has been very positive and has provided benefits both for technical debt elimination—such as deprecating Ganglia and Diamond—and adding support for newer technologies like Kubernetes.

Read more...

WikimediaDebug v2 is here!

Written by Krinkle on Dec 16 2019, 6:48 PM in The Speed of Thought.

WikimediaDebug is a set of tools for debugging and profiling MediaWiki web requests in a production environment. WikimediaDebug can be used through the accompanying browser extension, or from the command-line.

Read more...

Phabricator Status Update

Written by mmodell on Dec 9 2019, 9:00 PM in Phabricating Phabricator.

You may have noticed that Phabricator's real-time notification service ("Aphlict") is currently disabled. This means that you will not see real-time notification popups in Phabricator and workboard live-updates aren't happening.

Read more...

Production Excellence #16: October 2019

Written by Krinkle on Nov 8 2019, 5:57 AM in Doing the needful.

How’d we do in our strive for operational excellence last month? Read on to find out!

Read more...

Production Excellence #15: September 2019

Written by Krinkle on Oct 24 2019, 11:25 PM in Doing the needful.

How’d we do in our strive for operational excellence last month? Read on to find out!

Read more...

Integrating code coverage metrics with your development workflow

Written by kostajh on Oct 9 2019, 10:04 AM in Doing the needful.

In Changes and improvements to PHPUnit testing in MediaWiki, I wrote about efforts to help speed up PHPUnit code coverage generation for local development.[0] While this improves code coverage generation time for local development, it could be better.

Read more...

Introducing Phatality

Written by mmodell on Oct 7 2019, 12:36 AM in Doing the needful.

This past week marks the release of a little tool that I've been working on for a while. In fact, it's something I've wanted to build for more than a year. But before I tell you about the solution, I need to describe the problem that I set out to solve.

Read more...

Production Excellence #14: August 2019

Written by Krinkle on Oct 3 2019, 4:27 AM in Doing the needful.

How’d we do in our strive for operational excellence in August? Read on to find out!

Read more...

Wikipedia's JavaScript initialisation on a budget

Written by Krinkle on Sep 18 2019, 10:46 PM in The Speed of Thought.

This week saw the conclusion of a project that I've been shepherding on and off since September of last year. The goal was for the initialisation of our asynchronous JavaScript pipeline (at the time, 36 kilobytes in size) to fit within a budget of 28 KB – the size of two 14 KB bursts of Internet packets.

Read more...

Cloud-vps Puppetmasters Moved to VMs, thanks to Krenair

Written by Andrew on Sep 18 2019, 9:54 PM in Clouds & Unicorns.

Last week, we completed a piece of long-neglected work relating to Puppet, the tool that manages the configuration of every virtual machine in our cloud. Historically, each VM has received its configuration from a physical, production server (the 'puppetmaster'). This meant that there was a constant chatter of traffic back and forth between each VM and unrelated networks and hardware sitting in Wikimedia production. Now, the puppetmasters are located on VMs, so all of that chatter is internal to Cloud Services.

Read more...

Production Excellence #13: July 2019

Written by Krinkle on Aug 30 2019, 8:08 PM in Doing the needful.

How’re we doing on that strive for operational excellence? Read this first anniversary edition to find out!

Read more...

Production Excellence #12: June 2019

Written by Krinkle on Jul 31 2019, 6:44 PM in Doing the needful.

How’d we do in our strive for operational excellence last month? Read on to find out!

Read more...

Phabricator Features, July, 2019 Edition

Written by mmodell on Jul 26 2019, 9:23 AM in Phabricating Phabricator.

This will be a brief introduction to this new feature which appeared recently on Phabricator workboards.

Read more...

Changes and improvements to PHPUnit testing in MediaWiki

Written by kostajh on Jul 16 2019, 4:13 AM in Doing the needful.

Building off the work done at the Prague Hackathon (T216260), we're happy to announce some significant changes and improvements to the PHP testing tools included with MediaWiki.

Read more...

Production Excellence #11: May 2019

Written by Krinkle on Jul 1 2019, 6:56 PM in Doing the needful.

How’d we do in our strive for operational excellence last month? Read on to find out!

Read more...

Tracking down slow event handlers with Event Timing

Written by Gilles on Jun 19 2019, 3:17 PM in The Speed of Thought.

We're taking part in the ongoing Event Timing Chrome origin trial, in order to experiment with that API early and give feedback to its designers. The goal of this upcoming API is to surface slow events. This is an area of web performance that hasn't gotten a lot of attention before, but one that can be very frustrating for users. Essentially, when slow events occur, users are trying to interact with the page and it's being unresponsive. Not a desirable user experience.

Read more...

Performance perception: correlation to RUM metrics

Written by Gilles on Jun 17 2019, 11:56 AM in The Speed of Thought.

When we set out to ask Wikipedia visitors their opinion of page load performance, our main hope was to answer an age-old question: which RUM metric matters the most to users? And more interestingly, which ones matter the most to our users on our content.

Read more...

Performance perception: the effect of late-loading banners

Written by Gilles on Jun 13 2019, 2:12 PM in The Speed of Thought.

Unlike most websites, Wikipedia and its sister projects are ad-free. This is actually one of the reasons why our performance is so good. We don't have to deal with slow and invasive third-parties.

Read more...

Production Excellence #10: April 2019

Written by Krinkle on May 31 2019, 7:21 PM in Doing the needful.

How’d we do in our strive for operational excellence last month? Read on to find out!

Read more...

Performance perception: how satisfied are Wikipedia users?

Written by Gilles on May 29 2019, 5:17 PM in The Speed of Thought.

We've recently published research on performance perception that we did last year. The micro survey used in this study is still running on multiple Wikipedia languages and gives us insights into perceived performance.

Read more...

Introducing the codehealth pipeline beta

Written by kostajh on May 14 2019, 8:29 PM in Doing the needful.

After many months of discussion, work and consultation across teams and departments[0], and with much gratitude and appreciation to the hard work and patience of @thcipriani and @hashar, the Code-Health-Metrics group is pleased to announce the introduction of the code health pipeline. The pipeline is currently in beta and enabled for GrowthExperiments, soon to be followed by Notifications, PageCuration, and StructuredDiscussions. (If you'd like to enable the pipeline for an extension you maintain or contribute to, please reach out to us via the comments on this post.)

Read more...

Nova-network is gone!

Written by Andrew on May 2 2019, 9:05 PM in Clouds & Unicorns.

A couple of week ago we finally moved the last lingering VMs in our OpenStack platform from the nova-network region to the Neutron region (Blog Post: Neutron is here!). The bulk of the work had been done a month earlier, so the final nails in nova-network's coffin felt a bit anticlimactic -- nevertheless, this is a big step that represents a huge amount of work on the part of both staff and volunteers.

Read more...

Production Excellence #9: March 2019

Written by Krinkle on Apr 21 2019, 6:51 PM in Doing the needful.

How’d we do in our strive for operational excellence last month? Read on to find out!

Read more...

Evaluating Element Timing for Images

Written by Peter on Apr 10 2019, 8:24 AM in The Speed of Thought.

In the search for a better user experience metric, we have tried out the upcoming Element Timing for Images API in Chrome.

Read more...

Quibble hibernated, it is time to flourish

Written by hashar on Mar 28 2019, 11:48 AM in Doing the needful.

Writing blog is neither my job nor something that I enjoy, I am thus late in the Quibble updates. The last one Blog Post: Quibble in summer has been written in September 2018 and I forgot to publish it until now. You might want to read it first to get a glance about some nice changes that got implemented last summer.

Read more...

Quibble in summer

Written by hashar on Mar 28 2019, 10:42 AM in Doing the needful.

Note: this post has been published on 03/28 but has been originally written in September 2018 after Quibble 0.0.26 and never got published.

Read more...

Switching production traffic to Apache Traffic Server

Written by ema on Mar 28 2019, 9:21 AM in The Traffic Blog.

The plan to replace Varnish as the on-disk HTTP cache component of our CDN with Apache Traffic Server is starting to take shape.

Read more...

Autonomous Systems performance report

Written by Gilles on Mar 27 2019, 11:07 AM in The Speed of Thought.

Today we're publishing our first report of the performance experienced by visitors of Wikimedia websites, focused on the Autonomous Systems visitors are connecting from.

Read more...

CI working group report, with recommendations of new tools to try

Written by LarsWirzenius on Mar 25 2019, 6:29 PM in Doing the needful.

The working group to consider future CI tooling for Wikimedia has finished and produced a report. The report is at https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/CI_Futures_WG/Report and the short summary is that the release engineering team should do prototype implementations of Argo, GitLab CI/CD, and Zuul v3.

Read more...

Production Excellence #8: February 2019

Written by Krinkle on Mar 21 2019, 7:11 PM in Doing the needful.

How’d we do in our strive for operational excellence? Read on to find out!

Read more...

Help my CI job fails with exit status -11

Written by hashar on Mar 21 2019, 9:52 AM in Doing the needful.

For a few weeks, a CI job had PHPUnit tests abruptly ending with:

Read more...

Migrating code from MediaWiki's ResourceLoader to Webpack

Written by Jdlrobson on Mar 13 2019, 3:15 PM in Leave it to the prose.

The lack of tooling or support for tooling has been causing problems in complicated code bases like the codebase for our mobile site, so we carved out a proposal to create a bridge from our existing codebase to a more modern one using Webpack. I'll talk about what we did and why.

Read more...

Work progresses on CI tool evaluation

Written by LarsWirzenius on Mar 8 2019, 4:59 PM in Doing the needful.

The working group to consider future tooling for continuous integration is making progress (see previous blog post J148 for more information). We're looking at and evaluating alternatives and learning of new needs within WMF.

Read more...

Choosing tools for continuous integration

Written by LarsWirzenius on Feb 28 2019, 6:27 PM in Doing the needful.

The Release Engineering team has started a working group to discuss and consider our future continuous integration tooling. Please help!

Read more...

Projects, Forms and Subtypes oh my!

Written by mmodell on Feb 28 2019, 2:24 AM in Phabricating Phabricator.

Significant new functionality just landed in the wmf/stable branch of rPHAB Phabricator which resolves some minor headaches we've been living with for quite some time.

Read more...

Phab Phebruary

Written by mmodell on Feb 22 2019, 7:23 PM in Phabricating Phabricator.

After many months with only a few minor updates deployed to the wmf/stable branch of Phabricator, we were long over-due for a major update. With All-hands 2019 behind us I was finally able to find the time to merge and deploy a huge batch of upstream changes.

Read more...

Minimal MediaWiki for frontend engineers

Written by Jdlrobson on Feb 21 2019, 7:07 PM in Leave it to the prose.

I use OSX. Vagrant has not been kind to me, but I'm hopeful that Docker will make development a lot easier for me in future.
Until then, I use MAMP which provides a pretty easy LAMP setup. I wanted to share it with other frontend engineers as this minimal setup works well for me - it's fast, it minimises the extensions I need to update and most importantly brings me closer to problems with frontend end-users are experiencing.

Read more...

Debugging production with X-Wikimedia-Debug

Written by kostajh on Feb 20 2019, 4:15 PM in The Speed of Thought.

In February 2018, a user reported that some topics created by users on Flow discussion boards were not appearing in the Recent Changes feeds, including EventStreams and the IRC-RC feed. Various automated patrol systems rely on EventStreams, so the bug meant a number of edits bypassed those systems on Flow-enabled wikis.

Read more...

Perf Matters at Wikipedia 2015

Written by Krinkle on Feb 14 2019, 4:05 PM in The Speed of Thought.

This year we achieved another milestone in our multi-year effort to prepare Wikipedia for serving traffic from multiple data centres.

Read more...

Production Excellence #7: January 2019

Written by Krinkle on Feb 13 2019, 3:53 AM in Doing the needful.

How’d we do in our strive for operational excellence last month? Read on to find out!

Read more...

Production Excellence #6: December 2018

Written by Krinkle on Jan 22 2019, 2:54 AM in Doing the needful.

How’d we do in our strive for operational excellence last month? Read on to find out!

Read more...

Gerrit now automatically adds reviewers

Written by hashar on Jan 17 2019, 4:53 PM in Doing the needful.

Finding reviewers for a change is often a challenge, especially for a newcomer or folks proposing changes to projects they are not familiar with. Since January 16th, 2019, Gerrit automatically adds reviewers on your behalf based on who last changed the code you are affecting.

Read more...

Toolforge: Trusty deprecation and grid engine migration

Written by bd808 on Jan 12 2019, 12:51 AM in Clouds & Unicorns.

Ubuntu Trusty was released in April 2014, and support for it (including security updates) will cease in April 2019. We need to shut down all Trusty hosts before the end of support date to ensure that Toolforge remains a secure platform. This migration will take several months because many people still use the Trusty hosts and our users are working on tools in their spare time.

Read more...

Code Health Metrics and SonarQube

Written by zeljkofilipin on Jan 10 2019, 2:54 PM in Doing the needful.
  1. Code Health
Read more...

Migrating tools.wmflabs.org to HTTPS

Written by bd808 on Jan 3 2019, 9:06 PM in Clouds & Unicorns.

Starting 2019-01-03, GET and HEAD requests to http://tools.wmflabs.org will receive a 301 redirect to https://tools.wmflabs.org. This change should be transparent to most visitors. Some webservices may need to be updated to use explicit https:// or protocol relative URLs for stylesheets, images, JavaScript, and other content that is rendered as part of the pages they serve to their visitors.

Read more...

Why performance matters

Written by Imarlier on Dec 12 2018, 4:21 PM in The Speed of Thought.

There are practical reasons that web performance matters. From a user perspective, a site that’s slow results in frustration, annoyance, and ultimately a preference for alternatives. From the perspective of a site operator, frustrated users are users who aren’t going to return, and that makes it more difficult to accomplish your mission (be it commercial or public service). Optimizations keep people happy, keep them coming back, and keep them engaged[1].

Read more...

Production Excellence #5: November 2018

Written by Krinkle on Dec 12 2018, 4:40 AM in Doing the needful.

How’d we do in our strive for operational excellence last month? Read on to find out!

Read more...

Production Excellence #4: October 2018

Written by Krinkle on Nov 28 2018, 5:47 PM in Doing the needful.

How’d we do in our strive for operational excellence last month? Read on to find out!

Read more...

Incident Documentation: An Unexpected Journey

Written by zeljkofilipin on Nov 22 2018, 6:06 PM in Doing the needful.

The Release Engineering team wants to continually improve the quality of our software over time. One of the ways in which we hoped to do that this year is by creating more useful Selenium smoke tests. (From now on, test will be used instead of Selenium test.) This blog post is about how we determined where the tests should focus and the relative priority.

Read more...

Bring in 'da noise, bring in defunct. It's a zombie party!

Written by dduvall on Nov 16 2018, 7:22 PM in Doing the needful.

Halloween is a full two weeks behind us here in the United States, but it's still on my mind. It happens to be my favorite holiday, and I receive it both gleefully and somberly.

Read more...

Machine learning: how to undersample the wrong way

Written by Gilles on Oct 15 2018, 1:37 PM in The Speed of Thought.

For the past couple of months, in collaboration with researchers, I've been applying machine learning to RUM metrics in order to model the microsurvey we've been running since June on some wikis. The goal being to gain some insight into which RUM metrics matter most to real users.

Read more...

translatewiki.net security incident

Written by JBennett on Oct 10 2018, 8:14 PM in Wikimedia Security Team.

What happened?
On September 24, 2018 a series of malicious edit attempts were detected on translatewiki.net. In general, these included attempts to inject malicious javascript, threatening messages and porn.

Read more...

Best friends forever

Written by Peter on Oct 3 2018, 9:43 AM in The Speed of Thought.

We use both synthetic and RUM testing for Wikipedia. These two ways of testing performance are best friends and help us verify regressions. Today, we will look at two regressions where it helped us to get metrics both ways.

Read more...

Neutron is here!

Written by Andrew on Sep 27 2018, 3:18 PM in Clouds & Unicorns.

As promised in an earlier post (Blog Post: Neutron is (finally) coming), we've started moving a few projects on our Cloud-VPS service into a new OpenStack region that is using Neutron for its software-defined networking layer. It's going pretty well! The new region, 'eqiad1', is currently very small, and growth is currently blocked by hardware issues (see T199125 for details) but we hope to resolve that issue soon.

Read more...

Production Excellence #3: September 2018

Written by Krinkle on Sep 25 2018, 6:41 PM in Doing the needful.

How’d we do in our strive for operational excellence last month? Read on to find out!

Read more...

Performance testing in a controlled lab environment - the metrics

Written by Peter on Sep 21 2018, 7:49 AM in The Speed of Thought.

One of the Performance Team responsibilities at Wikimedia is to keep track of Wikipedias performance. Why is performance important for us? In our case it is easy: We have so many users and if we have a performance regression, we are really affecting people's lives. Maybe you remember our hiring tweet from a couple of years ago?

Read more...

An introduction to Task Types in Phabricator

Written by mmodell on Sep 20 2018, 5:22 PM in Doing the needful.

This blog post will describe a bit about how we are utilizing the "Task Types" feature in Phabricator to facilitate better tracking of work and to streamline workflows with custom fields. Additionally, I will be soliciting feedback about potential use-cases which could potentially take further advantage of this feature.

Read more...

Additional details on OurMine

Written by JBennett on Sep 7 2018, 6:37 PM in Wikimedia Security Team.

The guard rails I'll be following will be around the original blog post created by Darian Patrick in November 2016. I'll do my best to fill in what gaps I can.

Read more...

Neutron is (finally) coming

Written by Andrew on Aug 22 2018, 9:00 PM in Clouds & Unicorns.

Just as we were settling into nova-network (along with other early OpenStack adopters), the core developers were already moving on. A new project (originally named 'Quantum' but eventually renamed 'Neutron') would provide stand-alone APIs, independent from the Nova APIs, to construct all manners of software-defined networks. With every release Neutron became more elaborate and more reliable, and became the standard for networking in new OpenStack clouds.

Read more...

Wikipedia goes 100% Forward Secret

Written by Vgutierrez on Aug 3 2018, 7:19 PM in The Traffic Blog.

On August 1st, we finished up a 6 week transition process which eliminated the last non-forward-secret cipher supported by our TLS termination, which was bare AES128-SHA. This change guarantees that all TLS connections to Wikipedia and its sister projects use cipher suites that provide forward secrecy. Wikipedia is the first major high-traffic site to take this step.

Read more...

2017 Toolforge survey results

Written by bd808 on Jun 29 2018, 8:54 PM in Clouds & Unicorns.

Between 2017-11-20 and 2017-12-01, the Wikimedia Foundation ran a direct response user survey of registered Toolforge users. 141 email recipients participated in the survey which represents 11% of those who were contacted.

Read more...

Mobile web performance: the importance of the device

Written by Gilles on Jun 22 2018, 2:19 PM in The Speed of Thought.

This week at our team offsite in Dublin, I looked at our performance data from an angle we haven't explored before: mobile device type. Most mobile devices expose their make and model in the User Agent string, which allows to look at data for a particular type of device. As per our data retention guidelines, we only keep user agent information for 90 days, but that's already plenty of data to draw conclusions.

Read more...

mediawiki_selenium 1.8.1 Ruby Gem Released

Written by zeljkofilipin on Jun 14 2018, 3:05 PM in Doing the needful.

It has been a while since the last mediawiki_selenium release! 💎

Read more...

Quibble in May

Written by hashar on Jun 1 2018, 8:36 PM in Doing the needful.

[Quibble] is the new test runner for MediaWiki (see the intro Blog Post: Introducing Quibble). This post is to give an update of what happened during May 2018.

Read more...

Technical Debt - The Contagion Effect

Written by Jrbranaa on May 24 2018, 11:16 PM in Doing the needful.

One particularly interesting topic discussed during the Hackathon Technical Debt session (T194934) was that of the contagious aspect of technical debt. Although this makes sense in hindsight, it's not something that I had really given much thought to previously.

Read more...

Thank you for taking the survey!

Written by egalvezwmf on May 3 2018, 7:00 PM in The surveyor.

I would like to share my deepest gratitude for everyone who responded to the Wikimedia Communities and Contributors Survey. The survey has closed for this year. The quality of the results has improved because more people responded this year. We heard from over 200 people who work in volunteer developer spaces like Phabricator, IRC, Mediawiki, mailing lists, and many others, which was a large increase from last year.

Read more...

Status Update (May 2, 2018)

Written by awight on May 2 2018, 8:05 PM in Score all the things.

Highlights

  • We've started work on JADE in earnest, and the prototype is deployed to the beta cluster where it's available for testing and tool development.
  • Draft topic prerequisites are mostly falling into place, so we should be able to get the initial model deployed this month.
  • New, dynamic ORES support table shows up-to-date information about our progress for each wiki: https://tools.wmflabs.org/ores-support-checklist/
  • ORES is served from its own cluster, which gave us a tremendous benefit in both performance and stability.
  • More ORES support for Arabic, Bengali, Catalan, Hungarian, Latvian, Swedish Wikipedia
Read more...

Run Selenium tests using Quibble and Docker

Written by zeljkofilipin on May 2 2018, 1:46 PM in Doing the needful.

Dependencies are Git Python 3, and Docker Community Edition (CE).

Read more...

Introducing Quibble

Written by hashar on Apr 30 2018, 9:09 AM in Doing the needful.

Running all tests for MediaWiki and matching what CI/Jenkins is running has been a constant challenge for everyone, myself included. Today I am introducing Quibble, a python script that clone MediaWiki, set it up and run test commands.

Read more...