Standardise procedures for deprecating public-facing code
Open, NormalPublic

Description

It's about time we thought about how we deprecate public-facing APIs used by gadgets, bots, Scribunto, and others. Our current procedures can sometimes take community tool developers by surprise, despite the length of time that code is typically marked as deprecated for. By creating a standard procedure for community outreach when deprecating code, we can increase the number of tools that get updated in a timely manner and prevent disruption for the editors that use them.

Expectations

In my experience, expectations from communities regarding deprecated code are quite different from those from MediaWiki developers. The community attitude can be summed up as:

  1. If you break it, you fix it.
  2. If you want to get my attention, leave me a message on my talk page.
  3. If you want to let the wider community know about something, post it on the relevant noticeboard.

The MediaWiki developer expectations, however, go something like this:

  1. You are responsible for fixing code you maintain. We will be nice and give you lots of time before removing deprecated code from MediaWiki.
  2. If you want to let tool developers know about deprecated code, show them a warning in the console.
  3. If you want to notify the community about something, use wikitech-l, wikitech-ambassadors and/or Tech News.

There are problems with a number of these expectations. For example, a significant number of gadgets on WMF wikis don't have an active maintainer, and even if gadgets/bots have an active maintainer, not all of them will notice warnings in the console. Also, most maintainers aren't subscribed to wikitech-l or wikitech-ambassadors, and by the time impending removals of deprecated code reach the mailing lists and Tech News, it may be too late to fix tools in time. This is particularly true for tools that are still in active use but don't have an active maintainer, as they must be updated by other technically-minded users in the community.

From the community side, it isn't practical to expect MediaWiki developers to update all tools using deprecated code. There are hundreds (thousands?) of gadgets on WMF wikis now that use deprecated code, and updating them all may take a prohibitive amount of time, depending on what needs updating and how many scripts are affected. Leaving messages on the talk pages of all users using deprecated code can also be very time-consuming. We need to think about how we can get community tools updated in a way that doesn't use excessive developer time.

Closing the gap

There are several things we can do to bring communities' and developers' expectations closer in line with each other.

  • T115341 Set a specific timetable for notifying communities about removal of deprecated code. For example, you could send notifications 6 months before removal, then 1 month before, then 1 week before, then 1 day before, then after removal.
  • Create infrastructure for tracking deprecated code so that technically-minded users can work on the backlog with lots of time to spare.
  • T35355 Create infrastructure for tracking how widely used gadgets are, so that the important ones can be fixed first, and the ones that don't matter can be left alone.
  • Create infrastructure for notifying the probable maintainers of gadgets on their talk pages.
  • Prohibit the removal of code if it is used in a gadget that is used by, say, more than 100 active users. Make it the responsibility of WMF Community Tech to update such gadgets.
  • Move very highly used gadgets to MediaWiki extensions. Good candidates for this would be Popups, HotCat, and Contribsrange.

On a longer-term basis, we could also think about extending this infrastructure to deprecate features of wikitext. This kind of backlog-tracking and community outreach would be key to finally fixing T14974. It would also be useful for replacing uses of the Timeline extension with something based on the new Graph extension.

Expected outcome

At the Developer Summit, we should decide whether we want to implement a procedure like this, and if so, what the main points of it should be. We should also decide which of the technical aspects we are willing/able to implement, and commit to a rough timetable for them. The fine details of the procedure can be worked out as part of a community consultation after the summit.

Relevant links


See Also:
T114384: Standardise procedures for deprecating public-facing code
T149727: Deprecated MediaWiki extension API functions should be moved to a compatibility library instead of being dropped entirely
T146965: [RfC] Deprecation policy for PHP code in MediaWiki
T115341: Create a standard timetable for deprecating public-facing code across all WMF projects

Related Objects

There are a very large number of changes, so older changes are hidden. Show Older Changes
MrStradivarius raised the priority of this task from to Normal.Oct 1 2015, 4:32 PM
MrStradivarius added a subscriber: MrStradivarius.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 1 2015, 4:32 PM
MrStradivarius set Security to None.

Thanks! Adding Community-Tech and Developer-Relations as they might also have some thoughts on this / experience with this.

Ricordisamoa added a subscriber: Ricordisamoa.
Qgil added a subscriber: Qgil.Oct 3 2015, 8:56 PM

Congratulations! This is one of the 52 proposals that made it through the first deadline of the Wikimedia-Developer-Summit-2016 selection process. Please pay attention to the next one: > By 6 Nov 2015, all Summit proposals must have active discussions and a Summit plan documented in the description. Proposals not reaching this critical mass can continue at their own path out of the Summit.

At the Developer Summit, we should decide whether...

In order to reach to such decision at the Summit, what would need to start being discussed now?

I wonder whether Architecture or Team-Practices have anything to do with this proposal?

Anomie moved this task from Unsorted to Non-Code on the MediaWiki-API board.Oct 13 2015, 12:44 AM

The procedure for api.php deprecation and updates is documented at https://www.mediawiki.org/wiki/Requests_for_comment/API_roadmap (could be moved to a better page I suppose).

At the Developer Summit, we should decide whether...

In order to reach to such decision at the Summit, what would need to start being discussed now?

We need to get an idea of the advantages and disadvantages of the proposals in the "Closing the gap" section, and think whether we could improve any of them. We should also think about how hard the technical aspects would be to implement. This is probably best done in individual tasks, so I'll start writing some of them up.

It would also be great to have more ideas on how we could better engage the community when we deprecate code - the more ideas the better. We can use this task as a place for brainstorming those ideas, as well as for general discussion about the idea of an overarching process for code deprecation across MediaWiki and the Wikimedia wikis.

The procedure for api.php deprecation and updates is documented at https://www.mediawiki.org/wiki/Requests_for_comment/API_roadmap (could be moved to a better page I suppose).

Thanks, I wasn't aware of that page before - I'll put it in the summary. Are there any other guidelines about deprecation lying around anywhere? This would be a good time to assemble them all in one place.

Also, I see that the RfC makes a distinction between "deprecation of major features" and "minor changes", which is something that we also need to think about here. We will want to save really large-scale notifications for things that will break gadgets and bots. If we send out a large-scale notification for every little thing, people are likely to start ignoring the messages. Are there any criteria that you use to determine whether something is minor or not, or is it more of an intuitive thing?

Isarra added a subscriber: Isarra.Oct 14 2015, 3:00 PM

The same procedures could also be potentially applied to internal interfaces commonly used by extensions and skins, which becomes particularly relevant when you consider how we encourage third party wikis write their own extensions for their own use cases, and the only way to make a custom theme (skin) is to use these interfaces as well.

ksmith added a subscriber: ksmith.

Removing Team-Practices; it doesn't seem to fall within our scope.

Elitre added a subscriber: Elitre.Oct 26 2015, 11:49 AM
Qgil added a comment.Oct 26 2015, 7:55 PM

This task is mainly about agreeing on expectations on the developers changing APIs, right?

  • How much effort are they expected to put discussing changes with the users of their APIs?
  • How much effort are they expected to put keeping backward compatibility?
  • How much effort are they expected communicating changes beyond mediawiki-api-announce?
  • How much effort are they expected to put in fixing community-maintained software breaking after the change?

Also, what are the expectations on WMF's Community Engagement and Community Tech to communicate changes and fix broken software?

ksmith removed a subscriber: ksmith.Oct 26 2015, 9:56 PM

This task is mainly about agreeing on expectations on the developers changing APIs, right?

This is a big part of it, yes. Another part is making it easier for community developers to update gadgets written by other people by keeping track of gadget usage and of instances of deprecated code.

He7d3r added a subscriber: He7d3r.Nov 5 2015, 10:46 AM
Izno added a subscriber: Izno.Nov 20 2015, 3:59 PM
Krinkle changed the status of subtask T35355: Provide interface for gadget usage statistics from Duplicate to Resolved.Dec 8 2015, 5:55 PM
Krinkle added a subscriber: Krinkle.Dec 8 2015, 7:40 PM
Qgil added a comment.Jan 12 2016, 7:16 AM

Was this topic discussed at the Summit?

Was this topic discussed at the Summit?

No, it didn't have its own session, and it wasn't discussed otherwise, as far as I'm aware.

RobLa-WMF moved this task from Inbox to External on the Architecture board.Mar 17 2016, 10:41 PM
Tgr added a subscriber: Tgr.Aug 22 2016, 5:09 PM
Tgr added subscribers: bd808, demon.Aug 22 2016, 6:58 PM

Some comments:

API can be used in multiple senses:

  • web APIs (api.php and the RESTBase APIs)
  • the signatures of core PHP methods that extension PHP code is supposed to invoke
  • the signatures of JS methods that gadgets are supposed to invoke

(Probably less confusing to call the latter two interfaces, but technically they are APIs.) mediawiki-api[-announce] is dedicated for the first; PHP interface changes are typically announced on mediawiki-announce and maybe on wikitech-l (and major changes in the release notes), and JS interface changes on wikitech-ambassadors, and maybe Tech News. The Notice system might also be used for all three kinds of changes, but it's not done consistently.


All three of those APIs are public-facing in some sense of the work, but the public they are facing is very different.

  • For PHP interface changes the audience is third-party MediaWiki site owners and developers, who tend to be harder to reach with announcements. (Also other Wikimedia teams, but we have methods for that - Scrum of Scrums, internal lists etc.) They are affected by web API and JS interface changes as well, but probably to a much smaller extent than Wikimedia tool developers. They usually don't have expectations about unmaintained code staying functional forever; they are also somewhat more in control since they can just choose not to update.
  • For JS API changes the audience is the gadget community (mostly Wikipedia admins). There is lots of unmaintained code (which probably won't change until gadget sharing features are improved). Fixing other people's stuff is much easier than for extensions or Tool labs tools so people with sufficient knowledge sometimes rally to save broken unmaintained stuff. In my experience, long deprecation times offer no benefit here: activity happens right after announcements, or when stuff breaks. (Frequent announcements probably would help.)
  • For web API changes the audience is the Wikimedia tool developer community (including gadgets) plus external clients of the Wikimedia APIs. We have near-zero knowlegde of the latter group (e.g. whether they tend to read mediawiki-api-announce).
  • There is also the SQL schema interface: schema changes break some Tool Labs tools. The current expectation seems to be that tool owners will just have to cope with that and there is no deprecation or announcement system. (OTOH schema changes are quite rare.)

I think for PHP interfaces and for api.php it's fairly standard to deprecate (ie. show warnings) and then remove in the next MediaWiki release. That's 6 months from the POV of an end user who installs new releases as they come out (it could be anywhere between 1-12 from the POV of the developer merging the patches).
@demon pointed out recently that this practice is somewhat uncomfortable for the users of LTS releases who are more concerned about stability and only upgrade when new LTS releases come out, every 24 months, so backporting deprecations to the LTS release might be helpful (when practical - often it's not). That's not a standard procedure currently, AFAIK.


There is often a significant amount of developer time spent on killing or fixing individual not-well-maintained bots after web API changes. (I imagine gadgets are less problematic because they cannot misbehave in critical way (e.g. go crazy and send 1000 requests per second), and some bots are critical infrastructure but unmaintained which is less often true for gadgets where the code is public.) This probably overlaps a fair bit with @bd808's tool labs community strengthening project, but it would be nice to have guidelines on what notification efforts are expected before stopping/banning problematic bots.

bd808 added a comment.Aug 22 2016, 7:17 PM

This probably overlaps a fair bit with @bd808's tool labs community strengthening project, but it would be nice to have guidelines on what notification efforts are expected before stopping/banning problematic bots.

Labs-announce and/or labs-l should reach all active bot developers for major announcements. Ideally bot developers will also be subscribed to mediawiki-api-announce and wikitech-l. Abandonware bots are what they are and will break.

I'm of the opinion that on-wiki admins should block misbehaving bots as quickly as possible and then reach out to the bot owner(s) via the bot's talk page. I think most of us with super powers in Tool Labs are also happy to stop the jobs for a misbehaving bot with a phabricator ticket explaining the problem and a ping on irc.

Anomie added a subscriber: Anomie.Aug 22 2016, 7:29 PM

Labs-announce and/or labs-l should reach all active bot developers for major announcements.

All active bot developers that host their bots on Labs, anyway.

GWicke updated the task description. (Show Details)Aug 22 2016, 8:11 PM

Regarding deprecating hooks, some comments by S in https://phabricator.wikimedia.org/T54027#1124173

What if Release-Engineering-Team would have veto power for any API breaks that haven't been properly announced, both to developers via deprecation flags and to the wider communities via tech News etc?

Tgr added a comment.May 22 2017, 1:57 PM

What if Release-Engineering-Team would have veto power for any API breaks that haven't been properly announced, both to developers via deprecation flags and to the wider communities via tech News etc?

Everyone with CR+-2 on Gerrit already has veto power for pretty much anything (certainly for enforcing formally accepted policies). So what you are saying really amounts to "what if RelEng would be on the hook for reviewing all API changes and verifying whether breaking ones have been properly announced" :)

greg added a subscriber: greg.

So what you are saying really amounts to "what if RelEng would be on the hook for reviewing all API changes and verifying whether breaking ones have been properly announced" :)

I pass :)

In this case the correct solution (as I understand it) would be for the relevant 'product owner' (in scare quotes) or roadmap owner is responsible for this. Like the newly formed MW Platform team for MW code deprecations. We (RelEng) aren't experts in the various software products that are released.

We're more than happy to consult on the actual standardized procedure, of course :)

Like the newly formed MW Platform team for MW code deprecations.

I'm not sure that putting MW Platform on the hook for reviewing all changes in core for changes is too much better than putting any other team on the hook for it. There's a difference between "these are people you should have review your change" and "these are people who get blamed for not watching every change anyone else submits and merges". The latter seems unnecessary to me.

greg added a comment.May 23 2017, 7:45 AM

Yeah. My initial reaction was "find the right team to be responsible" because I was thinking of the old adage "if everyone is responsible for something no one is." But maybe we can treat any standardized deprecation procedure eg T146965 as we do other best practices/standards in MW Core: those with +2 need to ensure the changes they merge comply. That is an "everyone is responsible..." type situation but it's a known variant :)

Qgil added a comment.May 23 2017, 1:11 PM

Alright, I agree it is complex. This is the main scenario that I think we should be able to avoid:

  1. Team X knows that a change is coming that will break things. Usually something marked as deprecated years ago that is finally being removed.
  2. While the removal of the deprecated code is technically fine, the communication of this change has been missing / weak.
  3. Someone (i.e. a community liaison that happens to work with that team) becomes aware of the risk of i.e. gadgets breaking and raises the problem.
  4. Everybody shrugs, the train comes, the new code is deployed, and things break.

I guess a very basic version of the above would be that Technical Collaboration could ask to hold a deployment like this for a couple of weeks until it has been properly communicated. I know nobody is stopping us from making a request like this nowadays, but somehow we don't feel empowered, and we just let the train pass, hoping for the best.