qgil wrote on 2014-04-22 18:02:51 (UTC)

It looks like Phabricator provides enough elements for us to define this process.

aklapper wrote on 2014-04-23 06:37:15 (UTC)

+1 for using a project and making people responsible for backports receive automatic notifications about it, or query that project regularly (social thing to solve, not technical).

qgil wrote on 2014-04-23 13:51:20 (UTC)

"query that project regularly" = Herald

We still need to play with that tool and see what can it offer.

aklapper wrote on 2014-05-26 13:44:20 (UTC)

Greg: Any opinions here?

Currently I favor creating a generic "Backport to WMF servers" project and making you subscribe to that project's notifications by you setting up a rule in Maniphest's Herald.

(Note to myself: Updating the docs on https://www.mediawiki.org/wiki/Backporting_fixes is covered by T350)

jdforrester wrote on 2014-05-29 15:36:02 (UTC)

In T167#19, @Aklapper wrote:

Greg: Any opinions here?

Currently I favor creating a generic "Backport to WMF servers" project and making you subscribe to that project's notifications by you setting up a rule in Maniphest's Herald.

We'd discussed creating a project for each deployment – so "WMF server deployment 2014-05-29 (1.24wmf7)" etc.. This would potentially leave us with a lot of projects, but it would map well to the workflow of "are there any outstanding code commits for deployment?" which is common.

greg wrote on 2014-05-29 21:52:29 (UTC)

In T167#21, @jdforrester wrote:

We'd discussed creating a project for each deployment – so "WMF server deployment 2014-05-29 (1.24wmf7)" etc.. This would potentially leave us with a lot of projects, but it would map well to the workflow of "are there any outstanding code commits for deployment?" which is common.

I think that might be overkill initially, honestly. But I can be convinced otherwise.

In T167#19, @Aklapper wrote:

Greg: Any opinions here?

Currently I favor creating a generic "Backport to WMF servers" project and making you subscribe to that project's notifications by you setting up a rule in Maniphest's Herald.

I like this. Also a "Backport to MediaWiki Releases" project that the Release Team subscribe to.

greg wrote on 2014-05-29 22:17:33 (UTC)

In T167#24, @jdforrester wrote:

Also, I'd presume (hope!) that it would allow us to no longer create mw:MediaWiki 1.24/wmf7/Changelog and its ilk, but instead have it neatly and usefully listed on the project "WMF server deployment 2014-05-29 (1.24wmf7)".

I'm convinced.

aklapper wrote on 2014-05-30 15:37:56 (UTC)

We'd discussed creating a project for each deployment – so reproducible etc.. This would potentially leave us with a lot of projects, but it would map well to the workflow of "are there any outstanding code commits for deployment?" which is common.

I've also copied this comment to T166 as it relates not only to backporting. If we managed to automatically add a "WMF server deployment 2014-05-29 (1.24wmf7)" project to tasks whose status is set to resolved based on a timeframe, that would be lovely and would drop the need for a separate "Backport to WMF servers" project, as one could query for open tickets where that project was manually added.

jdforrester wrote on 2014-05-30 15:55:32 (UTC)

In T167#26, @Aklapper wrote:

We'd discussed creating a project for each deployment – so reproducible etc.. This would potentially leave us with a lot of projects, but it would map well to the workflow of "are there any outstanding code commits for deployment?" which is common.

I've also copied this comment to T166 as it relates not only to backporting. If we managed to automatically add a "WMF server deployment 2014-05-29 (1.24wmf7)" project to tasks whose status is set to resolved based on a timeframe, that would be lovely and would drop the need for a separate "Backport to WMF servers" project, as one could query for open tickets where that project was manually added.

Yeah; I was thinking that it would also answer the "what things are being proposed for backport to MW 1.23?" with a "MediaWiki 1.23" project.

jdforrester wrote on 2014-05-30 15:56:48 (UTC)

In T167#23, @greg wrote:

In T167#21, @jdforrester wrote:

We'd discussed creating a project for each deployment – so "WMF server deployment 2014-05-29 (1.24wmf7)" etc.. This would potentially leave us with a lot of projects, but it would map well to the workflow of "are there any outstanding code commits for deployment?" which is common.

I think that might be overkill initially, honestly. But I can be convinced otherwise.

If we don't do it for WMF-wide we'll need to do it for VisualEditor at least to preserve the existing milestones which are used to tracking bugs, regressions, patches and so on… The feedback we've had from community members is that the clarity of deployment target on every bug (for Phabricator, read: task) and patch is that it's been very helpful and clear for them in understand what has changed and when.

Also, I'd presume (hope!) that it would allow us to no longer create mw:MediaWiki 1.24/wmf7/Changelog and its ilk, but instead have it neatly and usefully listed on the project "WMF server deployment 2014-05-29 (1.24wmf7)".

Nemo_bis wrote on 2014-06-12 06:40:48 (UTC)

Discussion above is focused on 1 out of 3 backport processes. Were MediaWiki release managers asked to comment? https://www.mediawiki.org/wiki/Backporting_fixes#Backports_to_stable.2Fsupported_release
(Added Markus; I don't find hexmode.)

aklapper wrote on 2014-07-16 14:57:33 (UTC)

In T167#28, @Nemo_bis wrote:

Discussion above is focused on 1 out of 3 backport processes. Were MediaWiki release managers asked to comment?

The "backport to deployed version on WMF servers" process isn't that different to the "backport to stable branch for next tarball release" process and feels sufficiently covered to me.

As written in T166, subprojects in https://secure.phabricator.com/T3670 will probably the way to go here too.

greg wrote on 2014-07-16 16:58:31 (UTC)

In T167#32, @Aklapper wrote:

In T167#28, @Nemo_bis wrote:

Discussion above is focused on 1 out of 3 backport processes. Were MediaWiki release managers asked to comment?

The "backport to deployed version on WMF servers" process isn't that different to the "backport to stable branch for next tarball release" process and feels sufficiently covered to me.

+1 to "backport to WMF servers" and "backport to stable releases" are basically the same; I see zero substantial difference. So that's 2 of 3 use cases.

The last use case, security releases, will be handled differently than the other 2 use cases, and it basically won't change in the world of Phab. We'll still have a Security private project, and we'll still make security releases from those bugs, er, tasks. We don't have a "Backport to Security Release" flag because all security bugs are private and in the same queue together already, a flag would be redundant.

Mglaser wrote on 2014-07-17 09:29:33 (UTC)

+1 to @greg wrt security releases.

Another question is whether we need to be able to differentiate between backport to current stable and backport to legacy/lts. In particular, this would be useful during pre-release / RC phases, where we want to get some of the master changes still into the tarball. Using a separate flag might be a bit over-organized, though.

aklapper wrote on 2014-08-18 13:54:53 (UTC)

So it sounds like James and Greg want separate "WMF server deployment 2014-05-29 (1.24wmf7)"-style projects for each weekly deployment.

I have not tried yet if we can add such a "deployment of the week"-project by default: "If task is closed as "resolved" in certain timeframe, automatically set project X" for the sake of getting release notes by running one Maniphest query. Need to play more with Herald rules which are pretty powerful.

Question1: If above was possible (release notes) and we automatically set such a project: If a task with that "deployment of the week"-project set is reopened (e.g. not entirely fixed) and that project entry isn't removed from the task, wouldn't the next person looking at the task interpret it as a backport request, if I understand the proposed logic correctly?

So currently the summary would currently be:

• Security issues will be handled via a "Security" project. Out of discussion/focus here.
• For non-security fixes/bugs that are candidates to backport, we are going to use projects/tags:
  • "WMF_server_deployment_2014-05-29_(1.24wmf7)" (specific)
  • "MediaWiki_tarball_release" (generic, we can still change that)
• People can and should follow (=receive mail notifications from Phabricator) these projects by setting up a personal Herald rule in Phabricator. Question2: I do not see a way in Herald to define a rule "Projects | name includes the string | WMF_server_deployment" though, so somebody tell me I'm wrong that this would require subscribing to each project created every week? Uhm.
• General info: Projects can be marked as archived (e.g. tarballs that have been released) so they are not offered anymore.

Fallback: If my questions above indeed do make things complicated, we could also start simple and stupid with a generic "Backport_to_WMF_servers" project, and could switch to a more sophisticated process later on when Phabricator provides more functionality (cf. release notes) needed.

General reminder on the old "one task can only have one task status" vs. "we have 'branches' being git master, versions on servers, and tarballs" problem: Each Project board allows custom columns, and a task can belong to more than one project.

(And now to bikeshed about our favorite icon and color. Not!)

aklapper wrote on 2014-08-27 19:20:32 (UTC)

@greg, @jdforrester: Any comments? I'm leaning towards "keep it simple and stupid" at the beginning, due to the open questions in my previous comment which sound cumbersome and error-prone.

aklapper wrote on 2014-08-27 19:23:17 (UTC)

And to summarize what this task is about in one sentences: "define a process to mark tasks which have merged commits in git master and those commits should be backported into 1.24wmf123 and deployed on the Wikimedia servers, and automatically notify those folks doing backports and deployments by adding such a "backport" project to such a task"

greg wrote on 2014-08-28 22:27:10 (UTC)

In T167#36, @Aklapper wrote:

So it sounds like James and Greg want separate "WMF server deployment 2014-05-29 (1.24wmf7)"-style projects for each weekly deployment.

something, even just 1.XXwmfYY would be fine with me (dates may change due to holidays etc, so predicting them is hard, and we may change our deploy cycle/cadence in the future, too)

Question1: If above was possible (release notes) and we automatically set such a project: If a task with that "deployment of the week"-project set is reopened (e.g. not entirely fixed) and that project entry isn't removed from the task, wouldn't the next person looking at the task interpret it as a backport request, if I understand the proposed logic correctly?

If a task is reopened then who ever is triaging those should move the project to the latest 1.XXwmfYY project. That should be clear enough.
(This is where I want Launchpad's per project status... but I digress)

Question2: I do not see a way in Herald to define a rule "Projects | name includes the string | WMF_server_deployment" though, so somebody tell me I'm wrong that this would require subscribing to each project created every week? Uhm.

I couldn't find anything useful. I assume some script somewhere will be creating this projects (not Phab itself), so maybe the logic of who should be subscribed could stay in that script for now?

Fallback: If my questions above indeed do make things complicated, we could also start simple and stupid with a generic "Backport_to_WMF_servers" project, and could switch to a more sophisticated process later on when Phabricator provides more functionality (cf. release notes) needed.

I'm fine with that.

Mglaser wrote on 2014-08-28 22:33:43 (UTC)

Sounds like a good start. I assume the project "MediaWiki_tarball_release" will be generic and always points to the next maintenance release. It is then within the scope of the release team to decide whether to backport this to legacy and lts releases as well, right?

We need to define the process to be applied here and now.

In T101#21, @flimport wrote:

greg wrote on 2014-08-28 22:27:10 (UTC)

Question2: I do not see a way in Herald to define a rule "Projects | name includes the string | WMF_server_deployment" though, so somebody tell me I'm wrong that this would require subscribing to each project created every week? Uhm.

I couldn't find anything useful. I assume some script somewhere will be creating this projects (not Phab itself), so maybe the logic of who should be subscribed could stay in that script for now?

Such script doesn't exists today...

Fallback: If my questions above indeed do make things complicated, we could also start simple and stupid with a generic "Backport_to_WMF_servers" project, and could switch to a more sophisticated process later on when Phabricator provides more functionality (cf. release notes) needed.

I'm fine with that.

So this might be it for now.

Alright, let's keep it simple for the start: Create a "Backport_to_WMF_servers" project in Phabricator and ask interested people (@greg, SWAT) to become members of that project (so they are subscribed to the project) or even "watch" that project (more or less the same).

Anybody can propose a backport for the fix of a task by adding "Backport_to_WMF_servers" to the Projects list for a task (similar to setting that flag in Bugzilla which also wasn't very discoverable, so not much worse).
We'll fall back to adding comments (and closing tasks) for clarification when a commit has been backported.
It's not better or worse than Bugzilla currently, where there's also some ambiguity whether to keep a ticket open or resolved when a fix has been merged into git master vs. all the wmf branches: It boils down to defining social workflows.

Documentation in https://www.mediawiki.org/wiki/Backporting_fixes will need updates.

We can still iterate on a more sophisticated solution after having migrated Bugzilla content.

Could we simplify the name? WMF-Backports?
What about backports to MediaWiki releases? MediaWiki-Backports?

WMF-Backports?

No strong opinion - just not sure if that's clear enough, keeping in mind that we will also have projects like "Wikimedia-Site_requests" after importing from Bugzilla, see T43

MediaWiki-Backports?

...and that would be in a Product-Component row with stuff like "MediaWiki-Categories", "MediaWiki-Page_editing" and so on, but the different color and tag (like a keyword) might be sufficient.

No strong opinion either, and no willingness to bikeshed at all. Just trying to apply a principle of starting simple, knowing that we can write more complex names if the simple are confusing.

Project naming:

"WMF-Backports" and "MediaWiki-Backports" doesn't feel explanatory enough to me. Sounds too similar to other subprojects which have a defined codebase area. "WMF-Server-Backports" and "MediaWiki-Tarball-Backports" will work for me. Let's go for that?

Bugzilla Backport Flags migration aspect only:

Tickets with any of our two Bugzilla flags (Backport_WMF and Backport_to_Stable) set will not have the flag information migrated to Phabricator. I don't see any historical need to know whether something was backported at some undefined point in the past (and that information is available in comments anyway if somebody is curious).

Regarding currently open tickets with Backport requests that have not been decided on, there are less than 10 of them hence the effort is not worth it to write migration code covering that.
I have cleaned up WMF backport requests yesterday and I've sent an email today to the Tarball Release Team to please take a look at the queue of undecided tarball backport requests.
Furthermore Bugzilla will be available as read-only for some time after the migration so we could still look up which Bugzilla tickets had this flag if we wanted.

In T101#9607, @Aklapper wrote:

Project naming:

"WMF-Backports" and "MediaWiki-Backports" doesn't feel explanatory enough to me. Sounds too similar to other subprojects which have a defined codebase area. "WMF-Server-Backports" and "MediaWiki-Tarball-Backports" will work for me. Let's go for that?

I'll probably make a short version for WMF-Server-Backports (aka "WMF-Backport"), but sure :) (just like I made "Release-Engineering-Team" for "Release-Engineering-Team")

Bugzilla Backport Flags migration aspect only:

Tickets with any of our two Bugzilla flags (Backport_WMF and Backport_to_Stable) set will not have the flag information migrated to Phabricator. I don't see any historical need to know whether something was backported at some undefined point in the past (and that information is available in comments anyway if somebody is curious).

Regarding currently open tickets with Backport requests that have not been decided on, there are less than 10 of them hence the effort is not worth it to write migration code covering that.
I have cleaned up WMF backport requests yesterday and I've sent an email today to the Tarball Release Team to please take a look at the queue of undecided tarball backport requests.
Furthermore Bugzilla will be available as read-only for some time after the migration so we could still look up which Bugzilla tickets had this flag if we wanted.

Probably fine. We can get backport information from git but it'll obviously not have the bug info (if there was a bug).

In T101#11104, @Aklapper wrote:

Project names: "WMF-Server-Backports" and "MediaWiki-Tarball-Backports"

as usual confused whether the action to create and set up these two projects should now be separate tasks...

@greg, @MarkAHershberger, @Mglaser: I have created WMF-Server-Backports and MediaWiki-Tarball-Backports and added you accordingly. Was unsure which type/kind of project these are so I sticked with the default (briefcase/blue) for the time being.

@MarkAHershberger, @Mglaser: Task 76168 got backported to 1.24. Not sure if you want the MediaWiki-Tarball-Backports project to be added to that task. https://www.mediawiki.org/wiki/Backporting_fixes#Backports_to_stable.2Fsupported_release does not cover this "retroactively" - whatever works for you and isn't overhead.