@chasemp knows the technical details and I will leave to him the confirmation of the resoltion (Declined, in my view). The main point though is that this is technically very very complex, not worth the fight.

Since old Bugzilla URLs will point to new Phabricator tasks, you can always use that as a way to land in the Phabricator tasks that you had literally in mind.

This is also a temporary problem for an elite of users that do remember more than a handful of bug report numbers by heart. New numbers will come and some of them will be remembered.

In T857#14484, @Qgil wrote:

@chasemp knows the technical details and I will leave to him the confirmation of the resoltion (Declined, in my view). The main point though is that this is technically very very complex, not worth the fight.

Since old Bugzilla URLs will point to new Phabricator tasks, you can always use that as a way to land in the Phabricator tasks that you had literally in mind.

This is also a temporary problem for an elite of users that do remember more than a handful of bug report numbers by heart. New numbers will come and some of them will be remembered.

… but this was a fundamental part of the promise for the migration. Replacing thousands of {{bugzilla|foo}} templates with {{phabricator|foo}} is a bot task (or indeed just a redirect). Replacing them with {{phabricator|bar}} is a non-starter. It's hugely disappointing for this to have been reversed without any notice. :-(

+1 to everything James said :(

I don't recall making this promise. How could we, since the early days it has been clear that keeping numbers was really hard considering how the migration technically works.

What we discussed is this: {{bugzilla|foo}} will point to Bugzilla redirects that will send users automatically to the Phabricator tasks they are looking for.

I think if we'd had BZ ready to import for day 1 instead of waiting and importing labs content first was the problem. Otherwise we could've just started with T1: Get puppet runs into logstash at import.

In T857#14507, @Qgil wrote:

I don't recall making this promise. How could we, since the early days it has been clear that keeping numbers was really hard considering how the migration technically works.

What we discussed is this: {{bugzilla|foo}} will point to Bugzilla redirects that will send users automatically to the Phabricator tasks they are looking for.

For a few years. We're talking here about links that need to work in a decade.

In T857#14509, @Chad wrote:

I think if we'd had BZ ready to import for day 1 instead of waiting and importing labs content first was the problem. Otherwise we could've just started with T1: Get puppet runs into logstash at import.

Indeed, I was surprised at this instance being started prematurely but assumed that they were all going to get renumbered up to start from 100,000 as had been previously discussed.

Related: T40, T197, T625.

Proposing DECLINED one more time.

Proposing BLOCKER one more time.

I think this is a sufficiently-large issue that it should absolutely definitely be a blocker for migration. The deadline is just an arbitrary aspiration; fundamentally breaking existing links with a hack to redirect people. Discarding this requirement in this way is not cool.

In T857#14509, @Chad wrote:

I think if we'd had BZ ready to import for day 1 instead of waiting and importing labs content first was the problem. Otherwise we could've just started with T1: Get puppet runs into logstash at import.

As far as I know, this is not related about T1 not being available today, but about the script migrating Bugzilla reports one by one or in batches. I guess @chasemp could fine tune his script to import task 1 completely, then 2, then 3... And I guess we could assign them task numbers *1, *2, *3... here. However, this would lead to a much slower migration. People are already uncomfortable with our 1-3 days Bugzilla+Phabricator downtime. Would they willing to add 1-2 days just to keep the numbers?

In T857#14511, @Jdforrester-WMF wrote:

For a few years. We're talking here about links that need to work in a decade.

What is the problem of keeping redirects during a decade or more? It's a static piece of information that doesn't bother the servers.

In T857#14529, @Qgil wrote:

In T857#14509, @Chad wrote:

I think if we'd had BZ ready to import for day 1 instead of waiting and importing labs content first was the problem. Otherwise we could've just started with T1: Get puppet runs into logstash at import.

As far as I know, this is not related about T1 not being available today, but about the script migrating Bugzilla reports one by one or in batches. I guess @chasemp could fine tune his script to import task 1 completely, then 2, then 3... And I guess we could assign them task numbers *1, *2, *3... here. However, this would lead to a much slower migration. People are already uncomfortable with our 1-3 days time frame. Would they willing to add 1-2 days just to keep the numbers?

In general, the downtime is bad. But for this? Yes, I think so.

In T857#14511, @Jdforrester-WMF wrote:

For a few years. We're talking here about links that need to work in a decade.

What is the problem of keeping redirects during a decade or more? It's a static piece of information that doesn't bother the servers.

It's yet another thing to break. ;-)

fundamentally breaking existing links

What is "fundamentally breaking" here? You go to the Bugzilla URL for Bugzilla ticket 12345 and you end up at the corresponding Phab ticket (whatever number it'll have). That's not "breaking" to me.

In T857#14537, @Aklapper wrote:

fundamentally breaking existing links

What is "fundamentally breaking" here? You go to the Bugzilla URL for Bugzilla ticket 12345 and you end up at the corresponding Phab ticket (whatever number it'll have). That's not "breaking" to me.

Because a link to foo now goes to bar. Same content, different system, unexpected user outcome. The redirect is a hack to rescue people that have gone to the wrong place; it's not the main form through which people should come to Phabricator.

I think an extra 1-2 days of downtime to do this right is well worth it.

Re-numbering Phabricator tasks seems needlessly painful.

I kind of assumed that bug 1 would map to whichever task number was next available at time of import and that every subsequent imported Bugzilla bug would get the next available task number sequentially (e.g., bug 1 --> task 982, bug 10 --> task 991, etc.). A sequential import allows for a much easier translation between the two systems (in this example scenario, you'd simply add or subtract 981).

But even a non-sequential import is probably fine as long as the user is properly redirected. The issue here is that we cannot break links. Whether bug 1 maps to task 1 or task 982 doesn't really matter to me, personally.

From my point of view, the main concern is the potential for confusion between bug numbers and task numbers. Without further qualification, 982 could refer to either of two different issues -- using @MZMcBride's example, T1963 (formerly bug 982) or T982 (formerly bug 1).

I think Phabricator strongly encourages not to use numbers without prefixes as they may relate to different applications, so 982 will always refer to Bugzilla bug #982 and to refer to T982: Project creation request : Beta Features one would always write T982.

As long as the existing links continue to work for the foreseeable future, using different numbers is fine with me. We had a similar situation with the migration from Subversion to Git.

In T857#14538, @Jdforrester-WMF wrote:

In T857#14537, @Aklapper wrote:

... You go to the Bugzilla URL for Bugzilla ticket 12345 and you end up at the corresponding Phab ticket (whatever number it'll have). That's not "breaking" to me.

Because a link to foo now goes to bar. Same content, different system, unexpected user outcome. The redirect is a hack to rescue people that have gone to the wrong place; it's not the main form through which people should come to Phabricator.

Agreed, users will come to Phabricator through new T numbers. The transition will happen quickly if old-timers let it! If users are used to BZ they'll visit the old bugzilla.wikimedia.org URLs which will still work as a redirect. Perhaps your concern is if users see a mention of "bug 12345" and come to Phab with it and the wrong thing or nothing happens. The fix for that might be to make the Phabricator search handle 12345 specially, appending OR Ext_ref = bz12345 to the search.

T40: Set up redirects from old bugzilla.wikimedia.org URLs is indeed a blocker, and it needs to be proven to work before touching Bugzilla. When redirects work, legacy links work as well.

Note that these redirects don't rely on Bugzilla, and they will last for as long as needed with a maintenance cost of basically zero.

I don't think anybody has to dedicate time to unlearn and re-learn anything, just like I don't think anybody had to invest any time to learn those numbers consciously in the first place. Just use the old URLs as long as you need. I agree it be will inconvenient at the beginning, but (again, if the redirects are reliable) I don't see why this should be a relevant problem after a few weeks.

PS: I explained the problem and our resolution also in wikitech-l.

Seems to be there are 2 asks here:

1. Sequential ordering of tasks created from bugs (makes it trivial to translate from old numbers to new numbers)
2. Identical numbering of tasks created from bugs (makes it unnecessary to translate the number at all)

The response so far has been that translation is doable, but sequential numbering will make parallelization impossible or impractical thus making the transition slower, estimated to be around 1-2 days.

I feel like it's being underestimated how valuable it is to retain the numbers - it's not just inbound links, it's also in the content of MediaWiki:CodeReview, Gerrit commit messages and discussions, change logs, engineering reports and of course within the content of the bugs themselves. Sure, it could redirect, but the user reads "Bug 12345" and then is (hopefully) provided a link that redirects to task/54321, which is a total disconnect.

So if I see a code comment "// Fixes bug 1234" I have to search in two places?

I think the idea is people would use T857 or 'Task: 857' in the future, not 'Bug: 857'. The latter would thus always refer to Bugzilla numbers.

In T857#19224, @TrevorParscal wrote:

Seems to be there are 2 asks here:

1. Sequential ordering of tasks created from bugs (makes it trivial to translate from old numbers to new numbers)
2. Identical numbering of tasks created from bugs (makes it unnecessary to translate the number at all)

The response so far has been that translation is doable, but sequential numbering will make parallelization impossible or impractical thus making the transition slower, estimated to be around 1-2 days.

Indeed. And I for one thing an extra day or two of migration is well worth it for the sanity it'll provide.

As much as I would've preferred (2), that ship sadly sailed when we opened up Phabricator prior to the BZ migration being ready.

I feel like it's being underestimated how valuable it is to retain the numbers - it's not just inbound links, it's also in the content of MediaWiki:CodeReview, Gerrit commit messages and discussions, change logs, engineering reports and of course within the content of the bugs themselves. Sure, it could redirect, but the user reads "Bug 12345" and then is (hopefully) provided a link that redirects to task/54321, which is a total disconnect.

This. This. This.

It's not about the links--yes, redirects keep you going to the right place. It's about the complete mental disconnect between the numbers in the old and new system. Relying on an opaque mapping between old and new means that we're all relying on that redirect for eternity because you won't be able to just know what task a bug refers to.

There are two basic cases of references to bug numbers in web pages:

• Bug numbers with links to the Bugzilla page. Automatically redirected. Users click and land in the place they were looking for.
• Bug numbers without any link, just a number. If the user doesn't recognize that bug number then, no matter what, the user will need to copy the number and paste it (or type it manually) somewhere. Ideally, that somewhere would be the Phabricator search box in the header, resembling the Bugzilla search box (T991).

In T857#19232, @valhallasw wrote:

I think the idea is people would use T857 or 'Task: 857' in the future, not 'Bug: 857'. The latter would thus always refer to Bugzilla numbers.

Already now in Gerrit, "Bug: 123" points to Bugzilla while "Bug: T123" points to Phabricator.

In T857#19236, @Qgil wrote:

There are two basic cases of references to bug numbers in web pages:

• Bug numbers with links to the Bugzilla page. Automatically redirected. Users click and land in the place they were looking for.
• Bug numbers without any link, just a number. If the user doesn't recognize that bug number then, no matter what, the user will need to copy the number and paste it (or type it manually) somewhere. Ideally, that somewhere would be the Phabricator search box in the header, resembling the Bugzilla search box (T991).

The latter case makes no sense to me. In that scenario: how would I get from Bug 1 to Task 495? Other than copy+pasting into Bugzilla.wm.o and letting it redirect me.

There's also the tons and tons of bug mentions that aren't in web pages. Like commit logs, e-mails, etc etc etc. Requiring people to copy+paste bug numbers and wait for it to round-trip and redirect you just to figure out where the discussion is does not strike me as acceptable.

In T857#19232, @valhallasw wrote:

I think the idea is people would use T857 or 'Task: 857' in the future, not 'Bug: 857'. The latter would thus always refer to Bugzilla numbers.

Already now in Gerrit, "Bug: 123" points to Bugzilla while "Bug: T123" points to Phabricator.

Which makes no sense tbh to me. "Bug: Task 123" doesn't parse well when reading.

@valhallasw: The syntax is "Bug: T857" so there is no ambiguity.

I would love to see some data on how many bug numbers the average developer remembers, and how many developers easily master the MediaWiki system with more than a million lines of code in core alone, sometimes very scarcely documented, but get totally derailed when Bugzilla's bug #12345 becomes Phabricator's T54321. I find it very hard to believe that someone looks at a dump like https://bugzillapreview.wmflabs.org/T2, accepts all those "bzimport" bits and only finds the number unappealing.

In T857#19224, @TrevorParscal wrote:

Sure, it could redirect, but the user reads "Bug 12345" and then is (hopefully) provided a link that redirects to task/54321, which is a total disconnect.

I think it is fair to say that for most users in most cases, the connection is not with the bug number but with the problem they care about. As long as they find the report, they won't be bothered about its number.

The numbers are relevant for a core group of heavy contributors. While the new numbers will indeed disturb their routines, I am convinced that after a few weeks the new routines with the new numbers will have taken over, and only from time to time they will be annoyed by some bug number that won't be linked and they won't recognize.

What is the alternative, anyway?

Let's look strictly to the import of Bugzilla data. Currently the script imports batches of 15-20 bugs at a time, and based on the previews and other tests, we estimate that it will require about 13 hours to import the >70,000 tickets (which is just one part of the migration, we are booking 3-4 days for a reason). This means that in order to import bugs one by one, we would need about 8 days + completing the rest of tasks. Chances are high that errors will happen during those days, adding more time and more risk of breaking the identical numbers.

In addition to that, if we really want to keep identical numbers we would need to do something with the current >1000 tasks in Phabricator. Removing and place them at the end? Easier said than done, more uncharted territory that would imply more hours of migration and more chances of something going wrong.

Being conservative, we can say that promising to keep identical numbers would imply an extra week of migration, if we are lucky, and it would be quite a wild promise because the chances of not being ultimately able to deliver those identical numbers within a manageable time frame would be quite high. Finally, how to explain more than one week without bug reporting tool to the majority of users that don't recall and don't care about bug numbers?

Honestly, I don't think we want to go that route. This is why the Wikimedia Phabricator team prefers to be frank, go ahead with an already pretty complex migration, and focus instead on infallible redirects easy to use and cheap to maintain for as long as Phabricator exists.

fwiw, this also affects links in SAL (server admin log) on wikitech. perhaps a bot could fix the links ?

Given the reaction that engineers are having to this change, I know that this is not what people signed up for.

Maybe nobody promised the numbers would stay the same, but this side-effect appears to not have been sufficiently advertised.

Dismissing this as only something that helps people who have memorized bug numbers, or only the 0.01% of developers who are advanced enough to notice is not helpful. You've been given concrete examples of locations where these numbers are in use, in many cases without hyperlinks.

This is going to make a mess of a system that has worked just fine for a decade. You are defending this short-cut by suggesting that it's too costly to do it properly. I suggest you are simply transferring the cost away from the migration, and distributing it to developers of tracked projects and users of the new system for years to come.

About the use case of the bug number without a link, let me expand on what I tried to say above.

In T857#19236, @Qgil wrote:

• Bug numbers without any link, just a number. If the user doesn't recognize that bug number then, no matter what, the user will need to copy the number and paste it (or type it manually) somewhere. Ideally, that somewhere would be the Phabricator search box in the header, resembling the Bugzilla search box (T991).

Let's compare scenarios. A user finds a bug number without a link, and then...

• Bugzilla: goes to Bugzilla and places the number in the search bar.
• Phabricator with identical numbers: goes to Phabricator and places the number in the search bar.
• Phabricator without identical numbers, today: goes to Maniphest advanced search and introduces reference.
• Phabricator without identical numbers, if T991 can be implemented: goes to Phabricator and places the number in the search bar.

Considering the cost and risk of keeping identical numbers (explained above), I would rather invest the resources in T991, which looks like a feasible, reliable, and satisfactory solution.

In T857#19304, @Qgil wrote:

About the use case of the bug number without a link, let me expand on what I tried to say above.

In T857#19236, @Qgil wrote:

• Bug numbers without any link, just a number. If the user doesn't recognize that bug number then, no matter what, the user will need to copy the number and paste it (or type it manually) somewhere. Ideally, that somewhere would be the Phabricator search box in the header, resembling the Bugzilla search box (T991).

Let's compare scenarios. A user finds a bug number without a link, and then...

• Bugzilla: goes to Bugzilla and places the number in the search bar.
• Phabricator with identical numbers: goes to Phabricator and places the number in the search bar.
• Phabricator without identical numbers, today: goes to Maniphest advanced search and introduces reference.
• Phabricator without identical numbers, if T991 can be implemented: goes to Phabricator and places the number in the search bar.

• Command line, or any other place that isn't a browser already on Phabricator or Bugzilla - Copy/paste, open your browser to BZ or Phabricator, search for old bug, eventually get to new task.

Considering the cost and risk of keeping identical numbers (explained above), I would rather invest the resources in T991, which looks like a feasible, reliable, and satisfactory solution.

Which is why I said the ship for identical bug numbers has already sailed. Can we please discuss having sequential numbers at the very least?

None of this precludes T991: Searchable "Reference" custom field, which is a good idea in its own right.

In T857#19315, @Chad wrote:

Can we please discuss having sequential numbers at the very least?

The theoretical options would be (@chasemp, feel free to add/correct):

• Bug 1 == T100001, Bug 7200 == T107200. We should find out what happens to the 98k task numbers between the current most recent Phabricator task and T100000. We could still expect some users annoyed because keeping the T1+5digit format requires a moment of concentration and is prone to error (they only need to type a zero more or less).
• Bug 1 == B1, Bug 72000 == B72000. We should find out what it takes to create a new range in Phabricator and to integrate it with the search in order to get the same behavior users get now when searching for T****. This approach is easier to type, but then again it gets pretty close to the cheaper and cleaner T991, where users could type "B72000" or simply "72000" and find their task.

Users would still need to know the trick, although if the target is indeed a smaller group of dedicated users this might be less of a problem.

We would still need to deal with the hypothetical 8 extra days of migration, and I don't see any way forward here other than making a test with a sample of some 1000s of recent bugs to check how much time it takes and the chances of getting errors.

Are there other solutions? The costs and benefits of the two approaches above are not very appealing.

In T857#19394, @Qgil wrote:

• Bug 1 == T100001, Bug 7200 == T107200. We should find out what happens to the 98k task numbers between the current most recent Phabricator task and T100000. We could still expect some users annoyed because keeping the T1+5digit format requires a moment of concentration and is prone to error (they only need to type a zero more or less).
• Bug 1 == B1, Bug 72000 == B72000. We should find out what it takes to create a new range in Phabricator and to integrate it with the search in order to get the same behavior users get now when searching for T****. This approach is easier to type, but then again it gets pretty close to the cheaper and cleaner T991, where users could type "B72000" or simply "72000" and find their task.

Option (2) is not a good option -- then we have T123 and B123 next to eachother, which is even worse in terms of confusion.

As for option (1) -- I think this should be technically possible without Bugzilla downtime (but with a longer phabricator downtime!)

Depending on the code (we discussed this shortly on IRC but I feel we did not really come to a conclusion), it might be possible to do a 'mirroring' kind of set-up: A script which mirrors {state at time=0} into phab, then applies all comments/changes between time=0 and time=1, and repeats that.

If this is possible without throwing too much existing engineering out of the window, this gives us some advantages:

1. Bug numbers can be sequential, as there's no issue with the migration taking several days -- each update will take less and less time.
2. The final downtime can be much shorter: close off bugzilla, apply the last changes, open phabricator

And, if this is first done into a testing phabricator instance:

3. All bugs are converted instead of just the test bunch in the current preview instance. This means we can fix more problems that still exist in the conversion.

However, there are downsides:

1. Needs more engineering effort
2. Needs to lock off phabricator for a much longer period of time

I think downside (2) is worth it, but I'm hesitant to make an estimate for how much engineering effort is required to do this -- this could easily be more than it's worth to save a few days in migration.

Clearly it's hard to quantify how much time would be wasted over bug number confusion, but the fact that it could be non zero and the fact it would be forever means we should take this one opportunity to get it right. Compared to that, phab downtime and promised release schedules are insignificant and will quickly be forgotten.

In T857#19413, @Esanders wrote:

phab downtime and promised release schedules are insignificant

What about Bugzilla in read-only mode during a week or more?

Just for reference, about 419 Bugzilla reports were created in the past seven days, and comments were posted in about 923 bugs. If we are evaluating confusion, we should take this unprecedented break into account as well.

@valhallasw's idea is interesting in theory, but I wonder how much dedication a reliable implementation would require.

You know well that resources are not unlimited, neither free. Other users are waiting for other features, and other teams and projects are waiting for us. This is a project management situation familiar to most of you.

I've integrated a quotation from Trevor in the task description.

Note that these redirects don't rely on Bugzilla, and they will last for as long as needed with a maintenance cost of basically zero.

Thanks.

https://phabricator.wikimedia.org/T40#22103 said:

It's going to be reading from a cross reference table in the phabricator-maniphest database, using the user account that @chasemp already procured for this purpose.

Doesn't really sound like "basically zero" but might be negligible.

So I really still don't understand what the cost of fixing this bug amounts to. Is it essentially the cost of longer downtime? Can this be (dis)confirmed then:

As for option (1) -- I think this should be technically possible without Bugzilla downtime (but with a longer phabricator downtime!)

Also:

Which is why I said the ship for identical bug numbers has already sailed.

I don't understand why. The current tasks in the phabricator.wikimedia.org can just be thrown away and get new numbers sequentially (or even randomly) from whatever other integer. They've already been renumbered when importing the tasks from the wmflabs.org instance, so it seems cleary nobody cares.

Quim's email to wikitech-l said:

"We cannot assign to Phabricator tasks the same number as their Bugzilla
equivalents. Instead, automatic redirects will link old Bugzilla URLs with
their corresponding new Phabricator tasks. phabricator.wikimedia.org has
already >1300 tasks with numbers taken. The migration needs to be done by
batches of bugs instead of sequentially, which makes the mapping of numbers
more complicated. Still, smaller numbers will correspond to older bugs, and
we will do our best during the weekend to improve the sorting."

Seems like it isn't going to happen, then.

This task is open because we are still pursuing the goal of having sequential numbers for Bugzilla migrated tasks.

Keeping identical numbers would imply a bigger hack, moving the current >1300 Phabricator tasks somewhere else, breaking all the existing internal and external links. As opposed to fab.wmflabs.org, phabricator.wikimedia.org is a production instance, and we are committed to the integrity of its data.

Big kudos for implementing the N+2000 renumbering!

I still hope that at some point the Turquoise Fairy will make that a +0 with some more magic deck shuffling down the road.

In T857#780519, @Nemo_bis wrote:

Big kudos for implementing the N+2000 renumbering!

I still hope that at some point the Turquoise Fairy will make that a +0 with some more magic deck shuffling down the road.

2000+ is good enough for me. No need to do anything else :)

+2000 is not ideal but at least it is easy to remember and the bug -> task is not a complete random renumbering. Thank you @chasemp and @Qgil !

In T857#780635, @Chad wrote:

2000+ is good enough for me. No need to do anything else :)

I showed this to a friend of mine within the past week. It's definitely a neat trick. :-)