Page MenuHomePhabricator

Bugzilla to Maniphest import script (tracking)
Closed, ResolvedPublic

Description

This is a tracking bug for the Bugzilla -> Maniphest (Phabricator issue tracker) import script so we can keep track of subtasks that are specifically about the automated import.

Elements of a Bugzilla bug report: see https://www.mediawiki.org/wiki/Phabricator/versus_Bugzilla#Bugzilla_data_migrated

Details

Reference
fl423

Related Objects

StatusSubtypeAssignedTask
ResolvedQgil
ResolvedQgil
ResolvedQgil
Resolved chasemp
ResolvedAklapper
Declined chasemp
Resolved chasemp
Resolved mmodell
Resolved chasemp
DeclinedAklapper
Resolved chasemp
Resolved mmodell
ResolvedAklapper
ResolvedAklapper
Declined mmodell
Resolved chasemp
DeclinedQgil
Resolved chasemp
InvalidQgil
Resolved chasemp
Resolved chasemp
Resolved chasemp
Resolved chasemp
DuplicateNone
Resolved chasemp
Resolved chasemp
Resolved chasemp

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

mattflaschen wrote on 2014-07-26 02:41:44 (UTC)

In T423#20, @Aklapper wrote:

There might be a chicken and egg problem on importing User accounts and Tickets, and what to create first.

Can someone explain this? If the script takes the approach of importing users (rather than just faking it by posting everything by import-script), I can't think of any problems that would be caused by importing users first.

Importing Bugzilla tickets then Bugzilla users would not work.

henzen wrote on 2014-07-30 06:21:19 (UTC)

Greets,

A common feature with bugz deployments is the use of top level classifications... Commonly used to separate/group bugs into companies or departments, etc, allowing you to then drill down to products, components, etc.

We use it extensively across our group of companies.

I don't see Classification being addressed as part of the import process in this task. I can see ourselves and very likely others embracing your import process to migrate from bugz to phab, and classification is a vital aspect which should not be ignored.

Regards,
Henry

aklapper wrote on 2014-07-30 10:42:20 (UTC)

In T423#45, @henzen wrote:

I don't see Classification being addressed as part of the import process in this task.

We do not have classifications enabled in Wikimedia Bugzilla so it does not affect or interest us. Anybody will be free to pick up the migration code and enhance it to suit their needs and Bugzilla configurations, but this is out of scope for Wikimedia.

noisy wrote on 2014-08-18 21:16:09 (UTC)

Is there a plan to make this migration script public? It would be very useful for other people who want to migrate from bugzilla to phabricator.

Rush wrote on 2014-08-18 21:41:35 (UTC)

@noisy

keep an eye on https://gerrit.wikimedia.org/r/#/admin/projects/phabricator/tools

eventually it will look like real tools and not just tinkering :)

but yes everything will be public

mattflaschen wrote on 2014-08-19 05:59:46 (UTC)

In T423#54, @Rush wrote:

Direct link: https://git.wikimedia.org/tree/phabricator%2Ftools.git/

but yes everything will be public

Could you also add a license to that repo?

aklapper wrote on 2014-08-22 20:58:40 (UTC)

For the records, we ran into the problem of invalid byte sequences when exporting certain tickets from Bugzilla via its XML RPC API, resulting in script crashing and token invalidation. This is tracked in https://bugzilla.wikimedia.org/show_bug.cgi?id=69747

aklapper wrote on 2014-08-31 19:31:52 (UTC)

In T423#58, @Aklapper wrote:

For the records, we ran into the problem of invalid byte sequences

And for the records of the records, I worked around that problem.

aklapper wrote on 2014-09-07 20:58:19 (UTC)

  • Phab API does not provide a way to set dependencies between tasks. @mmodell might hack together an API class so Chase's script could access edges from the API. Should this be upstreamed?
  • Realized today that script does not tackle yet that accounts in Bugzilla can be 1) disabled (no login possible and no bugmail delivered) and 2) have bugmail delivery disabled only. Related Bugzilla DB SQL queries to identify them:
    • 1) SELECT login_name FROM profiles WHERE disabledtext!='';
    • 2) SELECT login_name FROM profiles WHERE disable_mail=1;

aklapper wrote on 2014-09-08 13:22:55 (UTC)

In T423#62, @Aklapper wrote:

Related Bugzilla DB SQL queries

Requested output in https://rt.wikimedia.org/Ticket/Display.html?id=8304

As discussed in previous meetings, we need to document what will happen to these values in Bugzilla reports

  • Reported: YYYY-MM-DD HH:MM UTC by Author
  • Modified: YYYY-MM-DD HH:MM UTC by Author

The dates of the comments could be kept in the fab migration, and we need to confirm that we will keep them in the Bugzilla migration as well.

The description of this task has been copied and formatted as table at https://www.mediawiki.org/wiki/Phabricator/versus_Bugzilla#.C2.A0Bugzilla_data_migrated

Maybe you want to delete the description here and point to the wiki page instead, just like we did with the timeline?

mattflaschen wrote on 2014-07-26 02:41:44 (UTC)

@Aklapper wrote:

There might be a chicken and egg problem on importing User accounts and Tickets, and what to create first.

Can someone explain this?

The migration of user accounts is covered in T419. Please see there (and ask there if something is unclear). Thank you!

In T259#3323, @flimport wrote:

aklapper wrote on 2014-09-07 20:58:19 (UTC)

  • Phab API does not provide a way to set dependencies between tasks. @mmodell might hack together an API class so Chase's script could access edges from the API. Should this be upstreamed?

Yes, I think so. Setting dependencies seems like general-purpose functionality that should be in the Phabricator API. Is there an upstream bug for this?

yes, it's not a matter of they don't want to do it. It's a matter of internal architecture that is in flux

https://secure.phabricator.com/T5873

Let's mark "blocker freeze" for the Bugzilla migration script. I have just added the tasks that were reported in the context of Bugzilla-Preview. If there are any new blockers proposed, they will be evaluated in our Phabricator team meetings.

Talked to Chase: There are some problems with some attachments marked as "obsolete" when trying to import them (attachments marked as "private" are excluded anyway). Hence we will not import attachments marked as "obsolete" in Bugzilla (Bugzilla hides them by default in the "Attachment" table of a report, providing a "Show Obsolete" link).

Note to myself: https://www.mediawiki.org/wiki/Phabricator/versus_Bugzilla needs updating for this; currently says "We will migrate all the public Bugzilla reports, including their attachments." plus the "Attachments" item in the "Bugzilla data migrated" table plus the "History of changes" item under "Missing data".

well, we're here so closing.

@chasemp , you're so humble. At this point, I feel this import script of yours is almost a member of the team. :) Thank you very much for your work on it. I really have no idea how we would have got here without this script, and of course without you.