Nemo_bis wrote on 2014-06-15 11:25:43 (UTC)

Is import of "non-real projects" a separate task?

aklapper wrote on 2014-06-16 10:42:41 (UTC)

In T336#8, @Nemo_bis wrote:

Is import of "non-real projects" a separate task?

Which ones are those?

Nemo_bis wrote on 2014-06-16 12:26:04 (UTC)

Which ones are those?

No idea, the complement of "real projects" mentioned in the summary. :) Should the summary just be "Import all projects"?

aklapper wrote on 2014-06-16 14:02:46 (UTC)

People can create any projects and create fake tickets here, just to play with this Phabricator test instance. I guess that's what the "non-real projects" are which are not worth to import.

qgil wrote on 2014-08-15 11:37:40 (UTC)

This task is blocking T294: Super epic: Refactor moveToStep into a smaller function, make controllers for each step (Step 2 in our migration plan), but it is currently marked as Low priority. Could you consider a higher priority, please (even if at the expense of other tasks not directly related to T294).

qgil wrote on 2014-08-15 17:11:00 (UTC)

What will we do after the migration of useful content to the production instance? Delete the migrated content and leave the rest? Reset everything?

This labs instance would be still useful for

• all testing related to #code_review_in_phabricator
• try modules not available in our production instance
• sandbox for learning to use Phabricator?

aklapper wrote on 2014-08-15 18:47:10 (UTC)

AFAIK the setup of this Labs instance (fab.wmflabs.org) is busted enough to replace it by a properly set up one using Puppet. Someone please correct me if I'm wrong.

aklapper wrote on 2014-08-20 18:45:12 (UTC)

Having the logic for this done before launching day 1 is required. Currently not clear how complicated it's going to be.

TODO: @Aklapper to identify and provide a list of non-test projects in fab.wmflabs.org by going through http://fab.wmflabs.org/project/query/all/

aklapper wrote on 2014-08-25 13:03:27 (UTC)

I edited the initial description to list non-test projects. Not sure about some of them, maybe @Qgil knows before I hunt down people? :-/

Note to my successors: Next time we should screw up less by forcing people to use "Test" in project names or so. In hindsight, randomly mixed open testing with "be our guinea pig and put real data here" without any rules wasn't a good idea. Plus cannot archive some obvious test projects as I'm not a member.

qgil wrote on 2014-08-26 07:08:28 (UTC)

The list in the description looks good to me. These projects should be archived here. All tasks related with any of these projects should be migrated to the production instance and deleted here (or closed as Invalid, your call).

The rest will remain here as it is now. This means that if we forgot a task or two people will be able to create their clones in production manually. No love lost.

Rush wrote on 2014-08-27 16:51:01 (UTC)

without the ability to make this instance "read only" I was going to shut this down on migration, to avoid ppl coming here and making new stuff without realizing?

we could back up the data so it's not lost but leaving this up after it's officially dead seems confusing.

aklapper wrote on 2014-08-27 22:07:11 (UTC)

In T336#35, @Rush wrote:

without the ability to make this instance "read only" I was going to shut this down on migration, to avoid ppl coming here and making new stuff without realizing?

Sounds reasonible.

And for the records and transparency, @Rush said this task is a bit more complicated because Phabricator does not easily support exporting comments via its API.

dzahn wrote on 2014-08-28 01:16:51 (UTC)

we should still have _a_ labs instance because it is useful for the things Quim listed above.

actually, we should always have something to test production changes on, but _this_ labs instance is not useful for that,
because it is setup all manually and has no real relation to the prod instance.

instead, we should have new labs instance, without content, which is installed by puppet entirely, by applying the puppet roles used for production on a labs instance. people should never actually put content in it that should be kept though

qgil wrote on 2014-08-28 06:43:34 (UTC)

One way to make this instance read-only-ish is to make it editable only by admins, which is an easy step. Then again, the idea of getting rid of this instance in exchange of e.g. phabricator-test.labswmf.org puppetized and etc is spot on.

About the difficulties on content migration, one option is to replicate the database as is in the production instance and then delete/unpublish the testing content manually. If there is no other choice, I volunteer to do this in one of those evenings.

Possible steps:

1. Freeze fab.wmflabs.org -- only admins can edit and they won't edit.
2. Open phabricator.wikimedia.org editable only to admins with the fab.w.o content (with or without testing content, to be seen)
3. Clean and fine-tune phabricator.w.o (removing test content if needed, setting the right permissions.
4. Release phabricator.wikimedia.org. (Whether this is Day 0 (no Bugzilla content yet) or Day 1 (Bugzilla content imported) is not critical for this task here)
5. Setup a puppetized phabricator-test.wmflabs.org (this can be done at the beginning if you prefer)
6. Leave fab.wmflabs.org frozen for a month just in case people still need something froim there. Then shut it down & redirect to phabricator.w.o
7. Keep frozen fab.w.o. during a month

Rush wrote on 2014-08-28 14:31:25 (UTC)

can we remove this from every project that exists and use the list in the description? duplicating the list just makes things more confusing, and this shouldn't really show up under rt-mgiration for example as it does now.

aklapper wrote on 2014-08-28 14:32:59 (UTC)

Yeah. I admit it also confused me a bit.

aklapper wrote on 2014-08-28 14:42:47 (UTC)

[OT] @Qgil: Currently neither @Rush nor I can remove projects from this task and you're not on IRC. Projects simply get readded after editing. And @Rush is a member of acl*Batch-Editors. If you have an idea, reverting is welcome.

Rush wrote on 2014-08-28 14:46:17 (UTC)

In T336#43, @Qgil wrote:

One way to make this instance read-only-ish is to make it editable only by admins, which is an easy step. Then again, the idea of getting rid of this instance in exchange of e.g. phabricator-test.labswmf.org puppetized and etc is spot on.

About the difficulties on content migration, one option is to replicate the database as is in the production instance and then delete/unpublish the testing content manually. If there is no other choice, I volunteer to do this in one of those evenings.

Possible steps:

1. Freeze fab.wmflabs.org -- only admins can edit and they won't edit.
2. Open phabricator.wikimedia.org editable only to admins with the fab.w.o content (with or without testing content, to be seen)
3. Clean and fine-tune phabricator.w.o (removing test content if needed, setting the right permissions.
4. Release phabricator.wikimedia.org. (Whether this is Day 0 (no Bugzilla content yet) or Day 1 (Bugzilla content imported) is not critical for this task here)
5. Setup a puppetized phabricator-test.wmflabs.org (this can be done at the beginning if you prefer)
6. Leave fab.wmflabs.org frozen for a month just in case people still need something froim there. Then shut it down & redirect to phabricator.w.o
7. Keep frozen fab.w.o. during a month

regarding the idea of moving over this database from labs into production and then cleaning up the surface mess of maniphests and projects. It isn't a practical or feasible approach for many reasons. There is a lot more mess here than just those two applications, and flushing undesirable data from other other databases would be really fraught with errors. We also would be carrying over all users and their local login stuff which we flat out don't want and have gone to great lengths to avoid having in prod. I'm trying not to sound bullish on this since I think it's cool of you to make the offer on dealing with it, but aside from the huge security risk of dropping this db from labs into production and then hoping we can flush out the below the surface mess in a way that doesn't cause long term problems it is going to be far more work to tinker under the hood of phabricator and make things come out the other side in a consistent state than the mess that is exporting comments the hard way.

As far as this particular instance and its fate, I’m pretty ambivalent on the whole thing. If we are saying this is the data that matters then what is the point? But if someone finds it useful to take this over after cutover I have no qualms about it, but I am going to be making it unavailable for a period during data export and then after that anything that happens here is persona non grata in my mind.

qgil wrote on 2014-08-28 16:50:06 (UTC)

In T336#46, @Aklapper wrote:

[OT] @Qgil: Currently neither @Rush nor I can remove projects from this task and you're not on IRC. Projects simply get readded after editing. And @Rush is a member of acl*Batch-Editors. If you have an idea, reverting is welcome.

Done. As long as the description contains project tags listed (#etc) they are included in CC. This has nothing to do with the restriction of permissions.

Rush wrote on 2014-08-28 18:02:29 (UTC)

In T336#48, @Qgil wrote:

In T336#46, @Aklapper wrote:

[OT] @Qgil: Currently neither @Rush nor I can remove projects from this task and you're not on IRC. Projects simply get readded after editing. And @Rush is a member of acl*Batch-Editors. If you have an idea, reverting is welcome.

Done. As long as the description contains project tags listed (#etc) they are included in CC. This has nothing to do with the restriction of permissions.

that makes sense now that you explained but I honestly didn't know that! nice

Rush wrote on 2014-09-02 17:40:50 (UTC)

it may be a problem to get attachments to tickets from this instance, I don't see many ... so maybe it's not a big deal.

qgil wrote on 2014-09-03 15:45:59 (UTC)

I think ignoring attachments, files and mocks in this instance is totally fine.

qgil wrote on 2014-09-05 16:40:19 (UTC)

Please add Community Engagement. This is the last project I will create here in Labs before the migration (I had discussed about it with RachelDC back in July).

Rush wrote on 2014-09-05 16:51:56 (UTC)

done

Rush wrote on 2014-09-06 18:00:19 (UTC)

This instance will go read only early this week. I'm going to send an email about it soon.