Page MenuHomePhabricator

import old staff list archives ?
Open, Stalled, LowPublic

Description

the old staff list (https://lists.wikimedia.org/mailman/listinfo/staff) is disabled. if you login as admin you see that "emergency moderation" is enabled

the list archives have like one random looking mail thread from 2007 but are otherwise empty

Tim Starling provided us with .mbox files of this from 2007 thru 2011.

Should we try to import them again from that?

Event Timeline

Dzahn created this task.Aug 18 2015, 12:16 AM
Dzahn claimed this task.
Dzahn raised the priority of this task from to High.
Dzahn updated the task description. (Show Details)
Dzahn added subscribers: ori, MZMcBride, Dzahn and 5 others.

was it on purpose or by accident that archives are deleted?

let's ask Philippe

Asked James on IRC, he's going to follow a response up. +cc Philippe and James.

Dzahn added a comment.Aug 18 2015, 2:05 AM

note to self: sed -e '/^[ TABKEY]/H; x; /^Received:/!p; $!d; x; p'

Dzahn added a comment.Aug 18 2015, 4:40 PM

I tested that importing works on fermium (that doesn't make it public, just to check if it technically works)

I used cat to combine multiple mbox files (the existing one and the ones provided by Tim, one per year) into a single mbox file.

Then ran this script by Tim to remove all the Received headers:

#!/usr/bin/perl -w

my $inReceived = 0;
while (my $line = <STDIN>) {
	if ($line =~ /^Received:/) {
		$inReceived = 1;
	} elsif ($inReceived) {
		if ($line !~ /^\t/) {
			$inReceived = 0;
			print $line;
		}
	} else {
		print $line;
	}
}

Then imported the staff list config and archives with my script from T108073#1542059 which creates directories in the ./archives/ directory and copied the .mbox file in place in /var/lib/mailman/archive/private/staff.mbox/staff.mbox
and finally ran ./arch --wipe staff from the mailman bin directory to recreate HTML from the mbox file and wipe old files.

Speaking for Philippe and myself I'd go with a no for now .

Not because it isn't a nice to have (it is) but because it's just incredibly low priority. These are old archives of an old/archived list that was private at the time and will remain private (too much risk of confidential information). The archives themselves would be accessible to only a small number of people and while I trust that Tim's backup is unedited we don't actually have any way to verify that easily especially given how far back it's from. It would be nice to have them around for historical reasons but I think it's so low on the priority list at the moment and I know that normally we have quite a lot backed up.

Re: Was it purposely deleted, I'll look into this but I doubt it.

We already did most of the work, at this point it does not take much time to do it. Priority doesn't affect it much. It's more that we want to know whether it was an accident (then it's a bug) or not.

regarding who gets access, we could just give it to people subscribed on "wmfall".

It's true that it's not a blocker for the migration, but it would be convenient to just do it now because we have to import ALL lists from the old server anyways.

Dzahn lowered the priority of this task from High to Medium.Aug 22 2015, 12:00 AM
Dzahn set Security to None.
Dzahn added a subscriber: tstarling.
Dzahn changed the task status from Open to Stalled.Aug 31 2015, 10:49 PM
Dzahn added a comment.Sep 2 2015, 7:31 PM

which current list is the continuation of the old staff list? wmfall@? wmfreqs@?

brion added a comment.Sep 2 2015, 7:52 PM

I'd like to have these accessible; list archives provide records of decisions made, why we made them, etc. Not everybody is as good about keeping custom archives going back the entire length of our company as Tim personally is, especially since few employees have been around that long themselves. :)

It would be easier if we could just re-import them to the 'staff' list where they belonged in the first place but for some reason 'staff' was removed a couple years back and replaced with other lists, which I never quite understood...

which current list is the continuation of the old staff list? wmfall@? wmfreqs@?

officially neither... but wmfreqs is probably the closest (the old staff@ list did not have non req contractors (in general... except where it did.... complicated)

Dzahn added a comment.Oct 7 2015, 9:46 PM

Ideas for the best way to "un-stall" this?

Dzahn lowered the priority of this task from Medium to Low.Jan 4 2016, 7:18 PM

Ideas for the best way to "un-stall" this?

You could notify the authors of the messages included in the mbox file and see if someone complains.

Importing the messages under their original list sounds like the best option for a first step, as it allows the involved persons to find and exchange links to discussions they had back then, while not automatically extending access to all wmfall members.

P.s.: The list owner should be updated.

Dzahn added a comment.Feb 11 2016, 7:42 PM

You could notify the authors of the messages included in the mbox file and see if someone complains.

All of them? :o And since this is an ancient staff list, what i expect is a bunch of @wikimedia.org email addresses that don't exist anymore, so you would first have to track down how to even reach them nowadays. ? hmmm

All of them? :o And since this is an ancient staff list, what i expect is a bunch of wikimedia.org email addresses that don't exist anymore, so you would first have to track down how to even reach them nowadays. ? hmmm

Well, if we deleted their email address it probably means we no longer care about their opinions :P, so to say.

Now if we had a current list of staff we might be able to compare how many of them ever posted back on this list, but i'm afraid we don't have one.

Now if we had a current list of staff we might be able to compare how many of them ever posted back on this list, but i'm afraid we don't have one.

Compare to wmfall?

Dzahn removed Dzahn as the assignee of this task.Jan 8 2019, 11:56 PM