Please provide the following information:

1. md5 checksum of the file (md5sum 2006*.bz2)
2. your version of bzip2 (bzip2 --version)
3. your operating system and version
4. your cpu architecture

zhangxiaoquan wrote:

1. md5 checksum of the file (md5sum 2006*.bz2)

d521c59d94852920648565a80f5e1b90
it is different from the one on the website, but I had no problem while
downloading this file, and I tried it several times.

2. your version of bzip2 (bzip2 --version)

1.0.3  15-Feb-2005

3. your operating system and version

RedHat Enterprise Server 3

4. your cpu architecture

Intel Pentium III (Coppermine) 1GHz

Looks like your download is corrupt then.

Check in particular that your download program handles
large files (this is over 33 gigabytes; if your download
is less than 4 gigabytes then you have a buggy download
tool or a buggy HTTP proxy).

Note that the 7zip version of this file is smaller and
faster to download, but still over 5 gigabytes so you need
to confirm that your download was correct.

zhangxiaoquan wrote:

Thanks for the tip. I used wget 1.10.2, the file size is correct. I never had
a problem with wget, but I admit 33GB might be too much for any download program.

I wonder if you can create a special version for us with the content in <text>
tags removed, for our purpose of research, we only need the modification history
(who modified what at what time, etc.) Thanks, please let us know.

The reason we want to go for bzip2 (instead of 7zip) is that I wrote a perl
parser to read directly from bzip2 files and write a new file without
information in the <text> tags. I'm not sure if it is feasible with 7zip format.

Tried reading from a pipe on stdin? (7zip also decompresses
about 10 times faster than bzip2.)

zhangxiaoquan wrote:

I'll give it a try, can you leave this open till I download and verify the
md5checksum?
Thanks a lot for the help!

Really old, so gonna go ahead and close.