This ticket was created as a result of investingation carried out in T403793: 20250901 enwiki dump is triple the normal size
It has been observed that the RecombineDump tasks within the SQL/XML Dumps framework can create corrupted files.
This could occur on any of the recombine jobs that use dd to concatenate files.
I believe that this list is the following, although it is still to be verified.
| Task name | Example file names |
| articlesdumprecombine | metahistorybz2dumprecombine |
| articlesmultistreamdumprecombine | enwiki-20250901-pages-articles-multistream.xml.bz2 |
| enwiki-20250901-pages-articles-multistream-index.txt.bz2 | |
| xmlpagelogsdumprecombine | enwiki-20250901-pages-logging.xml.gz |
Essentially, it is anywhere that the dd_recombine function might be called.
We have so far only become aware of it happening once, in the articlesmultistreamdumprecombine job of enwiki, during the 20250901 dump run, although there may have been other occurrences.
The trigger for the bug is when a recombine task is killed whilst in process. The likely cause of the first known incident was an out-of-memory error, but it might conceivably be triggered by planned maintenance or another type of error.
If the task is killed, any temporary file that has been generated by a succession of dd operations is not removed.
When Airflow reschedules the task, the command used is along the lines of:
/bin/dd if=/mnt/dumpsdata/xmldatadumps/public/enwiki/20250901/enwiki-20250901-pages-articles-multistream1.xml-p1p41242.bz2 of=/mnt/dumpsdata/xmldatadumps/public/enwiki/20250901/enwiki-20250901-pages-articles-multistream.xml.bz2.inprog skip=0 count=575 iflag=skip_bytes,count_bytes oflag=append conv=notrunc bs=256k
Takes the first file to be concatenated and it opens the output file, but it does not check whether or not that file already exists.
The oflag=append and conv=notrunc flags ensure that it does not truncate an existing file and merely appends to it.