Currently, enwiki provides the full history dumps in many small files since 2011/2012. It was proposed on Xmldatadumps-l that this feature be expanded to dewiki, but it didn't happen (presumably due to my initial opposition as it was incompatible with the archiving scripts back then).
I propose that we implement this feature for both dewiki and frwiki as they are the next 2 largest wikis. While frwiki's files are not as big as dewiki, they both pose problems during the rsync to Labs, and for the last few dumps dewiki has never had a successful complete copy to Labs without manual intervention. Splitting the dumps will allow the whole dump to be successfully copied over to Labs without much issues, as evident in the enwiki dumps.
This will need discussion from users of these dumps.
The impact
- File sizes will become smaller for certain dump types
- The pageids will be available in the file name, making it easier to obtain just what you need
- Increases the overall reliability of the dump production process