Using a library like stream_gzip, compress all outputs. Decompress when reading.
In the past, I seem to remember that this library uses slightly odd typing for its input or output, so we might need to include a small wrapper to adapt to Stream and Flow pipelines.
Smaller, final output files with a single object such as "-summary.json" shouldn't be compressed.
Code to review:
https://gitlab.com/wmde/technical-wishes/scrape-wiki-html-dump/-/merge_requests/49