Change Details

Something is wrong with the file format, or with our parsing: ``` mix run pipeline.exs ltwikisource ** (ErlangError) Erlang error: {16, :invalid_json} ```

The checkpointing is incompatible with the gzip format, something is not finalized in the output--or maybe we're concatenating on top of a truncated file without ending a block safely.

Something is wrong with the file format, or with our parsing: ``` mix run pipeline.exs ltwikisource ** (ErlangError) Erlang error: {16The checkpointing is incompatible with the gzip format, :invalid_json} ```something is not finalized in the output--or maybe we're concatenating on top of a truncated file without ending a block safely.