Change Details

Before we destroy the instance, let's take the opportunity to check for errors: * Check that .json files can be parsed. * Check that each line of .ndjson can be parsed. * Check for duplicate entries in each page summary and mapdata output file, by building a set keyed on revid.

Before we destroy the instance, let's take the opportunity to check for errors: * Check that .json files can be parsed. * Check that each line of .ndjson can be parsed. * Check for duplicate entries in each page summary and mapdata output file, by building a set keyed on revid. Implementation: https://gitlab.com/wmde/technical-wishes/scrape-wiki-html-dump/-/tree/validate-outputs?ref_type=heads

Before we destroy the instance, let's take the opportunity to check for errors: * Check that .json files can be parsed. * Check that each line of .ndjson can be parsed. * Check for duplicate entries in each page summary and mapdata output file, by building a set keyed on revid. Implementation: https://gitlab.com/wmde/technical-wishes/scrape-wiki-html-dump/-/tree/validate-outputs?ref_type=heads