Page MenuHomePhabricator

md5 and sha1 checksums are not available in dumpstatus.json for multistream dumps
Closed, ResolvedPublic

Description

All md5 and sha1 checksums for dump files are available in dumpstatus.json for the main database dumps, except for the multistream dumps. The same information is available in the -md5sums.txt, -sha1sums.txt and their respective JSON files, but just not in dumpstatus.json.

The md5 and sha1 checksums should be added into the dumpstatus.json file for the multistream dumps. A step is probably missing here.

Event Timeline

Hydriz created this task.Jan 22 2018, 9:10 AM
Hydriz updated the task description. (Show Details)
ArielGlenn triaged this task as Normal priority.Jan 23 2018, 2:22 PM

Change 409028 had a related patch set uploaded (by ArielGlenn; owner: ArielGlenn):
[operations/dumps@master] copy lists of file hashes into place before they are used for status reports

https://gerrit.wikimedia.org/r/409028

Change 409028 merged by ArielGlenn:
[operations/dumps@master] copy lists of file hashes into place before they are used for status reports

https://gerrit.wikimedia.org/r/409028

Mentioned in SAL (#wikimedia-operations) [2018-02-08T13:53:44Z] <ariel@tin> Started deploy [dumps/dumps@9b7841f]: make sure all hashes appear in dumpstatus file , T185454

Mentioned in SAL (#wikimedia-operations) [2018-02-08T13:53:56Z] <ariel@tin> Finished deploy [dumps/dumps@9b7841f]: make sure all hashes appear in dumpstatus file , T185454 (duration: 00m 02s)

This is now deployed but will not go into effect until the next run on the 20th of the month. Keeping open til I see good results from some wikis on that run.

ArielGlenn closed this task as Resolved.Feb 27 2018, 2:30 PM

The missing hashes now appear in dumpstatus.json as they should, verified for the Feb 20 run. Closing.

ArielGlenn moved this task from Incoming to Done on the Datasets-Archiving board.Aug 30 2018, 9:54 AM