Invalid description of Wikimedia Enterprise HTML Dumps
Closed, ResolvedPublic1 Estimated Story Points
Actions

Assigned To

Authored By

	Mitar
	Jan 2 2022, 8:20 PM

Description

Reading here: https://dumps.wikimedia.org/other/enterprise_html/

It says:

Each dump output file consists of a tar.gz archive which, when uncompressed and untarred, contains one file

Emphasis mine. That is not true. Tar contains multiple files. E.g.:

enwiki_0.ndjson
enwiki_10.ndjson
enwiki_11.ndjson
enwiki_12.ndjson
enwiki_13.ndjson
enwiki_14.ndjson
enwiki_15.ndjson
enwiki_16.ndjson
enwiki_17.ndjson
enwiki_18.ndjson
enwiki_19.ndjson
enwiki_1.ndjson
enwiki_20.ndjson
enwiki_21.ndjson
enwiki_22.ndjson
enwiki_23.ndjson
enwiki_24.ndjson
enwiki_25.ndjson
enwiki_26.ndjson
enwiki_27.ndjson
enwiki_28.ndjson
enwiki_2.ndjson
enwiki_3.ndjson
enwiki_4.ndjson
enwiki_5.ndjson
enwiki_6.ndjson
enwiki_7.ndjson
enwiki_8.ndjson
enwiki_9.ndjson

Related Objects

Mentioned Here: T390556: Decommission dumps job `download_enterprise_htmldumps`

Event Timeline

Mitar created this task.Jan 2 2022, 8:20 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 2 2022, 8:20 PM

Protsack.stephan subscribed.Jan 4 2022, 7:39 AM

AnnaMikla added a project: Wikimedia Enterprise Volunteer Request.Jan 5 2022, 9:54 AM

Should be changed here: https://gerrit.wikimedia.org/g/operations/puppet/+/0fb35576ff6f1fb2aeefeef23b576343236b38ec/modules/dumps/files/web/html/enterprise_html_index.html

Protsack.stephan raised the priority of this task from Low to Needs Triage.Oct 12 2022, 9:42 AM

HShaikh moved this task from Incoming to Community requests on the Wikimedia Enterprise board.Mar 28 2023, 2:40 PM

creynolds subscribed.Dec 26 2023, 3:55 PM

A lot has happened since this ticket but to update:

Enterprise Dumps page has been updated with recent changes to dump runs; details on page.
New Gerrit commit sent to remove the incorrect language referenced in this ticket and simplify the page contextually based on said changes.
Updated the intro copy on Snapshot API (html dumps) page to clarify language about file contents referenced in this ticket.

With that all said, I'm going to close this ticket as the dumps page will be updated pending Gerrit approval(s) and deploy.

Oh, thanks. I didn't know that they are not made available anymore.

BTW, why has this directory been created, after the cut-off date? I hope this does not mean that old snapshots will slowly get removed by some cron job which is creating these empty directories?

Hey @Mitar those empty dirs shouldn't be created but there's a bug being worked on actively that'll fix it. There's also a talk page update about this too here.

Those empty dirs are still present: https://dumps.wikimedia.org/other/enterprise_html/runs/ Could they be removed please? So that it is easy to find the latest version of those dumps which are available archived?

@Mitar see T390556

Invalid description of Wikimedia Enterprise HTML DumpsClosed, ResolvedPublic1 Estimated Story PointsActions

Description

Related Objects

Event Timeline

Invalid description of Wikimedia Enterprise HTML Dumps
Closed, ResolvedPublic1 Estimated Story Points
Actions