Page MenuHomePhabricator
Create Task
Maniphest T118397

Add bzip2 support to DCAT-AP description of dumps
Closed, ResolvedPublic
Actions

Description

Since 2015-11-09 bzip2 compressed version of the Wikidata entity dumps are published alongside the gzip one. This compressed version is however not described by the DCAT-AP dump description.

Details

SubjectRepoBranchLines +/-
Support multiple compression formats for dumpsoperations/dumps/dcatmaster+105 -60
Customize query in gerrit

Related Objects
Search...

Event Timeline

Lokal_Profil created this task.Nov 11 2015, 3:50 PM
Lokal_Profil raised the priority of this task from to Needs Triage.
Lokal_Profil updated the task description. (Show Details)
Lokal_Profil added projects: Datasets-General-or-Unknown, Wikidata.
Lokal_Profil subscribed.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptNov 11 2015, 3:50 PM
Lokal_Profil added a comment.Nov 11 2015, 3:52 PM

Unlike different dump formats new compression formats are not as easy to add to the DCAT script.

Lokal_Profil mentioned this in T122500: dcat.rdf should also list the bzip2 compressed dumps.Dec 28 2015, 12:59 PM
Lokal_Profil merged a task: T122500: dcat.rdf should also list the bzip2 compressed dumps.
Lokal_Profil added a subscriber: hoo.
gerritbot subscribed.Jan 5 2016, 4:47 AM

Change 262422 had a related patch set uploaded (by Lokal Profil):
Support multiple compression formats for dumps

https://gerrit.wikimedia.org/r/262422

gerritbot added a project: Patch-For-Review.Jan 5 2016, 4:47 AM
Lokal_Profil added a comment.Jan 5 2016, 4:59 AM
In T118397#1915653, @gerritbot wrote:

Change 262422 had a related patch set uploaded (by Lokal Profil):
Support multiple compression formats for dumps

https://gerrit.wikimedia.org/r/262422

And https://gerrit.wikimedia.org/r/262423 for the config.json change

gerritbot added a comment.Jan 20 2016, 8:28 AM

Change 262422 merged by Hoo man:
Support multiple compression formats for dumps

https://gerrit.wikimedia.org/r/262422

hoo added a comment.Jan 20 2016, 8:29 AM

Merged into master, but still needs to be deployed (and I want localization updates unstuck before doing that). I'll keep this open so that we don't forget about it.

Lokal_Profil added a subtask: T122975: Automate translation updates for DCAT-AP.Jan 20 2016, 9:43 AM
Lokal_Profil added a comment.Jan 20 2016, 9:46 AM

I also added a follow-up patch to deal with (some of) the concerns about parameter passing.

https://gerrit.wikimedia.org/r/#/c/263169/

Lokal_Profil added a comment.Jan 21 2016, 8:09 AM
In T118397#1947211, @hoo wrote:

Merged into master, but still needs to be deployed (and I want localization updates unstuck before doing that). I'll keep this open so that we don't forget about it.

We also need to remember to merge+deploy the operations/puppet patch before the operations/dumps/dcat patch is deployed.

Lokal_Profil added a comment.Feb 2 2016, 12:19 PM

@hoo: Can https://gerrit.wikimedia.org/r/#/c/262423/ be merged? Should work fine even before the operations/dumps/dcat patch is deployed.

hoo added a comment.Feb 2 2016, 3:20 PM
In T118397#1989482, @Lokal_Profil wrote:

@hoo: Can https://gerrit.wikimedia.org/r/#/c/262423/ be merged? Should work fine even before the operations/dumps/dcat patch is deployed.

I talked to Ariel about this today… I guess it will be taken care of either today or tomorrow.

ArielGlenn subscribed.Feb 3 2016, 3:43 PM

I merged this today. Please check that everything's working as you expect.

ArielGlenn moved this task from Backlog to Active on the Datasets-General-or-Unknown board.Feb 3 2016, 3:43 PM
Lokal_Profil added a comment.Feb 3 2016, 6:19 PM
In T118397#1994095, @ArielGlenn wrote:

I merged this today. Please check that everything's working as you expect.

Thanks. Nothing should visibly change until https://gerrit.wikimedia.org/r/#/c/262422/ is deployed. And for that we are waiting to see if the next l10n-bot patch is succesfully submitted. I updated the sv translation so there should be a push in the works.

Lokal_Profil added a subscriber: Nikerabbit.Feb 4 2016, 4:31 PM

Hum.. so l10n-bot doesn't seem to be pushing the last changes. Any thoughts @Nikerabbit?

Nikerabbit added a comment.Feb 6 2016, 8:13 AM

I am semi-regularly running it on Mondays and Thursdays, but I had to skip it this Thursday.

Lokal_Profil added a comment.Feb 8 2016, 11:15 AM
In T118397#2005125, @Nikerabbit wrote:

I am semi-regularly running it on Mondays and Thursdays, but I had to skip it this Thursday.

That explains it :)

I just saw that the latest l10n-bot patch was submitted correctly so the setings seem to be ok now.

I guess that was the last blocker for deploying operations/dumps/dcat?

hoo closed this task as Resolved.Feb 9 2016, 8:00 PM
hoo claimed this task.

I just deployed 92ab37d94e and regenerated the RDF.

hoo removed a project: Patch-For-Review.Feb 9 2016, 8:00 PM
hoo set Security to None.
Lokal_Profil closed subtask T122975: Automate translation updates for DCAT-AP as Resolved.Mar 3 2016, 3:47 PM
ArielGlenn moved this task from Active to Done on the Datasets-General-or-Unknown board.Mar 4 2016, 10:36 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL