Page MenuHomePhabricator
Create Task
Maniphest T221916

Create RDF export for structured data stored for files
Closed, ResolvedPublic
Actions

Description

Structured data information on Commons should have RDF representation, just like the information on Wikidata has. We need to create code that would allow RDF representation of that data.

The URL should be like this:
https://commons.wikimedia.org/wiki/Special:EntityData/M77666670.ttl (just like JSON one works now).

One of the ideas of how RDF could look like is here: https://www.mediawiki.org/wiki/User:Tpt/WikibaseMediaInfo_RDF_Dump_Format

Open questions:

  • What should be the main prefix? wd: is wikidata, so we can't use that. sdc: maybe?
  • How to ensure that URLs used by Wikidata commonsMedia type match what we're using here? Right now commonsMedia uses Special:FilePath URL, which is different from sitelink. Should we add another triple? Which URL should we use for schema:contentUrl - current full URL or maybe FilePath redirect?

Details

SubjectRepoBranchLines +/-
Adds more File metadata to RDF outputmediawiki/extensions/WikibaseMediaInfomaster+106 -18
Adds some RDF triples from File metadatamediawiki/extensions/WikibaseMediaInfomaster+213 -10
Enable RDF output for MediaInfooperations/mediawiki-configmaster+1 -1
RDF export of MediaInfo entitiesmediawiki/extensions/WikibaseMediaInfomaster+325 -1
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 DeclineddchenT118706 Conduct heuristic evaluation of image upload and insert flow in VisualEditor
 OpenNoneT115858 Design improvements for mw.ForeignStructuredUpload.BookletLayout
 OpenNoneT115865 Insert image in content immediately after it's uploaded, skipping the "General settings" step
 DuplicateNoneT115864 Figure out if the description of the image can be used as the caption on-wiki
 OpenFeatureNoneT53032 When inserting an image, set its caption by default to be the Commons image description
 OpenFeatureNoneT39534 Wikimedia Commons should support searching by color
 DuplicateNoneT39535 Wikimedia Commons should support filtering by color
 ResolvedNoneT19503 Provide metadata support on Wikimedia Commons
 ResolvedNoneT51662 VisualEditor: Use Multimedia/Wikidata's proposed rich structured meta-data in the image insertion dialog
 ResolvedNoneT68108 [Epic] Store media information for files on Wikimedia Commons as structured data
 DuplicateNoneT141602 [Objective Fiscal 19-20/Q4] (9) Provide a Proof of Concept SPARQL endpoint in support of SDoC project
 ResolvedArielGlennT221917 Create RDF dump of structured data on Commons
 ResolvedSmalyshevT221916 Create RDF export for structured data stored for files
 ResolvedSmalyshevT222299 dumpRdf does not work with MediaInfo entities
 ResolvedSmalyshevT222302 Too many RDF label forms for MediaInfo
 ResolvedSmalyshevT222306 RDF export generates wrong IDs for federated entities
 ResolvedGehelT222321 Make /entity/ alias work for Commons
 ResolvedSmalyshevT222995 Decide which prefixes to use for MediaInfo RDF

Event Timeline

Smalyshev created this task.Apr 25 2019, 11:42 PM
Smalyshev added a parent task: T221917: Create RDF dump of structured data on Commons.Apr 25 2019, 11:45 PM
Smalyshev updated the task description. (Show Details)Apr 28 2019, 9:09 PM
Smalyshev added a project: User-Smalyshev.Apr 29 2019, 6:32 AM
Smalyshev claimed this task.May 1 2019, 7:02 PM
Smalyshev triaged this task as Medium priority.
Smalyshev moved this task from Backlog to Doing on the User-Smalyshev board.
gerritbot added a comment.May 1 2019, 7:05 PM

Change 507627 had a related patch set uploaded (by Smalyshev; owner: Smalyshev):
[mediawiki/extensions/WikibaseMediaInfo@master] RDF export of MediaInfo entities

https://gerrit.wikimedia.org/r/507627

gerritbot added a project: Patch-For-Review.May 1 2019, 7:05 PM
Smalyshev moved this task from Doing to Waiting/Blocked on the User-Smalyshev board.May 2 2019, 6:58 AM
Smalyshev merged a task: T215341: RDF for MediaInfo entities.May 2 2019, 7:04 AM
Smalyshev added a subscriber: WMDE-leszek.
Smalyshev moved this task from Incoming to SDAW on the Wikidata-Query-Service board.May 2 2019, 5:39 PM
Smalyshev added a project: Discovery-Wikidata-Query-Service-Sprint.May 28 2019, 5:32 PM
Smalyshev moved this task from Backlog to Needs review on the Discovery-Wikidata-Query-Service-Sprint board.
Smalyshev closed subtask T222995: Decide which prefixes to use for MediaInfo RDF as Resolved.Jun 10 2019, 10:34 PM
Addshore moved this task from incoming to in progress on the Wikidata board.Jun 21 2019, 11:25 PM
Smalyshev closed subtask T222299: dumpRdf does not work with MediaInfo entities as Resolved.Jun 26 2019, 6:18 PM
MB-one unsubscribed.Jun 27 2019, 7:21 AM
gerritbot added a comment.Jul 1 2019, 5:36 PM

Change 507627 merged by jenkins-bot:
[mediawiki/extensions/WikibaseMediaInfo@master] RDF export of MediaInfo entities

https://gerrit.wikimedia.org/r/507627

ReleaseTaggerBot added a project: MW-1.34-notes (1.34.0-wmf.13; 2019-07-09).Jul 1 2019, 6:00 PM
Maintenance_bot removed a project: Patch-For-Review.Jul 1 2019, 6:10 PM
gerritbot added a comment.Jul 1 2019, 8:24 PM

Change 520078 had a related patch set uploaded (by Smalyshev; owner: Smalyshev):
[operations/mediawiki-config@master] Enable RDF output for MediaInfo

https://gerrit.wikimedia.org/r/520078

gerritbot added a project: Patch-For-Review.Jul 1 2019, 8:24 PM
gerritbot added a comment.Jul 8 2019, 6:01 PM

Change 520078 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable RDF output for MediaInfo

https://gerrit.wikimedia.org/r/520078

Stashbot added a comment.Jul 8 2019, 6:04 PM

Mentioned in SAL (#wikimedia-operations) [2019-07-08T18:04:14Z] <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[:gerrit:520078|Enable RDF output for MediaInfo]] (T221916) (duration: 00m 49s)

Maintenance_bot removed a project: Patch-For-Review.Jul 8 2019, 6:11 PM
Smalyshev closed subtask T222302: Too many RDF label forms for MediaInfo as Resolved.Jul 16 2019, 4:37 AM
gerritbot added a comment.Jul 17 2019, 5:02 PM

Change 522141 had a related patch set uploaded (by Smalyshev; owner: Tpt):
[mediawiki/extensions/WikibaseMediaInfo@master] Adds some RDF triples from File metadata

https://gerrit.wikimedia.org/r/522141

gerritbot added a project: Patch-For-Review.Jul 17 2019, 5:02 PM
Smalyshev moved this task from Needs review to In progress on the Discovery-Wikidata-Query-Service-Sprint board.Jul 23 2019, 5:43 PM
Smalyshev moved this task from In progress to Done on the Discovery-Wikidata-Query-Service-Sprint board.Aug 4 2019, 4:08 AM
Smalyshev closed this task as Resolved.Aug 19 2019, 7:01 PM

I think this is done since the export now exists and is enabled, we can tweak the specifics in follow-up tasks.

gerritbot added a comment.Aug 27 2019, 1:13 PM

Change 522141 merged by jenkins-bot:
[mediawiki/extensions/WikibaseMediaInfo@master] Adds some RDF triples from File metadata

https://gerrit.wikimedia.org/r/522141

ReleaseTaggerBot edited projects, added MW-1.34-notes (1.34.0-wmf.21; 2019-09-03); removed MW-1.34-notes (1.34.0-wmf.13; 2019-07-09).Aug 27 2019, 2:00 PM
Maintenance_bot removed a project: Patch-For-Review.Aug 27 2019, 2:10 PM
Smalyshev closed subtask T222306: RDF export generates wrong IDs for federated entities as Resolved.Aug 28 2019, 6:36 AM
gerritbot added a comment.Aug 28 2019, 6:57 PM

Change 533077 had a related patch set uploaded (by Tpt; owner: Tpt):
[mediawiki/extensions/WikibaseMediaInfo@master] Adds more File metadata to RDF output

https://gerrit.wikimedia.org/r/533077

gerritbot added a project: Patch-For-Review.Aug 28 2019, 6:57 PM
gerritbot added a comment.Sep 3 2019, 10:20 AM

Change 533077 merged by jenkins-bot:
[mediawiki/extensions/WikibaseMediaInfo@master] Adds more File metadata to RDF output

https://gerrit.wikimedia.org/r/533077

Maintenance_bot removed a project: Patch-For-Review.Sep 3 2019, 11:10 AM
DD063520 subscribed.Feb 19 2020, 10:49 AM

Hi,

is this export (dump) available to download somewhere? Is this planned? Should I add an additional ticket for that?

Thank you
D063520

TJones closed subtask T222321: Make /entity/ alias work for Commons as Resolved.Mar 4 2020, 4:24 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL