Page MenuHomePhabricator

Security review for the Wikidata primary sources tool MediaWiki extension
Closed, DeclinedPublic

Description

Project Information

Description of the tool/project

A platform to release and curate third-party datasets for Wikidata.
This MediaWiki extension serves as the front-end component of the tool.

Description of how the tool will be used at WMF

The target wiki is Wikidata. For a detailed description, see https://www.wikidata.org/wiki/Wikidata:Primary_sources_tool

Dependencies

List dependencies, or upstream projects that this project relies on.
Wikidata.

Has this project been reviewed before?

Please link to tasks or wiki pages of previous reviews.
No.

Working test environment

Please link or describe setup process for setting up a test environment.
The test environment is available at the following MediaWiki Vagrant test instance: https://wikidata-pst.wmflabs.org/

Post-deployment

Name of team responsible for tool/project after deployment and primary contact.
Wikidata

Event Timeline

Hjfocs created this task.Sep 17 2018, 3:52 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 17 2018, 3:52 PM
Addshore moved this task from incoming to monitoring on the Wikidata board.Sep 19 2018, 7:18 AM

Not seeing anything in master or REL1_32 for this. Is it somewhere else? If not, is there an estimate for completion?

Not seeing anything in master or REL1_32 for this. Is it somewhere else?

Not sure what you mean, the code is here: https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki/extensions/PrimarySources

If not, is there an estimate for completion?

If you are referring to the extension development, the first release is completed since several months.

Aklapper added a comment.EditedDec 14 2018, 11:50 AM

Not sure what you mean, the code is here:

@Hjfocs: There is no code. https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/PrimarySources/+/master is empty.

https://www.wikidata.org/wiki/Wikidata:Primary_sources_tool#Current_code_base instead links to Github and not Wikimedia Git/Gerrit. That Github link is a 404 error.

You cannot review code without documenting where the code to review is. :)

@Hjfocs: There is no code. https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/PrimarySources/+/master is empty.

Interesting, will find out what went wrong. Thanks for the extra pair of eyes!

https://www.wikidata.org/wiki/Wikidata:Primary_sources_tool#Current_code_base instead links to Github and not Wikimedia Git/Gerrit. That Github link is a 404 error.

You are mentioning references that describe version 1: I fixed the outdated links, thanks for pointing them out. I also updated the section title.

Anyway, the main GitHub repository is here:
https://github.com/Wikidata/primarysources/

sbassett triaged this task as Low priority.Dec 14 2018, 3:11 PM

Ok, thanks for the update, @Hjfocs.

sbassett updated the task description. (Show Details)Dec 14 2018, 7:38 PM

Hello @Hjfocs

Some follow-up here - apologies for the stop/go on this one:

  1. Did the mirroring issue with gerrit ever get addressed? It still looks to be an empty repo.
  2. I was curious if the tool is actually working in production. On wikidata.org, I added the gadget and selected "All Sources" within the config and the interstitial just kind of hangs indefinitely for me. (Chrome 71.0.3578.98 on Mac Mojave.) Is this expected for now?
  3. Is the current version (gadget/backend at pst.wmflabs.org) the indefinite production version for now? Looking at wikidata.org/wiki/Special:Version, I'm not seeing a deployed extension, so I assume that might be part of a forthcoming development cycle?
Hjfocs added a subscriber: MaxSem.Jan 21 2019, 5:09 PM

Hi @sbassett and thanks for the follow up

  1. Did the mirroring issue with gerrit ever get addressed? It still looks to be an empty repo.

@MaxSem spotted this (thanks for the review!):
https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/PrimarySources/+/436525/

Still, I remember I couldn't push the code into gerrit, failing with a permission denied error. Maybe you can give me a hand here?

  1. I was curious if the tool is actually working in production. On wikidata.org, I added the gadget and selected "All Sources" within the config and the interstitial just kind of hangs indefinitely for me. (Chrome 71.0.3578.98 on Mac Mojave.) Is this expected for now?

This is version 1, and, yes, it's probably broken because it's not maintained anymore.

Version 2, i.e., the one running a MediaWiki extension as its front end, is deployed in a test Wikidata instance: https://wikidata-pst.wmflabs.org/

The item-based view is not fully usable because the import of the full Wikidata dump only partially succeeded.
The importDump.php maintenance script didn't seem to work effectively for Wikidata.
You can refer to https://www.mediawiki.org/wiki/MediaWiki-Vagrant#How_to_import_a_Wikidata_dump for the import procedure I used.

  1. Is the current version (gadget/backend at pst.wmflabs.org) the indefinite production version for now?

Correct, although the caveats above apply.

Looking at wikidata.org/wiki/Special:Version, I'm not seeing a deployed extension, so I assume that might be part of a forthcoming development cycle?

Correct, version 2 should undergo the MediaWiki extension review process before going to https://test.wikidata.org, and finally to https://www.wikidata.org

Hope this helps clarify!

Post-deployment
Name of team responsible for tool/project after deployment and primary contact.
Wikidata

Are they aware of it?

MaxSem closed this task as Declined.Jan 22 2019, 2:21 AM

Per T196073#4896829, this is not ready for review yet. Please feel free to reopen when you're ready to work on it and the extension has been merged into Git master. For now, there's no need to distract the security team with this.

I'm sorry I don't have enough knowledge of your internal procedures to understand your point, @MaxSem:

Per T196073#4896829, this is not ready for review yet.

But you served as the first reviewer, what am I getting wrong?

when (...) the extension has been merged into Git master.

Can you please clarify what are the steps needed to address this? First of all, which Git master are you talking about? The extension code already lives in the master branch of its Git repository: https://github.com/Wikidata/primarysources/tree/master
I guess you are implicitly referring to some other Git master.

sbassett added a comment.EditedJan 23 2019, 3:15 PM

@Hjfocs -

But you served as the first reviewer, what am I getting wrong?

From T196073#4825203, it looks like @MaxSem found the PrimarySources code as an unmerged Gerrit patch set, and offered some initial feedback (thanks!) However, this isn't typical of a standard security review as performed by the Security-Team (see mw documentation). The Security-Team's expectations, in addition to receiving a request like this ticket would be:

  1. The code to be reviewed would be nearly ready for production deployment. It seems like PrimarySources may be in some extended testing phases and/or pilot launches, so I'm not sure it meets this criteria at the moment. Please correct me if this is not the case.
  2. Per this documentation, we typically expect Gerrit to be the canonical location for any Wikimedia code to be security-reviewed. We can work with an unmerged patch set, but I think there was some initial confusion as the Gerrit repository only had the code of conduct file in it (and still does) and we typically don't try to hunt things down on Github.
  3. It will soon be a requirement for submitters of security reviews to provide some kind of working test environment (Dockerfile, etc) or instructions on configuring such an environment (i.e. extension installation, dependencies, oddities, etc.) with any security review request. I currently do not see anything like this mentioned within this ticket or within any READMEs or other documentation (especially for the java backend) which would complicate or greatly inhibit the success of this review.

Once these issues are addressed, we can re-open this ticket and get it scheduled for a formal review.