Page MenuHomePhabricator

GSOC'17 Final Report : "Add a 'hierarchy' type to the Cargo extension"
Closed, ResolvedPublic

Description

GSOC Proposal Link: https://phabricator.wikimedia.org/T161609

Projects worked on: Cargo, Page Forms

Github Mirror Repository - Cargo: https://github.com/wikimedia/mediawiki-extensions-Cargo
Github Mirror Repository - Page Forms: https://github.com/wikimedia/mediawiki-extensions-PageForms

Primary Project Page: https://www.mediawiki.org/wiki/Extension:Cargo

About my GSoC project

The project is about adding support for hierarchy fields in Cargo MediaWiki extension. The project also involves the addition of support for hierarchy fields in Page Forms MediaWiki extension, to ensure easy input from users entering hierarchy data into the wiki.

What are hierarchy fields

There was a need for a field that allows to store and query data of the following form, efficiently.

Screen Shot 2017-08-29 at 2.45.09 AM.png (259×464 px, 38 KB)

The proposed hierarchy field stores the required information to understand such relation, in order to perform query efficiently.

What ease does it provide

Several times the data has such nature and the need arises to query the data over a value as the parent and the children (all the successors). My work allows to do certain queries over the hierarchy fields in an easy manner using "WITHIN" and "HOLDS WITHIN".

Let’s say there is a Cargo table called “Cuisines”, holding some Cuisines and it’s “Main Ingredient”, you can do the following query to look for Cuisines having "Main ingredient" as one of the “Root Vegetables”.

{{#cargo_query:
tables=Cuisines
|fields=_pageName=Cuisine,Main_ingredient
|where=Main_ingredient WITHIN “Root Vegetables”
}}

The hierarchy field can also hold a list of values. In that case, let’s say there is a table called “Movies”, which has information about some movies and contains a field which holds hierarchy as well as list of values called “Genres”. If you want to query all the movies with Genre - Fiction and its sub-genres, you could do the following query:

{{#cargo_query:
tables=Movies
|fields=_pageName=Movie
|where=Genres HOLDS WITHIN “Fiction”
}}

For easy usability, a tree input has been chosen as default input type through Page Forms which looks like following:

Screen Shot 2017-08-29 at 2.55.54 AM.png (231×1 px, 46 KB)

The drilldown feature of Cargo extension allows querying by clicks and allows to have an overview of the counts of the values in the pages. The drilldown has been adapted accordingly so as to allow usage with hierarchy fields.

Planned Objectives

  1. Add support for declaring and querying over hierarchy fields in Cargo Extension.
  2. Add support for query using new Cargo keyword - “WITHIN”.
  3. Modify Cargo’s 'Drilldown' interface to incorporate hierarchy fields.
  4. Add support for hierarchy fields in the Page Forms extension to set default input type as "tree" for hierarchy field.
  5. Add support for hierarchy fields in the helper pages of Page Forms extension for compatibility with Special:CreateClass and Special:CreateTemplate.

Status of the objectives

I have completed all the above-proposed objectives. In addition to that, I have fixed some other minor bugs which I have mentioned in the 'miscellaneous' section of Change logs.

The only work left to do is to update the project's documentation. It's been proposed to be done along with the release of the new version of the software. However, I have prepared my part of the documentation ( here).

Change log

Add support for declaring and querying over hierarchy fields in Cargo Extension
Change LinkDescription
https://gerrit.wikimedia.org/r/#/c/356901/Feature: Hierarchy option in Cargo Declare
https://gerrit.wikimedia.org/r/#/c/359793/Feature: Hierarchy Structure Table for Hierarchy Field
https://gerrit.wikimedia.org/r/#/c/360916/Fix Populate Hierarchy fields
https://gerrit.wikimedia.org/r/#/c/363523/Add new column "field_helper_tables" to "cargo_tables" , Fix dbjoin error by creating above new field
https://gerrit.wikimedia.org/r/#/c/373600/Make hierarchy additional parameter order insensitive
Add support for query using new Cargo keyword - “WITHIN”
Change LinkDescription
https://gerrit.wikimedia.org/r/#/c/363178/Add feature for WITHIN and HOLDS WITHIN for hierarchy fields
Modify Cargo’s Drilldown interface to incorporate hierarchy fields
Change LinkDescription
https://gerrit.wikimedia.org/r/#/c/367808/Make static function of CargoHierarchy suitable for extending the class
https://gerrit.wikimedia.org/r/#/c/365091/Add support for hierarchy fields in Drilldown
https://gerrit.wikimedia.org/r/#/c/372232/Add feature to collapse deeper levels of hierarchy to show maximum of \$wgCargoMaxHierarchyDrilldownValues nodes
https://gerrit.wikimedia.org/r/#/c/372526/Fix: Within filter not being applied with other filters
https://gerrit.wikimedia.org/r/#/c/372563/Perform literal "drilldown" in the applied filter line for hierarchy field in case of within filter
Add support for hierarchy fields in the Page Forms extension to set default input type as tree for hierarchy field
Change LinkDescription
https://gerrit.wikimedia.org/r/#/c/356928/Feature : Adding tree as default type for hierarchy
Add support for hierarchy fields in the helper pages of Page Forms extension for compatibility with Special:CreateClass and Special:CreateTemplate
Change LinkDescription
https://gerrit.wikimedia.org/r/#/c/371512/ , https://gerrit.wikimedia.org/r/#/c/371574/Fix : No comma escaping for hierarchy structure
https://gerrit.wikimedia.org/r/#/c/371160/Add hierarchy field creation in Special:CreateTemplate
https://gerrit.wikimedia.org/r/#/c/371583/Add hierarchy field creation in Special:CreateClass
Other miscellaneous changes that were required
Change LinkDescription
https://gerrit.wikimedia.org/r/#/c/361287/Fix : When table is deleted Drilldown crashes
https://gerrit.wikimedia.org/r/#/c/361291/Fix : cargoRecreateData.php crash after the template is deleted by the user.
https://gerrit.wikimedia.org/r/#/c/361275/Fix Index Naming of Multi valued fields (lists)
https://gerrit.wikimedia.org/r/#/c/371206/Rename CargoHierarchy.php and its class variable
https://gerrit.wikimedia.org/r/#/c/372082/Add validation for hierarchy input

Link to all the merged commits

View at Github: All merged commits in Cargo Extension Project
View at Github: All merged commits in Page Form Extension Project

Link to all the commits merged/open

View at Gerrit: All commits

Acknowledgements

First and foremost, I would like to thank my mentors Yaron Koren(@Yaron_Koren), Nischay Nahata(@Nischayn22) and Tobias Oetterer(@Oetterer) for guiding me, and solving all of my doubts patiently. This project would not have completed without their contribution.

Special thanks to Srishti Sethi(@srishakatux) and the complete Wikimedia Foundation, for all their support.

Last but not least, I would like to thank Google for providing this opportunity to explore the open source software through the "Google Summer of Code" program.