= Project Title =
Integration of IFTTT support for WikiData
= Personal Information =
Name: **Alangi Derick Ndimnain**
Email: **alangiderick@gmail.com**
IRC Nick: **d3r1ck**
= Background Information =
I am a fourth year Computer Engineering student, specializing in Software Engineering from the University Of Buea. I participated in as a mentor for the 2015 Google Code-In for Wikimedia Foundation(WMF) and mentored 6 projects. I have been contributing to WMF for about 7 months now and my contribution can be seen here: [[ https://gerrit.wikimedia.org/r/#/q/owner:D3r1ck01+status:merged,n,z | Contributions ]], where I have contributed patches across many different extensions.
= Programming Background =
- Google Code-In 2015 Mentoring - Projects Mentored.
-- T118390: Using Maniphest's advanced search Documentation Screen cast.
-- T116802: Watching a project to receive its notifications Documentation Screen cast.
-- T118389: Screencast showing how to request a project
-- T121911: Remove deprecated html elements like <font> from SemanticForms
-- T108432: Desktop: Search box inaccessible in Special:Gather
-- T122968: Add a composer.local.json-sample to MediaWiki core
- My Contributions to WMF Code base - Extensions Worked on
-- Echo Extension
-- Gather Extension
-- Graph Extension
-- Mailgun Extension (co-authoring)
-- MobileFrontend Extension
-- Newsletter Extension
-- Semantic Forms Extension
-- Thanks Extension
-- Wikibase Extension
-- WikibaseJavaScripAPI Extension
- Also worked on OOJs/UI and MediaWiki Core.
- Skills
-- Languages: PHP(Excellent), Python(Intermediate), JSON(Excellent), JavaScript/jQuery(Intermediate), CSS(Excellent), SQL(Proficient), HTML(Excellent).
-- Tools: Secure Shell, Git/Github, Gerrit, IFTTT, Linux OS(Ubuntu) and derivatives, Subversion, Vagrant, Composer.
-- Frameworks: Flask(A Python Micro-framework)
== Project Mentors ==
- Primary Mentors: @Slaporte, @Bene
- Secondary Mentors: @hoo, @Lydia_Pintscher
- Other Mentors: @Samtar
= Synopsis/Project Summary =
This project aims at integrating IFTTT(IF This Then That) feature to the WikiData extension. IFTTT technology being on of the most popular ways of making chains of web services communicate with one another, will be used to make WikiData to communicate with other web services across the web. This project involves studying the WikiData API, IFTTT technology internals and Communication of Web services.
= Detailed Project Description =
== Introduction ==
WikiData is a knowledge base used to keep information about objects (real life entities). These information can be edited and used by both humans and machines. WikiData stands as a central point of storage for all Wikimedia related projects and the information in WikiData is available in all languages supported by the Wikimedia projects. In addition, information is stored in WikiData in the form of a document-oriented manner and saved in the database with prefix Q for an Item. Example Q123 can be an item in WikiData holding information about a real data like; Social Media.
IFTTT or "If This then That" is a web service that amalgamates many other web applications and services into on place, and can then perform several action based on certain set of criteria. With IFTTT, you can create a channel for an application or a web service, for a example WikiData. A WikiData channel can be created in IFTTT which will be used to create a communication between IFTTT and WikiData. IFTTT technology is used to automate various task on the internet for users based on certain criteria(triggers - "this") and the response of the trigger is an action - "that".
== Implementation Approach ==
Since this feature has been implemented for Wikipedia by @Slaporte([[ https://github.com/wikimedia/ifttt | His Implementation ]]) which runs on Wikimedia Labs [[ http://tools.wmflabs.org/ifttt/ | here ]], I will use a similar approach in integrating the feature for WikiData. This will require me to write a web service application in Python(running on the flask framework) to read data from WikiData using the MediaWiki and WikiData API from time-to-time(intervals of 10 - 15 mins). This is to make sure that by chance, information in WikiData has changed such that the new information inputted in WikiData can be pulled using the WikiData API by the IFTTT WikiData web service to [[ http://www.ifttt.com | IFTTT]] for the actions to take place.
The IFTTT WikiData web service application we will develop will stand as an intermediate entity between [[ http://www.ifttt.com | IFTTT websitte ]] and WikiData. Below is the set-up or architecture on how the proposed system should work.
{F3549542}
WikiData uses the MediaWiki API to check if there is a change in the content of WikiData using revisions. These revisions will be filtered to extract the actual change(difference between the previous content and the current content) so that data can be returned to serve the purpose of triggers.
== Algorithm ==
This algorithm will be programmed to execute every 10 - 15 minutes, this is to ensure that the content of data in WikiData might have changed so that NULL information will not be returned all the time. Also the time is not very high because if it is, much data will be available which will increase the through put of the web service overloading it too much data slowing down the whole process.
```
IFTTT site connects and listens to get data from WikiData IFTTT web service;
IFTTT WikiData web service uses the WikiData API to read new data from WikiData;
if( changes in WikiData ) {
data = New data is return as JSON(or other formats) to the web service;
IFTTT WikiData web service sends data to IFTTT site;
IFTTT site furthers the processing;
} else {
No data gets returned;
Wait for 10 - 15 mins and tries to read data from WikiData;
}
```
== Components of the IFTTT WikiData web service ==
Based on the current implementation of this feature in Wikipedia, the various components will be used in the WikiData IFTTT web service.
- **Core**: This is the main application. The core will hold a list of the triggers that will be implemented and also some utilities that will be used in core of the application. The core will mostly consist of; function definition to handle basic HTTP errors like (401, 404, 500 etc..) and also routes that will be used to do the calls to the WikiData API from time to time to get recent changes.
- **Database Access Layer**: This is a component the IFTTT WikiData what will be using the databases. Any database transaction, mostly SQL queries will be handled in this component of the application.
- **Utilities**: This part will just contain functions for handling certain tasks. Basic utilities like; conversion from one time format to another and simple functions to handle very specific tasks. These component in a nut shell will hold general functions.
- **Triggers**: These will contain various classes that will handle all the triggers that will be decided and implemented, for example; Creating a New Item with a specific label or hash tag, Editing an item etc... This component will be responsible for reading data using the [[ https://www.wikidata.org/w/api.php | WikiData API ]] from WikiData. These data will be returned as JSON so that IFTTT can feed on it. WikiData uses the MediaWiki API to check if there is a change in the content of WikiData using revisions.
- **Views**: This will contain a class which will be used by all triggers. This class will have methods what will be used by all the triggers that will be implemented in this app. Its called views because it is what the front-end of IFTTT uses based on the user account settings to gather information and return to the front-end of the user(hence Views). Example method in this class can be a method to handle the post request to the WikiData API which will be used by all triggers to get specific information from WikiData.
= Testing and Verification =
== Testing ==
This phase will be done by a tool called "curl", its a command line Linux tool used to transfer data from or to a server, using one of the supported protocols (HTTP, POST, FTP and more...). Using this tool, we shall mostly be sending POST requests to the WikiData server in connection with the WikiData API to pull information(recent changes) from the server which will intern be sent to the IFTTT WikiData web service to push the data further to the IFTTT site.
Using the "curl" too, we can set the Content-type which is the form in which we want the curl to return the data. It can be in the form of JSON, XML etc... An example of a call to the Wikipedia "article of the day" trigger using curl and the current @Slaporte implementation of the IFTTT app is below:
```
curl -X POST --header "IFTTT-Channel-Key: <channel-key>" --header "Content-ype: application/json" --data '{"triggerFields":{"lang":"en"}}' http://tools.wmflabs.org/ifttt-dev/v1/triggers/article_of_the_day
```
The above command will return the article of the day trigger which will be fed to the IFTTT site and sent to all users who has configured their IFTTT accounts to get Articles of the Day trigger.
Automated unit testing shall also be used in other to ease the testing phase or have a pre-test by the jenkins bot before human test. This will make testing easier based on constraints like; syntax, etc... If there is a bot for automated unit testing, certain errors will easily be found and corrected before merging to the main repository. So in this light, a testing bot(jenkins) will be set-up to do automated unit testing.
== Verification ==
This process will involve me to check and see if my implementation of the WikiData IFTTT web service is functioning as planned. I will verify that every single functionality of the IFTTT web app is working correctly and giving the correct output to be fed into IFTTT site for further processing. In this process, I will verify all the implemented triggers, to make sure they work and return the correct values/data.
= Development Schedule/Timeline =
This is a plan that will be used during the GSoC period but can be slightly modified as GSoC proceeds
- April 23 - May 7 (2 weeks)
-- Community bonding period.
-- Study more docs related to the project(WikiData API, WikiData extension, IFTTT) in more detail
-- Check current implementation of IFTTT web app for Wikipedia Channel by @Slaporte [[ https://github.com/wikimedia/ifttt | here ]].
- May 8 - May 22 (2 weeks)
-- Community bonding period continue.
-- Discuss with mentors on implementation strategies.
-- Brain storm with the mentors and decide on the triggers what will be implemented.
- May 23 - June 13 (1 weeks)
-- Request for a Gerrit repo from MediaWiki admins for development of the Web service here: [[ https://www.mediawiki.org/wiki/Gerrit/New_repositories | Request New Gerrit Repo ]]
-- After repo has been created, set up jenkins bot for automated unit testing. This will be done when there is a push to the gerrit repo before merge.
- May 31 - June 13 (2 weeks)
-- Implement most of the triggers already supported for Wikipedia.
-- Implement triggers for; New Item with specific label(these labels are set up by users of IFTTT in their accounts in IFTTT), Editing an Item.
-- Testing and verification, regression testing, debugging and documentation.
- June 14 - June 21 (1 week)
-- Unit testing for the implemented triggers.
-- Bug fixes and more testing to make sure the implemented triggers passes all tests.
-- Submit work for mid-term evaluation.
- June 22 - July 6 (3 weeks)
-- Continue implementing more triggers
-- Implement triggers for; editing descriptions in items, site link added to an item, Creating a new property.
-- Testing and verification, debugging and documentation
- July 7 - July 14 (1 week)
-- Unit testing on the second set of triggers implemented.
-- Bug fixes and more testing to the second set of implemented triggers.
- July 14 - July 28 (2 weeks)
-- Finish up implementation of any triggers left; if during the GSoC period, more triggers are suggested, some of these will be implemented.
-- Testing and verification on the newly implemented triggers
-- Debugging, bug fixing and documentation.
- July 29 - August 22 (3.5 weeks)
-- Creating and configuration of a WikiData channel on IFTTT
-- Integration testing of the WikiData IFTTT web app to IFTTT
-- Unit testing and regression testing on the whole WikiData IFTTT web app
- August 23 - August 29 (1 week)
-- Pencils Down, Code clean up.
-- Improve and review documentation.
-- Final evaluation, Submission of code to Google.
= Time Availability =
I would be able to offer over 40 hours per week on the project. Also, to meet up with the demands of the project, I would be coding during weekends and regularly informing my mentors on the status of the project and regularly updating my logs in this respect.
= Why Wikimedia Foundation(WMF)? =
Wikimedia Foundation which focuses on encouraging the growth, development and distribution of free, multilingual, educational content, and to providing the full content of these wiki-based projects to the public free of charge is an organisation worth working with to make my continent(Africa) especially Cameroon to be sensitized about such organisations, their projects and projects which enriches knowledge to the society for free. This will go a long way to improve my community and Africa including the world as a whole in terms of education and academics "for free".
= Why WikiData Extension? =
Aiming at making my Final Year Project(FYP) to focus on Web Services and Data Manipulation, this extension has given me the opportunity to work on the project I have every dreamt. Integrating the IFTTT feature for the WikiData extension, this has made me to focus on this extension for my GSoC project 2016.
= Why me for the project? =
First and far most, looking the user facing nature of this project where hundreds of thousands of users will utilize, I am really inspired to work on the project and in addition to that, with the amount of knowledge I have acquired and my deep understanding of the requirements of the project, I think I'm in the best position to execute this project. Perhaps as I am going to use this project as my Final Year Project in my institution, this will give me an opportunity to promote WikiData and the MediaWiki project itself, sensitizing a lot of users in Africa to use this great feature.
= Work after the Summer of Code =
Due to the user-facing nature and Long Term Support(LTS) of this application in WikiData, after the summer, I will be always there to maintain and add more functionalities to the applications. This maintainability will span from user contributions, to personal features and more features that will be added to the project and the floor will be open for other contributors to work on the project and submit Pull Requests(PR). Also, as I mentioned in one of my mails, I will like to make Africa and specifically Cameroon(my country) to know about Wikimedia Foundation and its project so they can be sensitized and contribute to the project and also to make them aware of the free knowledge in the web through Wikipedia. In addition, I will use this project as my B.Eng thesis in my institution, creating a bridge between the Wikimedia Foundation(WMF) especially WikiData and my Institution and Cameroon as a whole.
= My Contributions =
Since I joined the Wikimedia Foundation(WMF) around September 2015 till now, I have contributed in several ways in the improvement of this organisation both in coding and mentoring perspectives.
- In terms of coding and submission of patches, I have over 20 patches merged, check [[ https://gerrit.wikimedia.org/r/#/q/owner:D3r1ck01+status:merged,n,z | here ]] and more are still to come.
- Also in terms of Mentoring, like I mentioned above in my Programming Background section, I mentored the Google Code-In 2015 program for the organisation. I mentored 6 projects.