This is a tracking task for our project to improvement the current article recommendation system in the Android app.
**Goal:** This project aims to increase user engagement on the Android app. The engagement should be increase by providing a better article recommendation system, which assists users in accessing relevant articles.
**Approach:** We want to develop a framework that allows the evaluation of recommender systems (RS) with respect to our goal. The framework requires the integration of additional RS into the MediaWiki/CirrusSearch API, the use of these recommendation in the Android app and measurement of performance metrics.= Summary
* We want to test a new recommendation algorithm, A fixed success criterion should determine whether a RS out-performs the current system.
At first,namely Citolytics. the link-based recommendations from [[ https://github.com/wikimedia/citolytics| Citolytics ]] should be tested.
**Sub-tasks:**Citolytics would be an additional source for the read more feature ([[ https://www.mediawiki.org/wiki/Reading/Web/Projects/Related_pages | related pages project page ]]).
# T149682 Define success criteria* In contrast to MoreLikeThis which is currently in use, Citolytics finds related Wikipedia pages based on links instead of the article text. With its different algorithmic approach, we hope to increase the user engagement by providing better recommendations.
* The evaluation framework can be used as general blueprint for future optimizations of the related pages feature.
= Implementation
In order to test the recommendations we must modify or access mainly three components of the Wikipedia system (see sub-tasks for details):
* MediaWiki/CirrusSearch: The article recommendations can be integrated with a custom KeywordFeature, e.g. CitolyticsKeywordFeature, that is trigged by the `citolytics:` prefix and retrieves the Citolytics recommendations. The `citolytics:` prefix then can be accessed via the API. T143197
* Android app: Citolytics needs to be added as additional read more source by allowing besides `morelike:` also `citolytics:` as query prefix.
* EventLogging: We want to use the tracking data from the Android app to evaluate the recommendation system and therefore we need to access aggregate data from the EventLogging system.
= Demo
* A MediaWiki setup that demonstrates the feature based on simplewiki can be found here: http://citolytics-demo.wmflabs.org/
* A guide for setting up the demo can be found here: https://github.com/mschwarzer/citolytics-demo/
* The Android app demo can be used as soon as the API changes are live.
= Goal Visibility & Success Metrics
* The Citolytics recommendations will be available in the Android app. See T148833.
* The Android app's event logging data should be also used to evaluate the success. See T149682
= Issues
* Deployment - How can we perform a test run and collect data to evaluate the recommendation performance?
* Evaluation data - What are the requirements for publishing data from the EventLogging system? The findings of this experiment should be as transparent as possible.
= Sub-tasks
# T142831 Review the current automatic recommendation mechanism (Lucene/MoreLikeThis)9682 Define success criteria
# T148833 Use Android Analytics framework for collecting click through-data
# T143197 Integrate additional RS (Citolytics) to CirrusSearch
# Evaluate how the [Click stream based evaluation](https://github.com/wikimedia/citolytics) measures change
**Issues**:
- citolytics currently works on article level. Figure out if there is clickstream data on section level
**Time frame** (deprecated)
- 20.10.2016 Finish implementation (ok)T149805 Clarify WMF involvement
- 05.11.2016 Start code review process
- 24.12.2016 finish data collection
- 24.01.2016 finish evaluation
We will update this ticket step by step.