Update week 2020-07-27:

• updated pipeline to generate reading sessions from actor-pageview table; this approach is more efficient (pre-filtered table is much smaller) such that models can be updated more easily in the future, it also provides a more consistent way to define unique devices
• repo https://github.com/martingerlach/reader-embedding-api/tree/master/code

Update week 2020-08-03:

• build pipeline to tune hyperparameters (splitting train-test, prediction and evaluation on testset)
• need to experiment with smaller sample datasets (eg. smaller wikis) to perform extensive grid search)

Update weel 2020-08-10:

• familiarized myself with cloudservice on horizon to host model with larger memory requirements
• adapted isaacs template for setup https://github.com/wikimedia/research-api-endpoint-template
• was able to setup simple case to query similar wikidata-items from an embedding of reading sessions (modelsize ~1GB which i couldnt host on toolforge due to memory-limitations),
  • • example query for Q81068910: covid-pandemic
  • yields most similar items: https://reader.wmcloud.org/api/v1/reader?qid=Q81068910

Update weel 2020-08-17:

• most time spent on building supplementary datasets in order to use as additional features for modeling reading-sessions, specifically relation of articles based on link-network and text; as well as position of links in articles

Update week 2020-08-24:

• None

Update week 2020-09-01:

• started to build a slightly more sophisticated tool on the toolforge-instance to explore related articles based on reader sessions:
  • example query: https://reader.toolforge.org/api/v1/reader/nn?qid=Q81068910&lang=en|de|pt&n=100&showurl=True
  • this reveals 100 related items to covid showing whether articles exist in the relevant languages (here: enwiki, dewiki, ptwiki); and adding the urls for easier exploration
  • more detailed documenation on usage: https://github.com/martingerlach/reader-embedding-api
  • this instance essentially just points to a cloud-vps endpoint (https://github.com/martingerlach/reader-embedding-api-endpoint) which is hosting the actual model (due to the size of the model it is not possible to host this on toolforge); in this way it is possible to potentially add improvements to the visualization by just interacting with the simpler toolforge instance

Update week 2020-09-07:

• None (worked on link-recommendation)

Update week 2020-09-14:

• performed thorough analysis of hyperparameters to fine-tune performance of models. proper choice of hyperparameters is crucial as there is strong dependence of performance with particular choice. in addition we have to take into account constraints with respect to the resulting model size which puts limit on, e.g., number of dimensions when hosting the model on a cloud-vps instance
• started to write up documentation to be moved to meta later https://docs.google.com/document/d/1LsqdacaUnsZhoBKj0mOSEL5Bsuf_HThASXN3O9bpIl0/edit#
  • pre-processing and training model
  • API-hosting
  • in-depth analysis on hyperparameters/convergence/etc.

Update week 2020-09-21:

• updated model to do additional filtering
• planning next steps for analysis in coming quarter
  • evaluation (e.g. coverage for different wikis,)
  • capture basic properties (how related to similar model based,e.g. the links-model, how dynamic in time, )
  • how to serve different use-cases such as list-building