Came across a semi-recent paper proposing generation of relevance predictions from click data using neural networks. The findings show a respectable improvement over DBN (which we use today) for ndcg@1 and ndcg@3. We have all the necessary input data to train this model, it would be nice to build it out and evaluate it's performance generating labels for our MLR pipeline.
This task can be done without an NDA, although an engineer under NDA will have to first prepare the data. The prepared data would
be aggregated click counts and contain no query strings. There is a patch attached to this ticket that can generate data files from the click data. This task will require a reasonable amount of compute, so may require access to wmf analytics compute resources.