Change Details

In {T234272}, we evaluated three different approaches for finding lists of articles around a given topic. For each of the three approaches, we did this by looking at 10 random articles from each topic (that were also newcomer tasks), and counting how many of those 10 seemed to fit in the topic. After doing this in four languages, we decided that the ORES text-based model was best. In {T240517}, we built that model into ElasticSearch and used it for the newcomer tasks feature. A problem with the text-based model is that separate models have to be trained for every language, which is a lot of human and machine time. Because of this, text-based models only exist in five languages right now, meaning low topic coverage in all other languages. Therefore, the Research team built the "link-based" model, which is easy to apply to every language and gives high coverage. Whereas the text-based model determines topics based on the words in an article, the link-based model looks at an article's wikilinks, and determines topics based on which other articles are linked. In this task, we are going to evaluate the new link-based model in five languages. Alongside it, we are going to re-evaluate the text-based model, to make sure we have an apples-to-apples comparison. If the link-based model performs strongly enough, we can start using the link-based model, and instantly be able to bring all Wikipedias to full topic coverage! Here's how we'll do it (hopefully, this will be a smoother process than last time): # [[ https://docs.google.com/spreadsheets/d/1fvd6xVdyK_lYkRmXDeEackBPQ13M2LOxxjkE1lFtrBc/edit#gid=1210083052 | Open this spreadsheet ]]. # Start on the "link-based" tab for your language. # Open each article and decide if it belongs to the topic listed in the "topic" column. There are ten random articles for each of the 64 topics. # In the "Correct?" column, put a 1 if the article belongs to that topic, and a 0 if it does not. # Go to the "text-based" tab for your language and repeat the process. When complete, we'll calculate accuracy for each model and decide how to proceed. [x] ar [X] cs [ ] en [x] fr [ ] vi

In {T234272}, we evaluated three different approaches for finding lists of articles around a given topic. For each of the three approaches, we did this by looking at 10 random articles from each topic (that were also newcomer tasks), and counting how many of those 10 seemed to fit in the topic. After doing this in four languages, we decided that the ORES text-based model was best. In {T240517}, we built that model into ElasticSearch and used it for the newcomer tasks feature. A problem with the text-based model is that separate models have to be trained for every language, which is a lot of human and machine time. Because of this, text-based models only exist in five languages right now, meaning low topic coverage in all other languages. Therefore, the Research team built the "link-based" model, which is easy to apply to every language and gives high coverage. Whereas the text-based model determines topics based on the words in an article, the link-based model looks at an article's wikilinks, and determines topics based on which other articles are linked. In this task, we are going to evaluate the new link-based model in five languages. Alongside it, we are going to re-evaluate the text-based model, to make sure we have an apples-to-apples comparison. If the link-based model performs strongly enough, we can start using the link-based model, and instantly be able to bring all Wikipedias to full topic coverage! Here's how we'll do it (hopefully, this will be a smoother process than last time): # [[ https://docs.google.com/spreadsheets/d/1fvd6xVdyK_lYkRmXDeEackBPQ13M2LOxxjkE1lFtrBc/edit#gid=1210083052 | Open this spreadsheet ]]. # Start on the "link-based" tab for your language. # Open each article and decide if it belongs to the topic listed in the "topic" column. There are ten random articles for each of the 64 topics. # In the "Correct?" column, put a 1 if the article belongs to that topic, and a 0 if it does not. # Go to the "text-based" tab for your language and repeat the process. When complete, we'll calculate accuracy for each model and decide how to proceed. [x] ar [X] cs [ ] en [x] fr [x] vi

In {T234272}, we evaluated three different approaches for finding lists of articles around a given topic. For each of the three approaches, we did this by looking at 10 random articles from each topic (that were also newcomer tasks), and counting how many of those 10 seemed to fit in the topic. After doing this in four languages, we decided that the ORES text-based model was best. In {T240517}, we built that model into ElasticSearch and used it for the newcomer tasks feature. A problem with the text-based model is that separate models have to be trained for every language, which is a lot of human and machine time. Because of this, text-based models only exist in five languages right now, meaning low topic coverage in all other languages. Therefore, the Research team built the "link-based" model, which is easy to apply to every language and gives high coverage. Whereas the text-based model determines topics based on the words in an article, the link-based model looks at an article's wikilinks, and determines topics based on which other articles are linked. In this task, we are going to evaluate the new link-based model in five languages. Alongside it, we are going to re-evaluate the text-based model, to make sure we have an apples-to-apples comparison. If the link-based model performs strongly enough, we can start using the link-based model, and instantly be able to bring all Wikipedias to full topic coverage! Here's how we'll do it (hopefully, this will be a smoother process than last time): # [[ https://docs.google.com/spreadsheets/d/1fvd6xVdyK_lYkRmXDeEackBPQ13M2LOxxjkE1lFtrBc/edit#gid=1210083052 | Open this spreadsheet ]]. # Start on the "link-based" tab for your language. # Open each article and decide if it belongs to the topic listed in the "topic" column. There are ten random articles for each of the 64 topics. # In the "Correct?" column, put a 1 if the article belongs to that topic, and a 0 if it does not. # Go to the "text-based" tab for your language and repeat the process. When complete, we'll calculate accuracy for each model and decide how to proceed. [x] ar [X] cs [ ] en [x] fr [ x] vi