Page MenuHomePhabricator

[Spike] Generate Filters and Filter list of articles for summary experiment
Closed, ResolvedPublic3 Estimated Story Points

Description

Background

  • The legal review of our summary experiment required a number of articles to be filtered from our full list.

User story

As a product team, we would like to filter certain categories of articles, so that our experiment plan can pass legal review

Requirements

  • Explore how much filtering we can do of articles in the following categories: articles about living children, medical articles, and articles about politicians
  • Share agreed-on filtering criteria prior to generating summaries
  • Share the list of articles prior to generating summaries - With some back of the envelope estimates of how many articles are in the following categories:
  • Biographies of living persons (BLP)
  • Articles on regulated topics (children, crime, suicide, terrorism)
  • Articles of the kind that might contain personal information (politics [political affiliation], sexuality ["list of notable bisexuals")

Communication criteria - does this need an announcement or discussion?

  • Deliver outcome to legal team

This task was created by Version 1.2.0 of the Web team task template using phabulous

Event Timeline

A possible approach:

@Jdlrobson my original thought to use Wikidata to find entities based on article title, then check if they're human, alive, etc. However, I think the categories approach scales better because it allows us to filter for more arbitrary categories or even parts of categories, like identify anything with a category that matches politic* medic* list of * etc. Not sure about living children, we might have to do some date math for that, but I think your suggested approach gets us most of the way there.

It's going to be a long regular expression. Personally I think we'd need "expanded categories" for every page.

For example, https://en.wikipedia.org/wiki/Nico_Parker is in https://en.wikipedia.org/wiki/Category:English_child_actresses
That category is a subcategory of https://en.wikipedia.org/wiki/Category:British_child_actresses
which is a subcategory of https://en.wikipedia.org/wiki/Category:British_child_actors
which is a subcategory of https://en.wikipedia.org/wiki/Category:European_child_actors
which is a subcategory of https://en.wikipedia.org/wiki/Category:Child_actors_by_continent
which is a subcategory of https://en.wikipedia.org/wiki/Category:Child_actors

It would follow that you should be able to list all articles which have the category 'Child_actors' or a child of that category.

However hitting the API
https://en.wikipedia.org/wiki/Special:ApiSandbox#action=query&format=json&list=categorymembers&formatversion=2&cmtitle=Category%3AChild_actors&cmnamespace=0 the only members are https://en.wikipedia.org/wiki/Child_actor and https://en.wikipedia.org/wiki/Lists_of_child_actors

Would be great if this API allowed you to include pages that were members of subcategories.

... but instead you need to hit the API for every subcategory:
https://en.wikipedia.org/wiki/Special:ApiSandbox#action=query&format=json&list=categorymembers&formatversion=2&cmtitle=Category%3AChild_actors&cmnamespace=14

Hitting the API for every subcategory of every category of every page seems a bit excessive. So far I've added just all categories for every page (not subcategories).

I've put together the following spreadsheet with top-level categories.

https://docs.google.com/spreadsheets/d/1knzKyZT2vCEjCqDjIHEA5MAgSWPG-4CWaK8W9BDhu_0

I haven't done "regulated topics" yet, for the "children" category I just used the year-of-birth categories e.g. "2006 births" combined with the "Living people" category.

So far I have:

  • Living people: 1780
  • Living politicians: 148
  • Living children (born after 2005): 5
Jdrewniak subscribed.

@ovasileva I'm assigning this to you to look through the existing filters and titles in this spreadsheet:
https://docs.google.com/spreadsheets/d/1knzKyZT2vCEjCqDjIHEA5MAgSWPG-4CWaK8W9BDhu_0
Living people, politician & children filters are set up. I don't know how well we can target other things via categories, but we can add more filters if you spot anything more that should be excluded.

We have filtered the list of articles and excluded all living persons and sensitive topics to the best of our abilities:

https://docs.google.com/spreadsheets/d/1knzKyZT2vCEjCqDjIHEA5MAgSWPG-4CWaK8W9BDhu_0/edit?gid=1019355060#gid=1019355060

This exclusion list has been committed to the summary generation repo:

https://gitlab.wikimedia.org/repos/web/simple-summary-server/-/blob/main/excluded_titles.csv?ref_type=heads

And will be checked when summaries are generated and exported.

I think we're all done here! Additional filtering can happen in a new ticket if necessary, or directly in the doc