Update:

I summarize the article types to extract into four categories based on their source data, and put the results at the end.

1. Page/Inline Templates
   • Idea: find articles that contains certain page/inline templates in their content.
   • Source data: template list, wmf_content.mediawiki_content_current_v1 table
   • There are some enwiki maintenance template ideas listed as page templates in this spreadsheet. But after checking them out, I found most are inline templates (e.g. {{Compared to?}},{{Editorializing}},{{Fact or opinion}} {{Opinion}},{{POV statement}).
   • Most page and inline templates listed do not have corresponding templates in Czech Wikipedia (cswiki). I only found this template {{Šablona:Vyhýbavá_slova}} which is similar. Its page also mentions related page and inline templates, so I ended up using these templates for cswiki.

2. Article Topics
   • Idea: find articles with the topics (Culture.Biography, Culture.Sports, Culture.Media.Media*, History_and_Society.Business_and_economics, History_and_Society.Politics_and_government) and probability > 0.5.
   • Source data: research.article_topics table
   • In the results, the Culture.Media.Media* topic contains the second-highest count. This topic isn't just about businesses; it is a catch all for all other items that appear within Media. E.g. Media.Book, Media.Entertainment, Media.Films, etc. It also includes any items that are generally related to Media that don't fit into the specific categories. (see https://www.mediawiki.org/wiki/ORES/Articletopic)

3. Article edited by new editors
   • Idea: find articles edited by users who registered within the last 3 months ('2025-06-01' to '2025-09-01').
   • Source data: mediawiki_history table (which has the "event_user_registration_timestamp" field)

4. Articles with relatively few pageviews (WIP)
   • Idea: find articles with fewer than 1000 total pageviews
   • Source data: should be https://wikitech.wikimedia.org/wiki/Data_Platform/Data_Lake/Traffic/Pageviews. Seems only contains hourly data, then we need to aggregate data.

Results:

1. Page/Inline Templates

+-------+------------------+-----+
|wiki_id|template          |count|
+-------+------------------+-----+
|enwiki |tone              |5343 |
|enwiki |peacock           |770  |
|enwiki |weasel            |506  |
|enwiki |buzzword inline   |441  |
|enwiki |weasel inline     |386  |
|enwiki |tone inline       |379  |
|enwiki |opinion           |312  |
|enwiki |pov statement     |245  |
|enwiki |editorializing    |216  |
|enwiki |peacock inline    |176  |
|enwiki |technical inline  |157  |
|enwiki |promotion inline  |95   |
|enwiki |fact or opinion   |80   |
|enwiki |unbalanced opinion|55   |
|enwiki |compared to?      |38   |
|enwiki |neologism inline  |7    |
|enwiki |idiom             |2    |
|enwiki |colloquialism     |1    |
+-------+------------------+-----+

+-------+-------------------+-----+
|wiki_id|template           |count|
+-------+-------------------+-----+
|cswiki |neověřeno          |12026|
|cswiki |neověřeno část     |840  |
|cswiki |ujasnit            |656  |
|cswiki |vlastní výzkum     |176  |
|cswiki |celkově zpochybněno|110  |
|cswiki |vyhýbavá slova     |16   |
+-------+-------------------+-----+

2. Article Topics

+-------+-------------------------------------------+-------+
|wiki_db|topic                                      |count  |
+-------+-------------------------------------------+-------+
|enwiki |Culture.Biography.Biography*               |1394816|
|enwiki |Culture.Media.Media*                       |1019480|
|enwiki |Culture.Sports                             |956776 |
|enwiki |History_and_Society.Politics_and_government|246764 |
|cswiki |Culture.Biography.Biography*               |131752 |
|enwiki |History_and_Society.Business_and_economics |104479 |
|cswiki |Culture.Sports                             |67564  |
|cswiki |Culture.Media.Media*                       |62480  |
|enwiki |Culture.Biography.Women                    |28913  |
|cswiki |History_and_Society.Politics_and_government|11151  |
|cswiki |History_and_Society.Business_and_economics |7669   |
|cswiki |Culture.Biography.Women                    |1639   |
+-------+-------------------------------------------+-------+

3. Article edited by new editors

+-------+-----+
|wiki_db|count|
+-------+-----+
|enwiki |31151|
|cswiki |2697 |
+-------+-----+

4. Articles with relatively few pageviews (WIP)
   • Idea: find articles with fewer than 1000 total pageviews
   • Source data: should be https://wikitech.wikimedia.org/wiki/Data_Platform/Data_Lake/Traffic/Pageviews. Seems only contains hourly data, then we need to aggregate data.

Include the following metadata about each paragraph:
A. [If possible] Number of pageviews to the article (all-time)
B. Number of pageviews to the article (last 90 days)

@diego, do you know if there is any Hive table in Data Lake that contains aggregate pageviews per article? Would really appreciate your insights on this :)

In T401968#11146527, @achou wrote:

@diego, do you know if there is any Hive table in Data Lake that contains aggregate pageviews per article? Would really appreciate your insights on this :)

You could use the pageview-hourly table, which contains aggregated number of pageviews per article (use the page_id to also get pageviews from redirects). You would only need to aggregate over a corresponding time window. (I am not Diego, but thought I would chime in when I saw the update). Hope this helps.

Thanks @MGerlach , I agree that is the main source of pageviews readable from Spark. I'm not aware of anything aggregated in larger buckets.

There is also a pageviews_daily (significantly smaller than the hourly) but it is just available in turnilo and think it doesn't have article level information.

The knowledge gap pipeline (which is snapshot based) aggregates the historical pageviews too, as the pageview-hourly dataset is sizable for larger time ranges. The knowledge_gaps.pageviews_daily contains a subset of pages (wikipedia, namespace 0 , agent_type=user) has a minimal schema and partitioned by date (i.e. day), one day of data is about is ~4GB of data compared to ~25GB for pageviews-hourly. The code is here. The dag runs weekly.

@MGerlach @diego @fkaelin Wow thank you for all your input! <3 I checked out the knowledge_gaps.pageviews_daily table and it's super easy to use :D

Data collection is complete and this spreadsheet has been updated.

Summary:

• Estimated number of structured tasks we can generate: 49,327 (enwiki; large wiki); 1,851 (cswiki; small wiki)
  • For enwiki, articles about people contribute the most estimated tasks (~28k), followed by articles about businesses (~10k)
  • For cswiki, while the number of positive paragraphs exceeds enwiki in sample articles, paragraphs with probability ≥ 0.8 are much fewer, sometimes none. Lowering the threshold to 0.7 would yield much more tasks. For example, we'd have 13,339 estimated tasks with probability ≥ 0.7 in articles about people.
• Some article types received no positive predictions with a probability score ≥ 0.8 in our random sample of 50 articles. This suggest the sample size may be too small for certain article types. For example, among 956,776 articles about sports in English, our sample of 50 articles gives 0 positive predictions with a probability score ≥ 0.8. We should try larger sample sets to verify this.
• The quality of structured tasks generated needs evaluation
  • I've added sample article paragraphs that received positive predictions with high probability scores, including metadata about each article, to the EN/CS tabs in the spreadsheet.

In T401968#11161603, @achou wrote:

• For cswiki, while the number of positive paragraphs exceeds enwiki in sample articles, paragraphs with probability ≥ 0.8 are much fewer, sometimes none. Lowering the threshold to 0.7 would yield much more tasks. For example, we'd have 13,339 estimated tasks with probability ≥ 0.7 in articles about people.

In T401968#11161638, @achou wrote:

In T401968#11161603, @achou wrote:

• For cswiki, while the number of positive paragraphs exceeds enwiki in sample articles, paragraphs with probability ≥ 0.8 are much fewer, sometimes none. Lowering the threshold to 0.7 would yield much more tasks. For example, we'd have 13,339 estimated tasks with probability ≥ 0.7 in articles about people.

A comparison between enwiki vs cswiki probability score distribution (from T398930) may explain this phenomenon? The cswiki's distribution appears denser than enwiki, with scores centering more between the probability range of 0.4-0.6.

Oh that is a fascinating graph, thank you! If(!) we assume that the model is at least directionally accurate, does this graph then imply that Czech Wikipedia is written in a subtly less neutral tone than English Wikipedia? Curious if anyone from Research has thoughts on this.

I'd also be curious if @Urbanecm_WMF or other native czech speakers have a gut-reaction to that. Does English Wikipedia feel like it has more "neutral" tone than Czech Wikipedia?

Update:

• Using a larger sample size (~1k)
  • EN: 46,910 total estimated tasks, similar to our previous estimates. Tasks in articles about sports increased (0 to 9,244), while articles about businesses decreased (10,448 to 3,913). Articles with templates and articles about people remained stable.
  • CS: Estimated numbers increased across all categories, raising the total tasks from 1,851 to 4,055. Lowering the threshold to 0.7 would increase this to 30,508 tasks.
  • Detailed category numbers can be found in the "Counts - larger sample size" tab in the spreadsheet.
• Additional sample article paragraphs can be found in the "EN - more samples" and "CS - more samples" tabs in the spreadsheet.