Page MenuHomePhabricator
Create Task
Maniphest T324582

Breaking News {Content Integrity}
Open, HighPublic
Actions

Description

TL;DR — Using a spike detection model to detect breaking news articles on Wikipedia.

Our objective is to build into our API schema, a Breaking News boolean signal as part of our Credibility Signals feature, that can reveal in payload — particularly for users of our Realtime: Stream API — whether editing activity on Wikipedia indicates the writing of entries concurrent with breaking news happening around the world.

Breaking News as an anomaly signal is a category. Because of the nature of breaking news and Wikipedia, explained above, BN Credibility Signal will grow and change. At first, this will operate as a 24h boolean for new entries.

Done is

KPIs

  • all "big unforeseen" news in English.
  • lower ingestion latency of new entries for reusers.

Related Objects
Search...

StatusSubtypeAssignedTask
 OpenNoneT324582 Breaking News {Content Integrity}
 ResolvedAlexander.lauieT321148 Investigation: Credibility Signals — BreakingNews PoC
 ResolvedprabhatT322260 Create stream listener handler for Breaking News PoC
 ResolvedProtsack.stephanT322259 Create an API endpoint for Breaking News PoC
 ResolvedProtsack.stephanT322257 Create simple UI for Breaking News PoC
 ResolvedBUG REPORTLDlulisa-WMFT328020 Breaking news PoC failing to load
 ResolvedBUG REPORTLDlulisa-WMFT329797 Refresh BreakingNews PoC site UI to make results legible
 InvalidNoneT329859 [STUB] Tweak breaking news PoC to increase result accuracy
 ResolvedNoneT329861 Bring Breaking News into beta/staging area
 DeclinedNoneT330798 [STUB] Store of BreakingNews boolean true results for accuracy testing
 ResolvedAlexander.lauieT331465 Prepare POC and UI for iteration
 ResolvedREsquito-WMFT331666 [investigation] Productization of Breaking News (Time box ~ 8 day(s))
 ResolvedAlexander.lauieT334164 Create "Page Move" handler in event-bridge service
 ResolvedTim.abdullinT334167 Add "Page Move" handler to IaC
ResolvedprabhatT334165 Create "Page Create" handler in event-bridge service
 ResolvedLDlulisa-WMFT334169 Add "Page Create" handler to IaC
 ResolvedREsquito-WMFT334574 Add Content Integrity service to the IaC
 ResolvedTim.abdullinT336283 Add workaround to regenerate kafka config archive on apply
 ResolvedREsquito-WMFT335503 Deploy Breaking news to UAT environment
 ResolvedTim.abdullinT335952 Create Dashboard for Content-Integrity Redis
 ResolvedAnribolonT334170 Add Redis cluster for content integrity to IaC
 ResolvedTim.abdullinT334576 Add data listener to the Content Integrity service
ResolvedREsquito-WMFT334593 Add "Get Integrity" API endpoint to Content Integrity service
 ResolvedprabhatT334978 Update services to propagate breaking news data
 ResolvedLDlulisa-WMFT334981 Update website documentation with breaking news fields
 OpenNoneT336333 Create Breaking News Runbook
 ResolvedprabhatT344726 Assign wmcs user to group_3
 ResolvedProtsack.stephanT339206 Breaking news production release
 ResolvedFNavas-foundationT342255 "Turn on" productionized breaking news feature
ResolvedJArguello-WMFT334052 Content Integrity Big Data Analysis
 ResolvedprabhatT330787 [investigation] Template usage metrics for BreakingNews iteration and signal quality (timebox ~ 10 day(s))
OpenNoneT334017 Split frequency distribution table into two parts: with duration > 1 day and < 1 day
 ResolvedNoneT334018 Optimize data processing & collection by piping spark query to hdfs
OpenNoneT334020 Investigate process to recollect the template usage data
 ResolvedprabhatT343357 Fix CI pipeline for npm-based project and deploy BCN UI
 ResolvedprabhatT344340 Deploy breaking news public demo to production
 OpenNoneT344756 Breaking News Public Demo Feedback, Questions and Technical Issues
 ResolvedprabhatT341741 Create public demo version of Enterprise Breaking News PoC
 ResolvedLDlulisa-WMFT342568 Add reaction option to public Breaking news Demo
 OpenNoneT343569 Set up grafana dashboards for breaking news demo reactions (Q2)
 ResolvedNoneT343742 Dark Launch Production Breaking News
 OpenNoneT353756 Exclude templates from breaking news algo
 OpenNoneT355335 Prioritize current year in breaking news
 OpenNoneT357883 Create special "include" list for breaking news algo

Event Timeline

FNavas-foundation created this task.Dec 6 2022, 4:11 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 6 2022, 4:11 PM
FNavas-foundation renamed this task from Breaking News Detection to Breaking News.Dec 6 2022, 4:11 PM
FNavas-foundation added a project: Epic.
Izno subscribed.Dec 6 2022, 8:55 PM
AnnaMikla added a subtask: T321148: Investigation: Credibility Signals — BreakingNews PoC.Dec 6 2022, 10:05 PM
FNavas-foundation added a subtask: T328588: TESSST FRANCISCO.Feb 1 2023, 7:07 PM
FNavas-foundation removed a subtask: T328588: TESSST FRANCISCO.
FNavas-foundation added a subtask: T328588: TESSST FRANCISCO.
FNavas-foundation closed subtask T328588: TESSST FRANCISCO as Invalid.Feb 1 2023, 7:14 PM
FNavas-foundation added a subtask: T328020: Breaking news PoC failing to load.Feb 15 2023, 9:31 PM
FNavas-foundation added a subtask: T329797: Refresh BreakingNews PoC site UI to make results legible.Feb 15 2023, 9:40 PM
FNavas-foundation added a subtask: T329859: [STUB] Tweak breaking news PoC to increase result accuracy.Feb 16 2023, 4:41 PM
FNavas-foundation renamed this task from Breaking News to Breaking News — Credibility Signals.Feb 16 2023, 4:51 PM
FNavas-foundation added a subtask: T329861: Bring Breaking News into beta/staging area.Feb 16 2023, 4:55 PM
FNavas-foundation updated the task description. (Show Details)Feb 22 2023, 9:47 PM
FNavas-foundation added a subtask: T330787: [investigation] Template usage metrics for BreakingNews iteration and signal quality (timebox ~ 10 day(s)) .Feb 28 2023, 7:43 PM
FNavas-foundation added a subtask: T330798: [STUB] Store of BreakingNews boolean true results for accuracy testing.Feb 28 2023, 9:52 PM
SDelbecque-WMF renamed this task from Breaking News — Credibility Signals to {Credibility Signals} Breaking News.Mar 1 2023, 2:12 PM
FNavas-foundation added a subtask: T331465: Prepare POC and UI for iteration.Mar 7 2023, 6:02 PM
FNavas-foundation added a subtask: T331666: [investigation] Productization of Breaking News (Time box ~ 8 day(s)).Mar 9 2023, 8:01 PM
Aklapper removed a subtask: T328588: TESSST FRANCISCO.Mar 27 2023, 8:54 AM
Aklapper added a project: Wikimedia Enterprise-Breaking-News.
REsquito-WMF added a subtask: T334164: Create "Page Move" handler in event-bridge service.Apr 6 2023, 7:11 AM
REsquito-WMF added a subtask: T334165: Create "Page Create" handler in event-bridge service.Apr 6 2023, 7:13 AM
REsquito-WMF added a subtask: T334167: Add "Page Move" handler to IaC .Apr 6 2023, 7:19 AM
REsquito-WMF added a subtask: T334169: Add "Page Create" handler to IaC.
REsquito-WMF added a subtask: T334170: Add Redis cluster for content integrity to IaC.Apr 6 2023, 7:22 AM
Tim.abdullin changed the status of subtask T334167: Add "Page Move" handler to IaC from Open to In Progress.Apr 10 2023, 12:30 PM
prabhat changed the status of subtask T334165: Create "Page Create" handler in event-bridge service from Open to In Progress.Apr 11 2023, 1:23 PM
Protsack.stephan added a subtask: T334052: Content Integrity Big Data Analysis.Apr 11 2023, 5:22 PM
Protsack.stephan removed a subtask: T330787: [investigation] Template usage metrics for BreakingNews iteration and signal quality (timebox ~ 10 day(s)) .
Protsack.stephan removed a subtask: T334164: Create "Page Move" handler in event-bridge service.Apr 11 2023, 6:05 PM
Protsack.stephan removed a subtask: T334165: Create "Page Create" handler in event-bridge service.
Protsack.stephan removed a subtask: T334169: Add "Page Create" handler to IaC.Apr 11 2023, 6:08 PM
Protsack.stephan removed a subtask: T334167: Add "Page Move" handler to IaC .
Protsack.stephan removed a subtask: T334170: Add Redis cluster for content integrity to IaC.
Protsack.stephan closed subtask T331666: [investigation] Productization of Breaking News (Time box ~ 8 day(s)) as Resolved.Jul 11 2023, 3:20 PM
Protsack.stephan closed subtask T331465: Prepare POC and UI for iteration as Resolved.
Protsack.stephan closed subtask T321148: Investigation: Credibility Signals — BreakingNews PoC as Resolved.
Protsack.stephan closed subtask T328020: Breaking news PoC failing to load as Resolved.Jul 11 2023, 3:22 PM
Protsack.stephan closed subtask T329797: Refresh BreakingNews PoC site UI to make results legible as Resolved.
JArguello-WMF closed subtask T329859: [STUB] Tweak breaking news PoC to increase result accuracy as Invalid.Jul 24 2023, 7:17 PM
JArguello-WMF closed subtask T329861: Bring Breaking News into beta/staging area as Resolved.
JArguello-WMF added a subtask: T342568: Add reaction option to public Breaking news Demo.
JArguello-WMF closed subtask T330798: [STUB] Store of BreakingNews boolean true results for accuracy testing as Declined.
JArguello-WMF renamed this task from {Credibility Signals} Breaking News to Breaking News.Jul 27 2023, 4:28 AM
JArguello-WMF added a project: Wikimedia Enterprise - Content Integrity.
JArguello-WMF updated the task description. (Show Details)Aug 7 2023, 4:50 PM
JArguello-WMF mentioned this in T343742: Dark Launch Production Breaking News.Aug 11 2023, 7:34 PM
JArguello-WMF added a subtask: T343357: Fix CI pipeline for npm-based project and deploy BCN UI.Aug 15 2023, 4:06 PM
JArguello-WMF updated the task description. (Show Details)Aug 15 2023, 4:23 PM
JArguello-WMF changed the status of subtask T342568: Add reaction option to public Breaking news Demo from Open to In Progress.
creynolds subscribed.Aug 18 2023, 2:02 AM
JArguello-WMF closed subtask T343357: Fix CI pipeline for npm-based project and deploy BCN UI as Resolved.Aug 22 2023, 1:16 PM
JArguello-WMF added a subtask: T344340: Deploy breaking news public demo to production.Aug 22 2023, 3:13 PM
JArguello-WMF closed subtask T342568: Add reaction option to public Breaking news Demo as Resolved.Aug 22 2023, 3:15 PM
JArguello-WMF updated the task description. (Show Details)Aug 22 2023, 3:21 PM
JArguello-WMF added a subtask: T341741: Create public demo version of Enterprise Breaking News PoC.Aug 22 2023, 3:46 PM
JArguello-WMF triaged this task as High priority.Aug 22 2023, 5:54 PM
JArguello-WMF updated the task description. (Show Details)Sep 4 2023, 1:15 PM
JArguello-WMF closed subtask T344340: Deploy breaking news public demo to production as Resolved.Sep 5 2023, 1:14 PM
FNavas-foundation renamed this task from Breaking News to {Content Integrity} Breaking News.Sep 8 2023, 4:01 PM
JArguello-WMF updated the task description. (Show Details)Sep 20 2023, 12:30 PM
FNavas-foundation added a subtask: T353756: Exclude templates from breaking news algo.Dec 21 2023, 3:43 PM
FNavas-foundation added a subtask: T355335: Prioritize current year in breaking news.Jan 18 2024, 3:17 PM
FNavas-foundation renamed this task from {Content Integrity} Breaking News to Breaking News {Content Integrity}.Jan 24 2024, 9:28 PM
FNavas-foundation added a subtask: T357883: Create special "include" list for breaking news algo.Feb 19 2024, 10:05 AM
JArguello-WMF closed subtask T334052: Content Integrity Big Data Analysis as Resolved.Apr 9 2024, 1:26 PM
JArguello-WMF moved this task from Roadmap (Initiatives Q1-Q2 FY25-26) to Content Integrity PB on the Wikimedia Enterprise board.Apr 9 2024, 2:53 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits