Page MenuHomePhabricator
Create Task
Maniphest T347832

Search dag image_suggestions_weekly failed waiting for analytics_platform_eng.image_suggestions_search_index_delta/snapshot=2023-09-18
Closed, ResolvedPublic
Actions

Description

The airflow dag responsible for pushing weighted tags for image suggestions failed on 2023/10/02 waiting for the dataset analytics_platform_eng.image_suggestions_search_index_delta/snapshot=2023-09-18 which does not exist yet.

Related Objects

Event Timeline

• dcausse created this task.Oct 2 2023, 8:28 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 2 2023, 8:28 AM
mfossati added a comment.Oct 2 2023, 9:00 AM

The wmf.wikidata_item_page_link/snapshot=2023-09-18 upstream dependency isn’t available yet.
Corresponding DAG sensors timed out, causing all DAGs to fail.

Notified Experimentation Lab, will backfill as soon as the snapshot comes back. I’ll avoid skipping runs this time.

Gehel moved this task from needs triage to Current work on the Discovery-Search board.Oct 2 2023, 3:37 PM
Gehel edited projects, added Discovery-Search (Current work); removed Discovery-Search.
Gehel moved this task from Incoming to Blocked/Waiting on the Discovery-Search (Current work) board.
KStoller-WMF subscribed.Oct 2 2023, 3:54 PM
mfossati added a comment.Oct 3 2023, 9:15 AM

The upstream dependency is back, so I manually triggered all data pipelines.
Section topics and section alignment suggestions went fine, image suggestions running now.

mfossati added a comment.Oct 3 2023, 5:22 PM

All data pipelines succeeded.
The required dataset is now available, row counts of ALIS, SLIS and the delta are all consistent with the previous run.

• dcausse added a comment.Oct 4 2023, 8:49 AM

cleared the dag on our side, suggestions should start being populated.

• dcausse moved this task from Blocked/Waiting to Needs Reporting on the Discovery-Search (Current work) board.Oct 9 2023, 3:07 PM
Gehel closed this task as Resolved.Oct 13 2023, 1:03 PM
Gehel claimed this task.
Gehel mentioned this in T349472: Search dag image_suggestions_weekly failed waiting for analytics_platform_eng.image_suggestions_search_index_delta/snapshot=2023-10-09.Nov 3 2023, 9:10 AM
bking mentioned this in T350499: Search Platform Airflow jobs: Identify dependencies and configure alerts.Nov 3 2023, 4:50 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits