Page MenuHomePhabricator

recommendation API server fails to fill cache
Closed, ResolvedPublicBUG REPORT

Description

recommendation API server fails to fill cache with the following error. This is after we marked page like https://meta.wikimedia.org/wiki/Wikiproject_Women%27s_Health/Vital_articles with page-collection marker

DEBUG : recommendation : fetcher : get : GET: https://www.wikidata.org/w/api.php?action=wbgetentities&format=json&props=sitelinks&sites=zh-min-nanwiki&titles=Bah-l%C5%ABi|B%C3%A1-tah|B%C3%AC-luh|Cheng-li%C3%BB-chi%C3%BA|Chh%C3%A0i-se|Chh%C3%AC-juh|Chh%C3%B2%CD%98|Chi-ku-la%CC%8Dt|Chi%C3%BA|Hu%CC%8Dt-th%C3%B4|I%C3%A2m|Ka-pi|Kh%C3%AC-ch%C3%BAi|K%C3%A1u-ch%C3%A0n-thah|Leng|M%C3%A1-l%C3%AAng-ch%C3%AE|Nn%CC%84g_%28chia%CC%8Dh-mi%CC%8Dh%29|Pasta|Phang-bi%CC%8Dt|Ph%C3%A1ng|Ph%C3%B4-t%C3%B4|Ph%C3%B4-t%C3%B4-chi%C3%BA|Rum|Si%CC%8Dt-i%C5%8Dng-i%C3%BB|Thn%CC%82g|T%C3%AA|%C3%8Dm-li%C4%81u&formatversion=2, {'user-agent': 'WMF Recommendation API (https://recommend.wmcloud.org/; leila@wikimedia.org)', 'Host': 'www.wikidata.org'}
ERROR : recommendation : fetcher : get_articles_by_titles : Error fetching articles by QIDs: {'code': 'param-missing', 'info': 'Either provide the Item "ids" or pairs of "sites" and "titles" for corresponding page', 'messages': [{'name': 'wikibase-api-illegal-ids-or-sites-titles-selector', 'parameters': [], 'html': 'Either provide the Item "ids" or pairs of "sites" and "titles" for corresponding page'}], 'docref': 'See https://www.wikidata.org/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/postorius/lists/mediawiki-api-announce.lists.wikimedia.org/> for notice of APIdeprecations and breaking changes.'}
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/opt/work/recommendation-api/recommendation/utils/cache_updater.py", line 89, in start
asyncio.run(update_page_collection_cache())
File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/opt/work/recommendation-api/recommendation/utils/cache_updater.py", line 80, in update_page_collection_cache
await live_page_collection.fetch_articles()
File "/opt/work/recommendation-api/recommendation/api/translation/models.py", line 187, in fetch_articles
results = await asyncio.gather(*tasks)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/work/recommendation-api/recommendation/external_data/fetcher.py", line 603, in get_candidates_in_collection_page
wikidata_articles.extend(articles)
TypeError: 'NoneType' object is not iterable

The following API access fails:

https://www.wikidata.org/w/api.php?action=wbgetentities&format=json&props=sitelinks&sites=zh-min-nanwiki&titles=Bah-l%C5%ABi|B%C3%A1-tah|B%C3%AC-luh|Cheng-li%C3%BB-chi%C3%BA|Chh%C3%A0i-se|Chh%C3%AC-juh|Chh%C3%B2%CD%98|Chi-ku-la%CC%8Dt|Chi%C3%BA|Hu%CC%8Dt-th%C3%B4|I%C3%A2m|Ka-pi|Kh%C3%AC-ch%C3%BAi|K%C3%A1u-ch%C3%A0n-thah|Leng|M%C3%A1-l%C3%AAng-ch%C3%AE|Nn%CC%84g_%28chia%CC%8Dh-mi%CC%8Dh%29|Pasta|Phang-bi%CC%8Dt|Ph%C3%A1ng|Ph%C3%B4-t%C3%B4|Ph%C3%B4-t%C3%B4-chi%C3%BA|Rum|Si%CC%8Dt-i%C5%8Dng-i%C3%BB|Thn%CC%82g|T%C3%AA|%C3%8Dm-li%C4%81u&formatversion=2

Event Timeline

+ Recommendation-API (please add project tags so other people who don't know or don't care about team tags can also find tasks. Thanks!)

Change #1098068 had a related patch set uploaded (by Sbisson; author: Sbisson):

[research/recommendation-api@master] Return [] instead of None or "" when an array is expected

https://gerrit.wikimedia.org/r/1098068

Change #1098090 had a related patch set uploaded (by Sbisson; author: Sbisson):

[research/recommendation-api@master] Minimal fix for zh-min-nan site name

https://gerrit.wikimedia.org/r/1098090

Change #1098115 had a related patch set uploaded (by Sbisson; author: Sbisson):

[research/recommendation-api@master] Cache update: skip iw links already discovered through wikidata

https://gerrit.wikimedia.org/r/1098115

ProblemSolutionChanges
Unhandled exception crashing the service during the cache update processAlways return [] when the function signature says sogerrit:1098090
Incorrect assumption about language code to site name mapping (causing the unhandled exception above)Temporary fix that works for zh-min-nangerrit:1098090
Extremely slow cache update process for collections with duplication (Women's Health, wiki99, CEE Spring)Skip interwiki links already seengerrit:1098115
Truncated collection names in the CX dashboard (Women's, Essential)Use standard quotes in page marker syntaxdiff and diff
Long collection names messing up the CX dashboard layoutT380702: Consider length of Collection names on different viewstodo
The "Essential articles" collection can show "en:Vital articles" as a recommendationFilter out links that are not in the main namespacegerrit:1098130
page-collections api endpoint fails when the cache is emptyIt should return []gerrit:1098139
SBisson triaged this task as High priority.
SBisson moved this task from Backlog to Needs Code Review on the LPL Hypothesis board.

Change #1098068 merged by jenkins-bot:

[research/recommendation-api@master] Return [] instead of None or "" when an array is expected

https://gerrit.wikimedia.org/r/1098068

Change #1098124 had a related patch set uploaded (by Nik Gkountas; author: Nik Gkountas):

[research/recommendation-api@master] Use sitematrix and interwiki map to properly find dbname for links

https://gerrit.wikimedia.org/r/1098124

Change #1098139 had a related patch set uploaded (by Sbisson; author: Sbisson):

[research/recommendation-api@master] Let the page collection cache return [] when empty

https://gerrit.wikimedia.org/r/1098139

Change #1098124 merged by jenkins-bot:

[research/recommendation-api@master] Use sitematrix and interwiki map to properly find dbname for links

https://gerrit.wikimedia.org/r/1098124

Change #1098115 merged by jenkins-bot:

[research/recommendation-api@master] Cache update: skip iw links already discovered through wikidata

https://gerrit.wikimedia.org/r/1098115

Change #1098139 merged by jenkins-bot:

[research/recommendation-api@master] Let the page collection cache return [] when empty

https://gerrit.wikimedia.org/r/1098139

Change #1098412 had a related patch set uploaded (by Santhosh; author: Santhosh):

[research/recommendation-api@master] get_articles_by_titles: Skip if dbname does not exist

https://gerrit.wikimedia.org/r/1098412

Change #1098412 merged by jenkins-bot:

[research/recommendation-api@master] get_articles_by_titles: Skip if dbname does not exist

https://gerrit.wikimedia.org/r/1098412

Change #1098495 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update recommendation-api to 2024-11-27-065850-production

https://gerrit.wikimedia.org/r/1098495

Change #1098495 merged by jenkins-bot:

[operations/deployment-charts@master] Update recommendation-api to 2024-11-27-065850-production

https://gerrit.wikimedia.org/r/1098495

Change #1098090 abandoned by Sbisson:

[research/recommendation-api@master] Minimal fix for zh-min-nan site name

Reason:

General solution implemented in I186f8f32f6595de83e10a24edfb8e2c2b5c83800

https://gerrit.wikimedia.org/r/1098090

Mentioned in SAL (#wikimedia-operations) [2024-11-27T15:05:23Z] <kart_> Updated recommendation-api to 2024-11-27-142924-production (T380838, T379036, T380699)

Change #1098991 had a related patch set uploaded (by Sbisson; author: Sbisson):

[research/recommendation-api@master] Populate the cache async on startup

https://gerrit.wikimedia.org/r/1098991

Change #1098991 merged by jenkins-bot:

[research/recommendation-api@master] Populate the cache async on startup

https://gerrit.wikimedia.org/r/1098991

Change #1099152 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update recommendation-api to 2024-11-28-163815-production

https://gerrit.wikimedia.org/r/1099152

Change #1099152 merged by jenkins-bot:

[operations/deployment-charts@master] Update recommendation-api to 2024-11-28-163815-production

https://gerrit.wikimedia.org/r/1099152

The latest version was deployed to production. It filled it's cache successfully and is now serving the collections list and collection-based recommendations.