Page MenuHomePhabricator

Switch the Recommendation API to use the internal WDQS cluster
Closed, ResolvedPublic

Description

Now that we have a WDQS cluster that can be accessed only inside production (cf. T178492) we should switch the Recommendation API service to use wdqs-internal.discovery.wmnet for issuing requests.

Details

Related Gerrit Patches:

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 21 2018, 11:29 AM

The query does not always fail, e.g. I have just executed https://en.wikipedia.org/api/rest_v1/data/recommendation/article/creation/translation/es successfully. The issue that you are experiencing is known and the real culprit here is the WDQS cluster (which is used internally by the Recommendation API service). There are options being currently considered for making the cluster more robust, cf. T178492: Create a more controlled WDQS cluster.

mobrovac added subscribers: Gehel, Smalyshev.

@Gehel @Smalyshev should we now point the Recommendation API service to use a different LVS for WDQS? We are currently using wdqs.discovery.wmnet

Gehel added a comment.May 1 2018, 9:52 AM

Yes, we should migrate! I'm not sure where the recommendation API is configured. Mediawiki has been migrated to wdqs-internal.discovery.wmnet (T192942) around 0:00 UTC Apr 1st (just 10 hours ago), and I have no report of any issue yet. Note that wdqs-internal.discovery.wmnet only supports HTTP at this point (T193473).

Change 430052 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[operations/puppet@production] Recommendation API: Migrate to the new WDQS internal cluster

https://gerrit.wikimedia.org/r/430052

mobrovac claimed this task.May 1 2018, 10:10 AM
mobrovac edited projects, added Services (doing); removed Services (blocked).

Yes, we should migrate! I'm not sure where the recommendation API is configured. Mediawiki has been migrated to wdqs-internal.discovery.wmnet (T192942) around 0:00 UTC Apr 1st (just 10 hours ago), and I have no report of any issue yet. Note that wdqs-internal.discovery.wmnet only supports HTTP at this point (T193473).

Great! I submitted a patch, so when you have 10 mins this week, we should switch it.

mobrovac renamed this task from Recommendation API fails with internal error if the optional seed_article param is missing to Switch the Recommendation API to use the internal WDQS cluster.May 1 2018, 10:31 AM
mobrovac updated the task description. (Show Details)
mobrovac removed subscribers: Smalyshev, Aklapper.
mobrovac added a subscriber: bmansurov.

Change 430052 merged by Gehel:
[operations/puppet@production] Recommendation API: Migrate to the new WDQS internal cluster

https://gerrit.wikimedia.org/r/430052

Mentioned in SAL (#wikimedia-operations) [2018-05-07T08:58:05Z] <mobrovac@tin> Started restart [recommendation-api/deploy@ac66089]: Use the internal WDQS cluster LVS - T190266

mobrovac closed this task as Resolved.May 7 2018, 9:01 AM
mobrovac edited projects, added Services (done); removed Services (doing), Patch-For-Review.

The Recommendation API service is now using the internal WDQS cluster, which should greatly improve the service's stability. Resolving.