Migrate linkrecommendations to mw-api-int
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Clement_Goubert
	Apr 5 2023, 8:54 AM

Description

Switch backend mw-api calls from baremetal api_appserver cluster to kubernetes hosted mw-api-int

Related Objects
Search...

Status	Assigned	Task
Stalled	None	T255792 Quibble runs core:unit tests twice!
Open	None	T328919 Upgrade to PHPUnit 10
Open	None	T338103 Micro-optimize ApiResult::isMetadataKey with str_starts_with once we support PHP8+
Open	None	T328921 Drop PHP 7.4 support from MediaWiki
Stalled	None	T334726 Use return type `never` in Wikibase
Open	None	T328922 Drop PHP 8.0 support from MediaWiki
Stalled	None	T319055 Upgrade to psr/container 2.x
Stalled	Krinkle	T319432 Migrate WMF production from PHP 7.4 to PHP 8.1
Open	None	T291916 Tracking task for Bullseye migrations in production
Stalled	None	T356293 Migrate MW appservers' base images to bullseye
Open	None	T290536 Serve production traffic via Kubernetes
In Progress	Clement_Goubert	T333120 Migrate internal traffic to k8s
Resolved	Clement_Goubert	T334060 Migrate linkrecommendations to mw-api-int

Event Timeline

Clement_Goubert created this task.Apr 5 2023, 8:54 AM

Restricted Application removed a project: Patch-For-Review. · View Herald TranscriptApr 5 2023, 8:54 AM

Clement_Goubert triaged this task as Medium priority.Apr 5 2023, 8:55 AM

Clement_Goubert moved this task from Incoming 🐫 to this.quarter 🍕 on the serviceops board.

Clement_Goubert added a project: Add-Link.

Restricted Application added a project: Growth-Team. · View Herald TranscriptApr 5 2023, 8:55 AM

Clement_Goubert mentioned this in T333120: Migrate internal traffic to k8s.Apr 5 2023, 9:06 AM

https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/905941

@kostajh @Tgr If you could provide me with a way to test that linkrecommendations works correctly after changing api backend, it'd be much appreciated :)

Change 903646 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/deployment-charts@master] cxserver: Switch to mw-api-int-async on k8s

https://gerrit.wikimedia.org/r/903646

gerritbot added a project: Patch-For-Review.Apr 6 2023, 1:07 PM

In T334060#8761897, @Clement_Goubert wrote:

@kostajh @Tgr If you could provide me with a way to test that linkrecommendations works correctly after changing api backend, it'd be much appreciated :)

Grab a title from https://simple.wikipedia.org/w/index.php?search=hasrecommendation%3Alink&title=Special%3ASearch&ns0=1
Test the staging endpoint with
1. curl "https://staging.svc.eqiad.wmnet:4005/v1/linkrecommendations/wikipedia/simple/<title_with_underscores>?threshold=0.5&max_recommendations=1"
Test the internal endpoint with
1. curl "https://linkrecommendation.discovery.wmnet:4005/v1/linkrecommendations/wikipedia/simple/<title_with_underscores>?threshold=0.5&max_recommendations=1"
Test the external endpoint with
1. curl "https://api.wikimedia.org/service/linkrecommendation/v1/linkrecommendations/wikipedia/simple/<title_with_underscores>?threshold=0.5&max_recommendations=1"
2. curl "https://linkrecommendation.discovery.wmnet:4006/v1/linkrecommendations/wikipedia/simple/<title_with_underscores>?threshold=0.5&max_recommendations=1"

The output should be something like

{
  "links": [
    {
      "context_after": " at sea. G",
      "context_before": "ind their ",
      "link_index": 0,
      "link_target": "Longitude",
      "link_text": "longitude",
      "match_index": 0,
      "score": 0.6056926250457764,
      "wikitext_offset": 713
    }
  ],
  "links_count": 1,
  "meta": {
    "application_version": "8b645e0",
    "dataset_checksums": {
      "anchors": "d3a569ad213a1fe3e6003b120109c7ec49b73c7593abd7931eeae917f7354537",
      "model": "6b5ee9c0018c1792baa9d71c4701c4360805b63660e955b64c39f67d0f71cef4",
      "pageids": "deadca08a1ccca465b59cc1d94d98d96116171b8ffc4dc7ac1e70f0412eeac8e",
      "redirects": "ece0b58352fb5b15223520124ef3ccc8e67478e565bc40975918a156ab6958c3",
      "w2vfiltered": "1940d4ee2fad64771d743913e561a47c395c9316c34dd7a96e30a8946959694e"
    },
    "format_version": 1
  },
  "page_title": "Time zone",
  "pageid": 4003,
  "revid": 7817764
}

(without the pretty-printing)

https://wikitech.wikimedia.org/wiki/Add_Link#Link_Recommendation_Service and https://wikitech.wikimedia.org/wiki/Add_Link#Deployment_2 has more information if needed.

Tgr moved this task from Inbox to Triaged on the Growth-Team board.Apr 13 2023, 9:45 PM

Mentioned in SAL (#wikimedia-operations) [2023-04-17T12:59:19Z] <claime> Migrating linkrecommandation staging to mw-api-int - T334060

Mentioned in SAL (#wikimedia-operations) [2023-04-17T14:07:06Z] <claime> Migrating linkrecommandation to mw-api-int - T334060

Mentioned in SAL (#wikimedia-operations) [2023-04-17T14:12:08Z] <claime> Migrated linkrecommandation to mw-api-int - T334060

curl tests look good:

cgoubert@deploy2002:/srv/deployment-charts/helmfile.d/services/linkrecommendation$ curl "https://linkrecommendation.discovery.wmnet:4005/v1/li
nkrecommendations/wikipedia/simple/Portugal?threshold=0.5&max_recommendations=1" | jq                                                        
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                                              
                                 Dload  Upload   Total   Spent    Left  Speed                                                                
100   759  100   759    0     0   1971      0 --:--:-- --:--:-- --:--:--  1971                                                               
{
  "links": [
    {
      "context_after": ". Portugal",
      "context_before": " on the ",
      "link_index": 0,
      "link_target": "Iberian Peninsula",
      "link_text": "Iberian Peninsula",
      "match_index": 0,
      "score": 0.8166778683662415,
      "wikitext_offset": 9589
    }
  ],
  "links_count": 1,
  "meta": {
    "application_version": "8b645e0",
    "dataset_checksums": {
      "anchors": "d3a569ad213a1fe3e6003b120109c7ec49b73c7593abd7931eeae917f7354537",                                                         
      "model": "6b5ee9c0018c1792baa9d71c4701c4360805b63660e955b64c39f67d0f71cef4",                                                           
      "pageids": "deadca08a1ccca465b59cc1d94d98d96116171b8ffc4dc7ac1e70f0412eeac8e",                                                         
      "redirects": "ece0b58352fb5b15223520124ef3ccc8e67478e565bc40975918a156ab6958c3",                                                       
      "w2vfiltered": "1940d4ee2fad64771d743913e561a47c395c9316c34dd7a96e30a8946959694e"                                                      
    },
    "format_version": 1
  },
  "page_title": "Portugal",
  "pageid": 3466,
  "revid": 8692778
}
cgoubert@deploy2002:/srv/deployment-charts/helmfile.d/services/linkrecommendation$ curl "https://linkrecommendation.discovery.wmnet:4006/v1/li
nkrecommendations/wikipedia/simple/Portugal?threshold=0.5&max_recommendations=1" | jq                                                        
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                                              
                                 Dload  Upload   Total   Spent    Left  Speed                                                                
100   759  100   759    0     0   1851      0 --:--:-- --:--:-- --:--:--  1851                                                               
{
  "links": [
    {
      "context_after": ". Portugal",
      "context_before": " on the ",
      "link_index": 0,
      "link_target": "Iberian Peninsula",
      "link_text": "Iberian Peninsula",
      "match_index": 0,
      "score": 0.8166778683662415,
      "wikitext_offset": 9589
    }
  ],
  "links_count": 1,
  "meta": {
    "application_version": "8b645e0",
    "dataset_checksums": {
      "anchors": "d3a569ad213a1fe3e6003b120109c7ec49b73c7593abd7931eeae917f7354537",                                                         
      "model": "6b5ee9c0018c1792baa9d71c4701c4360805b63660e955b64c39f67d0f71cef4",                                                           
      "pageids": "deadca08a1ccca465b59cc1d94d98d96116171b8ffc4dc7ac1e70f0412eeac8e",                                                         
      "redirects": "ece0b58352fb5b15223520124ef3ccc8e67478e565bc40975918a156ab6958c3",                                                       
      "w2vfiltered": "1940d4ee2fad64771d743913e561a47c395c9316c34dd7a96e30a8946959694e"                                                      
    },
    "format_version": 1
  },
  "page_title": "Portugal",
  "pageid": 3466,
  "revid": 8692778
}

❯ curl "https://api.wikimedia.org/service/linkrecommendation/v1/linkrecommendations/wikipedia/simple/Portugal?threshold=0.5&max_recommendations=1" | jq 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   759  100   759    0     0    694      0  0:00:01  0:00:01 --:--:--   694
{
  "links": [
    {
      "context_after": ". Portugal",
      "context_before": " on the ",
      "link_index": 0,
      "link_target": "Iberian Peninsula",
      "link_text": "Iberian Peninsula",
      "match_index": 0,
      "score": 0.8166778683662415,
      "wikitext_offset": 9589
    }
  ],
  "links_count": 1,
  "meta": {
    "application_version": "8b645e0",
    "dataset_checksums": {
      "anchors": "d3a569ad213a1fe3e6003b120109c7ec49b73c7593abd7931eeae917f7354537",
      "model": "6b5ee9c0018c1792baa9d71c4701c4360805b63660e955b64c39f67d0f71cef4",
      "pageids": "deadca08a1ccca465b59cc1d94d98d96116171b8ffc4dc7ac1e70f0412eeac8e",
      "redirects": "ece0b58352fb5b15223520124ef3ccc8e67478e565bc40975918a156ab6958c3",
      "w2vfiltered": "1940d4ee2fad64771d743913e561a47c395c9316c34dd7a96e30a8946959694e"
    },
    "format_version": 1
  },
  "page_title": "Portugal",
  "pageid": 3466,
  "revid": 8692778
}

Marking as resolved, I'll keep an eye on https://grafana.wikimedia.org/goto/ZoPvJOEVk?orgId=1
Feel free to reopen in case there are any issues.

Migrate linkrecommendations to mw-api-intClosed, ResolvedPublicActions

Description

Related ObjectsSearch...

Event Timeline

Migrate linkrecommendations to mw-api-int
Closed, ResolvedPublic
Actions

Related Objects
Search...