Page MenuHomePhabricator

[S] Update the logo detection service request to call the production endpoint
Closed, ResolvedPublic

Description

In T364551: [SPIKE] Send an image thumbnail to the logo detection service we implemented a MW API endpoint that calls the logo detection service deployed in LiftWing staging.

Event Timeline

MarkTraceur renamed this task from Update the logo detection service request to call the production endpoint to [S] Update the logo detection service request to call the production endpoint.Jul 26 2024, 3:26 PM

Change #1062683 had a related patch set uploaded (by Matthias Mullie; author: Matthias Mullie):

[mediawiki/extensions/UploadWizard@master] Update request to call the production endpoint

https://gerrit.wikimedia.org/r/1062683

Change #1062683 merged by jenkins-bot:

[mediawiki/extensions/UploadWizard@master] Update request to call the production endpoint

https://gerrit.wikimedia.org/r/1062683

These updates were made (see https://github.com/wikimedia/mediawiki-extensions-UploadWizard/blob/master/includes/ApiMediaDetection.php#L29), but it looks like it still isn't accessible from k8s hosts.

These work:

curl -H 'X-Wikimedia-Debug: backend=mwdebug1001.eqiad.wmnet' https://commons.wikimedia.org/w/api.php\?action\=mediadetection\&format\=json\&formatversion\=2\&filename\=Cat_playing_with_a_lizard.jpg
{"predictions":[{"filename":"Cat_playing_with_a_lizard.jpg","target":"logo","prediction":0.0019,"out_of_domain":0.9981}]}
curl -H 'X-Wikimedia-Debug: backend=mwdebug2001.codfw.wmnet' https://commons.wikimedia.org/w/api.php\?action\=mediadetection\&format\=json\&formatversion\=2\&filename\=Cat_playing_with_a_lizard.jpg
{"predictions":[{"filename":"Cat_playing_with_a_lizard.jpg","target":"logo","prediction":0.0019,"out_of_domain":0.9981}]}

But then:

curl -H 'X-Wikimedia-Debug: backend=k8s-mwdebug' https://commons.wikimedia.org/w/api.php\?action\=mediadetection\&format\=json\&formatversion\=2\&filename\=Cat_playing_with_a_lizard.jpg
{"error":{"code":"http-curl-error","info":"Error fetching URL: Failed to connect to upload.wikimedia.org port 443: Connection timed out","docref":"See https://commons.wikimedia.org/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/postorius/lists/mediawiki-api-announce.lists.wikimedia.org/> for notice of API deprecations and breaking changes."},"servedby":"mw-debug.eqiad.pinkunicorn-78bb88b94-jrtcr"}

Hi @matthiasmullie, I have tried to reproduce this issue by accessing k8s-mwdebug through the REPL. As shown below, I was able to get a response from the logo-detection endpoint. However, when I tried to access upload.wikimedia.org, I ran into an error similar to the one you reported: Failed to connect to upload.wikimedia.org port 443: Connection timed out.

Could you please verify if you are able to connect to upload.wikimedia.org from k8s-mwdebug?

1.Successfully accessed logo-detection endpoint from the REPL:

kevinbazira@deploy1003:~$ mw-debug-repl commonswiki
Becoming root...
Finding a mw-debug pod in eqiad...
Now running shell.php for commonswiki inside pod/mw-debug.eqiad.pinkunicorn-78bb88b94-6qwl4...
Psy Shell v0.12.3 (PHP 7.4.33 — cli) by Justin Hileman
> 
> 
> // Initialize a cURL session
> $ch = curl_init();
= curl resource #1501

> 
> // Set the URL and other options
> curl_setopt($ch, CURLOPT_URL, "http://localhost:6031/v1/models/logo-detection:predict");
= true

> curl_setopt($ch, CURLOPT_POST, true);
= true

> curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode([
.     "instances" => [
.         [
.             "filename" => "Cambia_logo.png",
.             "image" => "iVBORw0KG...",  // Base64 image string
.             "target" => "logo"
.         ]
.     ]
. ]));
= true

> 
> curl_setopt($ch, CURLOPT_HTTPHEADER, [
.     "Host: logo-detection.logo-detection.wikimedia.org",
.     "Content-Type: application/json"
. ]);
= true

> 
> curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);
= true

> curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
= true

> // Execute the request
> $response = curl_exec($ch);
= "{"error":"Error decoding image Cambia_logo.png: Invalid base64-encoded string: number of data characters (9) cannot be 1 more than a multiple of 4"}"

> 
> 
> // Close the cURL session
> curl_close($ch);
= null
> 
> 
> exit

   INFO  Goodbye.

2.Ran into connection issue when attempting to access upload.wikimedia.org from the REPL:

kevinbazira@deploy1003:~$ mw-debug-repl commonswiki
Becoming root...
Finding a mw-debug pod in eqiad...
Now running shell.php for commonswiki inside pod/mw-debug.eqiad.pinkunicorn-78bb88b94-6qwl4...
Psy Shell v0.12.3 (PHP 7.4.33 — cli) by Justin Hileman
> // Initialize a cURL session
> $ch = curl_init();
= curl resource #1501

> 
> // Set the URL
> curl_setopt($ch, CURLOPT_URL, "https://upload.wikimedia.org/wikipedia/commons/2/20/Cambia_logo.png");
= true

> // Set the option to output the result as a string
> curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
= true

> // Execute the cURL session and store the output
> $output = curl_exec($ch);

= false

> 
> // Check for errors
> if ($output === false) {
.     echo "cURL Error: " . curl_error($ch);
. } else {
.     // Output the result
.     echo $output;
. }
cURL Error: Failed to connect to upload.wikimedia.org port 443: Connection timed out⏎
> 
> 
> // Close the cURL session
> curl_close($ch);
= null

> 
> exit

   INFO  Goodbye.

Thanks for the pointer, @kevinbazira

Looks like we now have a similar problem as we did before, this time not being able to access the thumbnails from k8s :p
I had a quick look; seems like we can get the thumb from http://localhost:6101 (swift-eqiad) instead of https://upload.wikimedia.org.

Change #1066754 had a related patch set uploaded (by Matthias Mullie; author: Matthias Mullie):

[mediawiki/extensions/UploadWizard@master] Update thumbnail origin

https://gerrit.wikimedia.org/r/1066754

Change #1066754 merged by jenkins-bot:

[mediawiki/extensions/UploadWizard@master] Update thumbnail origin

https://gerrit.wikimedia.org/r/1066754

Looks like this all works now!

curl -H 'X-Wikimedia-Debug: backend=mwdebug1001.eqiad.wmnet' https://commons.wikimedia.org/w/api.php\?action\=mediadetection\&format\=json\&formatversion\=2\&filename\=Cat_playing_with_a_lizard.jpg
{"predictions":[{"filename":"Cat_playing_with_a_lizard.jpg","target":"logo","prediction":0.0019,"out_of_domain":0.9981}]}
curl -H 'X-Wikimedia-Debug: backend=mwdebug2001.codfw.wmnet' https://commons.wikimedia.org/w/api.php\?action\=mediadetection\&format\=json\&formatversion\=2\&filename\=Cat_playing_with_a_lizard.jpg
{"predictions":[{"filename":"Cat_playing_with_a_lizard.jpg","target":"logo","prediction":0.0019,"out_of_domain":0.9981}]}
curl -H 'X-Wikimedia-Debug: backend=k8s-mwdebug' https://commons.wikimedia.org/w/api.php\?action\=mediadetection\&format\=json\&formatversion\=2\&filename\=Cat_playing_with_a_lizard.jpg
{"predictions":[{"filename":"Cat_playing_with_a_lizard.jpg","target":"logo","prediction":0.0019,"out_of_domain":0.9981}]}