Page MenuHomePhabricator
Create Task
Maniphest T250312

HTTP 403 Error when using google-api-proxy on VPS
Closed, ResolvedPublic
Actions

Description

Sign in to English Wikisource.
Activate the Google OCR in \Preferences\Gadgets.
Open a random page in edit mode.
https://en.wikisource.org/w/index.php?title=Page:Vol_6_History_of_Mexico_by_H_H_Bancroft.djvu/531&action=edit
Click on the Google OCR in the toolbar
The message appears each time.

image.png (219×395 px, 95 KB)

⧼error⧽ undefined Client error: `POST https://googlevision-api-proxy.wmflabs.org/v1/images:annotate?key=[KEY REDACTED]` resulted in a `403 Forbidden` response:
<html>
<head><title>403 Forbidden</title></head>
<body bgcolor="white">
<center><h1>403 Forbidden</h1></center>
<hr> (truncated...)

Related Objects

Event Timeline

Ineuw created this task.Apr 15 2020, 7:21 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 15 2020, 7:21 PM
Ineuw updated the task description. (Show Details)Apr 15 2020, 7:22 PM
Aklapper renamed this task from Google OCR in en.Wikisource is broken and displays an error message (image included) to Google OCR in en.Wikisource is broken and displays an error message.Apr 15 2020, 7:24 PM
Aklapper renamed this task from Google OCR in en.Wikisource is broken and displays an error message to HTTP 403 Error when trying to use Google OCR in en.Wikisource.
Aklapper added a project: Community-Tech.Apr 15 2020, 7:28 PM

No idea where to put this; no project tag in Phabricator (which is unhelpful for everybody). Looking at history, this might be Community-Tech?

Aklapper renamed this task from HTTP 403 Error when trying to use Google OCR in en.Wikisource to HTTP 403 Error for https://googlevision-api-proxy.wmflabs.org/ when trying to use Google OCR in en.Wikisource.Apr 15 2020, 7:33 PM
Aklapper updated the task description. (Show Details)
MusikAnimal claimed this task.EditedApr 15 2020, 7:54 PM
MusikAnimal subscribed.

Yep, Community-Tech is the correct tag.

I can confirm it's not an issue with Google, they stopped receiving requests from us earlier today:

Screenshot from 2020-04-15 14-18-09.png (233×740 px, 15 KB)

I checked the security groups for the API proxy on Horizon, and ingress is set to allow Toolforge, so that part should be okay.

I'll continue to debug.

MusikAnimal renamed this task from HTTP 403 Error for https://googlevision-api-proxy.wmflabs.org/ when trying to use Google OCR in en.Wikisource to HTTP 403 Error when using google-api-proxy on VPS.Apr 15 2020, 7:55 PM
MusikAnimal mentioned this in T243736: Earwig's Copyvio Detector down due to "The supplied API key is not configured for use from this IP address." from google-api-proxy.
ifried moved this task from New & TBD Tickets to Needs Discussion on the Community-Tech board.Apr 15 2020, 7:56 PM
MusikAnimal mentioned this in T135046: Allowlist Cloud VPS instances that need XFF header passed through the web proxy.Apr 15 2020, 8:05 PM
MusikAnimal added a comment.Apr 15 2020, 8:09 PM

I'm pretty sure this happened because of T135046#6060220, going by the puppet role.

Stashbot added a comment.Apr 15 2020, 8:22 PM

Mentioned in SAL (#wikimedia-cloud) [2020-04-15T20:22:54Z] <bd808> Added google-api-proxy.wmflabs.org & googlevision-api-proxy.wmflabs.org to profile::wmcs::novaproxy::xff_fqdns (T135046, T250312)

MusikAnimal closed this task as Resolved.Apr 15 2020, 8:26 PM

Should be working now! Sorry we didn't notice this ahead of time.

• aezell subscribed.Apr 15 2020, 8:28 PM

Any idea why this changed? I think I saw an email about some network changes in Toolforge. Is that what happened?

MusikAnimal added a comment.EditedApr 15 2020, 8:31 PM
In T250312#6060287, @aezell wrote:

Any idea why this changed? I think I saw an email about some network changes in Toolforge. Is that what happened?

Yep. XFF headers were intentionally removed for VPS traffic earlier today, after a two week window to request to be whitelisted. We just didn't realize google-api-proxy relied on them. All good now.

Aklapper added a subscriber: ifried.Apr 15 2020, 8:36 PM
In T250312#6060185, @MusikAnimal wrote:

Yep, Community-Tech is the correct tag.

@MusikAnimal, @ifried et al: Thanks. Could that be fixed (request dedicated code project tag, potential Herald rule to also add the Community-Tech tag), so people could put a task into the right basket (=project tag) without having to know about existence and names of WMF teams and what these teams work(ed) on? :)

MusikAnimal added a comment.Apr 15 2020, 8:56 PM
In T250312#6060306, @Aklapper wrote:

Could that be fixed (request dedicated code project tag, potential Herald rule to also add the Community-Tech tag), so people could put a task into the right basket (=project tag) without having to know about existence and names of WMF teams and what these teams work(ed) on? :)

I created T250319 for the proxy. We should create one for the Google OCR tool too (which uses the proxy), but from T247284 it seems we're not sure about naming yet. I'll start a discussion there.

MusikAnimal mentioned this in T247284: Improve OCR: Move ws-google-ocr repository to Gerrit.Apr 15 2020, 9:00 PM
Samwilson added a project: Wikimedia OCR.Apr 15 2020, 9:47 PM
Ineuw added a comment.Apr 16 2020, 1:05 AM

Just started using it in English Wikisource and its working.

MusikAnimal added a project: Google-api-proxy.Apr 16 2020, 6:48 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL