Page MenuHomePhabricator

SPIKE: Investigate Haaretz Proxy Issues [8 H]
Closed, ResolvedPublicSpike

Description

Haaretz English and Haaretz Hebrew has access isssues.

We reached out to Haaretz and they shared the following response.

It seems that we’re serving limited access articles due to proxy rule that your site is using.
We’re are using cookie based impl for paywall content, so the way that your sites is recognized as a paying is through specific cookie that your rule is somehow deleting .
Please ask your team to exclude our site cookies from being deleted - primarily : “sso_token”.

Our current proxy config is as follows:

# Haaretz English
# https://phabricator.wikimedia.org/T355668
AnonymousURL -OPTIONS +https://www.haaretz.com/srv/ipAcl?isProxyServer=true*
Title Haaretz English
URL https://www.haaretz.com/
HJ https://www.haaretz.com/
DJ haaretz.com
Find https:\u002F\u002Fwww.haaretz.com
Replace https:\u002F\u002F^swww.haaretz.com^


# Haaretz Hebrew
# https://phabricator.wikimedia.org/T355669
Title Haaretz Hebrew
AnonymousURL -OPTIONS +https://www.haaretz.co.il/srv/ipAcl?isProxyServer=true*
URL https://www.haaretz.co.il/
HJ https://www.haaretz.co.il/
DJ haaretz.co.il
Find https:\u002F\u002Fwww.haaretz.co.il
Replace https:\u002F\u002F^swww.haaretz.co.il^

Event Timeline

jsn.sherman renamed this task from Haaretz - Proxy Issues to SPIKE: Investigate Haaretz Proxy Issues.Jun 18 2024, 4:23 PM
jsn.sherman added a project: Spike.
jsn.sherman updated the task description. (Show Details)
Restricted Application changed the subtype of this task from "Task" to "Spike". · View Herald TranscriptJun 18 2024, 4:23 PM
Kgraessle renamed this task from SPIKE: Investigate Haaretz Proxy Issues to SPIKE: Investigate Haaretz Proxy Issues [8 H].Jun 25 2024, 3:19 PM
Kgraessle moved this task from To be estimated to Up next on the Moderator-Tools-Team board.
Kgraessle changed the task status from Open to Stalled.Jul 10 2024, 6:22 PM

I tested the following configurations to prevent cookies from being deleted, but I'm still seeing the issue.

Option DomainCookieOnly
Option CookiePassThrough

I'm wondering if it's due to the cookie being set by javscript.

Per the docs:

Option DomainCookieOnly specifies that EZproxy should attempt to perform all cookie handling itself, without involving remote user’s browser in cookie handling. The name of this option is somewhat misleading since it actually directs EZproxy to handle both domain and non- domain cookies. All cookies that are set by the web server using HTTP Set -Cookie will be stored by EZproxy and sent back to the remote web server without being presented to the remote user’s browser. Cookies set by JavaScript are blocked from reaching the remote web server.

There's also this Cookie Filter directive, but it looks like you can only use the enhanced features on versions greater than EZproxy v7.3, which we aren't.

@sjvipin Could you reach out to Haaretz and let them know that adding the cookie didn't fix the issues.

Dropping config we received from Haaretz in here for reference later:

Option CookiePassThrough
Title Haaretz
URL https://www.haaretz.co.il/
HJ https://www.haaretz.co.il
HJ www.haaretz.co.il
DJ haaretz.co.il
Option Cookie

Option CookiePassThrough
Title The Marker
URL https://www.themarker.com/
HJ https://www.themarker.com
HJ www.themarker.com
DJ themarker.com
Option Cookie

jsn.sherman changed the task status from Stalled to In Progress.Jul 29 2024, 5:52 PM
jsn.sherman moved this task from In Progress to Eng review on the Moderator-Tools-Team (Kanban) board.

This issue should be resolved for Haaretz English and Haaretz Hebrew. Both sites seem to work fine in basic tests, but please let us know if you have any troubles.

pull request (represents production)
https://github.com/WikipediaLibrary/twlight_ezproxy/pull/49

Looks quite good! I tried opening a few articles in English and Hebrew, and they all worked.

The wikipedialibrary.wmflabs.org site still says "Service issue" on the two Haaretz items, and perhaps these warnings can be removed now.

jsn.sherman moved this task from QA to Done on the Moderator-Tools-Team (Kanban) board.

Looks quite good! I tried opening a few articles in English and Hebrew, and they all worked.

thanks for checking!

The wikipedialibrary.wmflabs.org site still says "Service issue" on the two Haaretz items, and perhaps these warnings can be removed now.

Done.

I'm closing this out now, but anyone who encounters this problem again with one of the Haaretz sites should feel free to reopen this task.

TheMarker has the same issue. It’s the same publisher so I think that the solution will be similar.

Kgraessle changed the task status from Open to Stalled.Sep 12 2024, 4:14 PM
Kgraessle changed the task status from Stalled to Open.Sep 12 2024, 6:35 PM

I do not believe we ever established The Marker as a formal Library partner, the ezproxy configuration (including The Marker domains) was needed to get Haaretz proxy to work. I'm wondering if the solution here is to disable the Marker in the admin interface. If we do want to try to keep The Marker as a formal partner I can reach back out to our contact with Haaretz and let them know their configuration is not working.

Kgraessle changed the task status from Open to Stalled.Sep 16 2024, 3:15 PM
Kgraessle moved this task from In Progress to Eng review on the Moderator-Tools-Team (Kanban) board.
Kgraessle moved this task from Eng review to In Progress on the Moderator-Tools-Team (Kanban) board.