Page MenuHomePhabricator

Fix Newspapers.com access issues
Closed, ResolvedPublic

Assigned To
Authored By
sjvipin
Nov 11 2022, 10:48 AM
Referenced Files
F57768197: Screen Shot 2024-12-01 at 11.27.33 AM.png
Dec 1 2024, 4:27 PM
F57533843: Screenshot 2024-09-24 at 10.54.04.png
Sep 24 2024, 9:54 AM
F57410153: Screen Shot 2024-09-02 at 1.35.01 PM.png
Sep 2 2024, 5:41 PM
F12: project.png
Aug 29 2024, 8:31 AM
File Not Attached
Restricted File
Jul 4 2024, 4:12 AM
F41721934: image.png
Jan 27 2024, 7:22 AM
F41436405: image.png
Nov 3 2023, 4:06 AM
F37819456: Screenshot_20230927-201747.png
Sep 28 2023, 3:25 AM

Description

August 1 2024: For a full update on Newspapers.com please see T322916#10036209

Current issues

  • We do not currently seem to have access to some Publisher Extra content
  • Users cannot log in via Ancestry
  • Users cannot log in via Facebook
  • Users cannot register a new account via Email

Past issues
Cloudflare is preventing access. Users get stuck at www.newspapers.com needs to review the security of your connection before proceeding.

Users currently cannot log out of their account while proxied, or log in via Ancestry.

This change has now been deployed, however we are seeing some access issues. Some users cannot retrieve search results, and logging in to accounts sometimes doesn't work.

Users can't login as they see the following error:

image.png (133×482 px, 13 KB)

Newspapers.com fixed the initial issue here, which was enabling reCaptcha, however we're now getting a slightly different error about an "invalid domain key". We should be able to fix this issue, and are investigating.

Original task
Our existing partner newspapers.com has agreed to move to library bundle. We need to update their EZProxy configuration.

Partner id: 26 (https://wikipedialibrary.wmflabs.org/partners/26/)

Before making this change please send me a list of the user emails (privately) with active authorisations so I can let them know.

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

The cookie-duplication trick worked for me on the proxy site on Firefox 129.0, but Newspapers.com is still reporting that I am only a Registered Guest, and I cannot make any clippings whatsoever (whether Publisher Extra content or not).

I still have a problem with the website. I am using the most recent version of Chrome - and even deleted my cookies.

I'm not deleting any cookies. That causes too many problems.

Furthermore, now I can't sign into newspapers.com, period. It tells me either my username or password is incorrect. I am told to enter a code to reset my password but it never gets emailed to me.

Oh, wait, I'm signed in now. Because I typed "newspapers.com". But I'm betting I can't create clippings.

This comment was removed by BJCHK.
This comment was removed by KylieTastic.

Is this a new problem, and is it related?

There was a problem loading the search results. Please try again.
If the problem persists, please contact us.

Oh, wait, I'm signed in now. Because I typed "newspapers.com". But I'm betting I can't create clippings.

On a Mac it is: "Fn + F12" or "Cmd + Option + I"

https://developer.chrome.com/docs/devtools/open#shortcuts

I'm not on a Mac.

I was replying to the comment made before that was someone saying they were having issues opening the dev tools on a Mac.... they deleted the comment.. Not sure how I managed to respond to the wrong one. sorry for the confusion

I was having issues with searching (I saw the error 429 Too Many Requests in DevTools), but creating an account and copying the cookies fixed it!

This has been incredibly frustrating. I had Newspapers.com access before the new system, which just made things more annoying, and then this happened-making doing research much harder.

I'm still unable to sign into newspapers.com via the WikLibrary proxy. I repeatedly get the message "There was an error while loading the form. Please contact customer support" This issue has been ongoing for months.

@KewGardens613 and @Flibirigit : you may be able to get access by following the steps outlined above in the comments on August 14 and August 15. It's a bit complicated, but it worked for me and several others in this thread.

I was replying to the comment made before that was someone saying they were having issues opening the dev tools on a Mac.... they deleted the comment.. Not sure how I managed to respond to the wrong one. sorry for the confusion

Well, one thing's for sure, I wouldn't know how to do anything involving developer tools.

Is there a way to restore my access to the Publisher Extra content? IIRC when I first signed up for access there was a bullet point on the Wikipedia Library portal saying something to the effect of "You must renew your Newspapers.com subscription by reapplying annually", but that text seems to be gone. Was it just bad luck that the expiration coincided with this implementation of proxy servers, or am I misremembering things? My Newspapers.com account name is "digitaliceage"; my profile page currently has me listed as a "Registered Guest" when it wasn't before.

Is there a way to restore my access to the Publisher Extra content? IIRC when I first signed up for access there was a bullet point on the Wikipedia Library portal saying something to the effect of "You must renew your Newspapers.com subscription by reapplying annually", but that text seems to be gone. Was it just bad luck that the expiration coincided with this implementation of proxy servers, or am I misremembering things? My Newspapers.com account name is "digitaliceage"; my profile page currently has me listed as a "Registered Guest" when it wasn't before.

Please see T322916#10036209 for the latest update.

I think that there might be some confusion as to using the backend for the NP proxy login. It has worked for me when I am simultaneously logged into NP proper and the cookies are directly copied over into the proxy; this includes the duration date especially. If the login session on NP ends, as in a timed sign out after a month or so, you HAVE to do the process over again, which isn't terribly hard, but that's solely because the cookies need to be in sync with each other.

Would that there could be a fix for the non-technically inclined layman (which fortunately I am not), but this is probably the best we can do for the long term.

On Chrome, I was able to view the images that make up the page by going to the Sources tab, selecting the Page tab below that and "img-newspapers-com ...", opening the image I wanted in a new tab and changing the crop in the url. The first two numbers move the crop from side to side and the last two control the zoom in or out. Useful if you just want to verify something that doesn't show up on Ancestry's Obituary Index, or (as in my case) the workaround doesn't work.

When I get to this URL: https://www-newspapers-com.wikipedialibrary.idm.oclc.org/signin/?next_url=/, I get this error message: There was an error while loading the form. Please contact customer support. I tried changing the URL to www.newspapers.com instead of the hyphens but it automatically reverts back. Is there a fix for this?

When I get to this URL: https://www-newspapers-com.wikipedialibrary.idm.oclc.org/signin/?next_url=/, I get this error message: There was an error while loading the form. Please contact customer support. I tried changing the URL to www.newspapers.com instead of the hyphens but it automatically reverts back. Is there a fix for this?

The official update is in T322916#10036209. For a workaround in Firefox (with similarities to Chrome), see T322916#10063620 or the more detailed version in T322916#10065665.

Earlier this week I had trouble signing in to post when I was at a library, but I saved a URL in the form https://www-newspapers-com.wikipedialibrary.idm.oclc.org/image/1110761182 because I wanted to print a photo and couldn't make a clipping. It told me to subscribe. I was able to print the photo when I changed to https://www-newspapers-com.wikipedialibrary.idm.oclc.org/browse/united-states/north-carolina/asheville/asheville-citizen-times_4039/2024/08/20/

And I'm getting an error message "There was a problem verifying you are not a robot. Please try again or refresh the page." (Trying again and/or refreshing the page doesn't resolve the problem.) I've also tried several different browsers in Private mode to avoid any possible issues with cookies.

Good news! It looks like we'll be able to move back to the individual account setup method. Further details TBC.

Samwalton9-WMF assigned this task to sjvipin.

TL;DR: To regain access to Newspapers.com please go to https://wikipedialibrary.wmflabs.org/partners/26/ and click Apply. Access requests will be processed and sent to Newspapers.com over the coming weeks.

Newspapers.com have agreed to move back to our old method of granting access, whereby users must first register at Newspapers.com, then apply via The Wikipedia Library, and then accounts are upgraded to a subscribing one by Newspapers.com. We have updated Newspapers.com to accept applications in the library, so you can apply now! This change is unfortunately mutually exclusive with the limited proxy-based access working, so any of the technical workarounds that have been discussed here should no longer work, along with any proxy-based access to Newspapers.com. I hope you agree that this brief bump will be worth it in the long run. As a reminder, this change means that you will need to renew your access on a yearly basis. If you have an email configured in The Wikipedia Library you should receive an email a few weeks before this expiry to remind you to submit an extension request, but brief yearly loss of access may be a side effect of this access method. The good news is that this access method means that you shouldn't encounter any technical problems besides normal operational problems on Newspapers.com's end, which should be few and resolved quickly - otherwise the website should work just as if you were a normal customer.

We are planning to directly notify all users who have used Newspapers.com via the proxy over the past year separately to this announcement.

Thanks may be directed towards @sjvipin who has been working tirelessly to find a way forward with Newspapers.com :)

This is good news, thank you! However, the Apply button isn't working for me. I've tried it in Safari and Chrome, logged into my Wikipedia Library account. Nothing happens when I click the Apply button.

This is good news, thank you! However, the Apply button isn't working for me. I've tried it in Safari and Chrome, logged into my Wikipedia Library account. Nothing happens when I click the Apply button.

Very odd! This might be related to you having a previous application, but I see that other users have been able to request renewal.

Can you instead try clicking the 'View my applications' button in the library, and then clicking 'Reuest renewal' next to the Newspapers.com entry?

Screenshot 2024-09-24 at 10.54.04.png (582×1 px, 75 KB)

Thank you, that seems to have worked!

What is renewal of existing access grant? Have I done something wrong if it says no?

What is renewal of existing access grant? Have I done something wrong if it says no?

We still had data on past manually-granted accesses, so if you had an account prior to the shift to proxy-based access, your application may be filed as a renewal of your past application. This isn't anything to be worried about :)

Thank you to all concerned for the good work. I have a couple of questions, if you please. Upon having my application approved, I signed into my account, but was met with a prompt urging me to upgrade whenever I tried to view an article. Do I have to open a brand new account or is it possible to continue using my old one? If the latter, why am I getting this prompt?

@CurryTime7-24 : I believe having your application approved does not mean that it is ready to go. Approval comes from the people running TWL, after which your request is sent to Newspapers.com. It may take them awhile to set up approved account, so I'd just wait a while. My approval email mentioned that "You can expect to receive access details within a week or two once it has been processed." so I wouldn't worry til then.

@ARandomName123 Ah! I jumped the gun then. :) Thanks so much for clearing that up.

I had asked this question of Samwalton9 privately and was told,

Yep, we just need to wait for Newspapers.com to activate this batch of accounts. I think we sent them close to 200 so I don't know how quickly they'll get through them

Until yesterday I could get to individual pages. Today, even if I sign in to newspapers.com, I cannot.

The way we have been doing this was working, but now I get only part of the page, meaning the links and descriptions, but not the page itself.

When I used the regular newspapers.com address, I got to the page but was told to subscribe. When I signed in, I was told to get a subscription.

@CurryTime7-24 My account has now been updated, and is good to go. Should be the same for you!

I only saw this now somehow-I wish I got a notification.

TL;DR: To regain access to Newspapers.com please go to https://wikipedialibrary.wmflabs.org/partners/26/ and click Apply. Access requests will be processed and sent to Newspapers.com over the coming weeks.

Newspapers.com have agreed to move back to our old method of granting access, whereby users must first register at Newspapers.com, then apply via The Wikipedia Library, and then accounts are upgraded to a subscribing one by Newspapers.com. We have updated Newspapers.com to accept applications in the library, so you can apply now! This change is unfortunately mutually exclusive with the limited proxy-based access working, so any of the technical workarounds that have been discussed here should no longer work, along with any proxy-based access to Newspapers.com. I hope you agree that this brief bump will be worth it in the long run. As a reminder, this change means that you will need to renew your access on a yearly basis. If you have an email configured in The Wikipedia Library you should receive an email a few weeks before this expiry to remind you to submit an extension request, but brief yearly loss of access may be a side effect of this access method. The good news is that this access method means that you shouldn't encounter any technical problems besides normal operational problems on Newspapers.com's end, which should be few and resolved quickly - otherwise the website should work just as if you were a normal customer.

We are planning to directly notify all users who have used Newspapers.com via the proxy over the past year separately to this announcement.

Thanks may be directed towards @sjvipin who has been working tirelessly to find a way forward with Newspapers.com :)

I only saw this now somehow-I wish I got a notification.

We emailed everyone who had accessed Newspapers.com recently, but it's possible this ended up in spam or otherwise got missed :)

We emailed everyone who had accessed Newspapers.com recently, but it's possible this ended up in spam or otherwise got missed :)

There are actually many ways to end up accessing Newspapers.com, e.g., Ancestry links still point to the OCLC proxy which I use a lot and thereby end up using Newspapers.com a lot that way where I end up seeing the "Welcome from Wikimedia Foundation" banner. I have probably never directly clicked on the Newspapers.com link from the Wikipedia Library but I have used Newspapers.com very frequently via Ancestry links where I used Ancestry directly from the Wikipedia Library. I never received any notification about this change either.

Thanks for pointing this out @Uzume. I wonder if it would make sense at this point to remove the Newspapers.com config from our EZProxy configuration @jsn.sherman? I think that would solve the above issue of ending up in a poor experience via a link from another proxied resource?

Thanks for pointing this out @Uzume. I wonder if it would make sense at this point to remove the Newspapers.com config from our EZProxy configuration @jsn.sherman? I think that would solve the above issue of ending up in a poor experience via a link from another proxied resource?

Sounds reasonable; it looks like we have some other ezproxy changes in the pipeline right now, so let's put this in the top of the library queue after that, as it should be a very straightforward change.

Thanks for pointing this out @Uzume. I wonder if it would make sense at this point to remove the Newspapers.com config from our EZProxy configuration @jsn.sherman? I think that would solve the above issue of ending up in a poor experience via a link from another proxied resource?

This change is live. Could you please test @Uzume? You may need to manually log out of the proxy by visiting https://wikipedialibrary.idm.oclc.org/logout
and then re-enter ancestry from the Wikipedia Library for the change to take effect for you.

Thanks for pointing this out @Uzume. I wonder if it would make sense at this point to remove the Newspapers.com config from our EZProxy configuration @jsn.sherman? I think that would solve the above issue of ending up in a poor experience via a link from another proxied resource?

If you do that you should also remove all the other sites related to it like Ancestry, etc. otherwise links from those sites to their own internal sisters sites will be broken. That might also rectify yours issues with Ancestry (which I believe is still an ongoing issues). I have not applied for a newspapers.com account so I am still using Ancestry via EZProxy until such time as it is removed and I need to apply for such separately from the main the Wikipedia Library access (which I of course do have).

I really like the ease of EZProxy but I understand that it is not always applicable for everything.

Thanks for pointing this out @Uzume. I wonder if it would make sense at this point to remove the Newspapers.com config from our EZProxy configuration @jsn.sherman? I think that would solve the above issue of ending up in a poor experience via a link from another proxied resource?

This change is live. Could you please test @Uzume? You may need to manually log out of the proxy by visiting https://wikipedialibrary.idm.oclc.org/logout
and then re-enter ancestry from the Wikipedia Library for the change to take effect for you.

I am not really sure what you want me to test--that you removed the EZProxy configuration and I am now locked out of newspapers.com from Ancestry, etc. links? Like I said, I have never selected newspapers.com from the main Wikipedia Library and thus do not have an account there. I suppose if I want to use such things in the future, I will have to get access and manually translate Ancestry to Newspapers.com links to get access to the content in the future. As this point that seems...too much work. Maybe I will get around to chasing such things if I really need it. For now it is just another thing broken and taken away from me that causes more work to regain and maintain. I suppose most of life is like that, sadly.

For anyone who has trouble with links pointing to www-newspapers-com.wikipedialibrary.idm.oclc.org that no longer work, you can use a browser extension that automatically converts them to newspapers.com as soon as they are clicked. (I used to have it set in the other direction when we were on the proxy.) I use Redirector for Firefox; other options exist for other browsers.

Judging by the discussion at WT:TWL, access to newspapers.com is not currently working. I thought it was happening to me only because my annual renewal period had come up (and incidentally, it would be great if users could be notified when their access to a resource is about to expire so we can renew it as promptly as possible), but other users whose renewal isn't for months yet say they're having the same problem. I don't know if this is the best place to ask, but can someone look into this?

It seems to be working for me? (I'm logged into my account that I activated with the TWL application, and not using the proxy.)

Judging by the discussion at WT:TWL, access to newspapers.com is not currently working. I thought it was happening to me only because my annual renewal period had come up (and incidentally, it would be great if users could be notified when their access to a resource is about to expire so we can renew it as promptly as possible), but other users whose renewal isn't for months yet say they're having the same problem. I don't know if this is the best place to ask, but can someone look into this?

I'm running into "you must be subscribed to view this Publisher Extra page" right now when trying to access pages. I'm thinking it might be because of browser cookies or cache relating to my TWL subscription renewal? I forget if it takes a bit of time to push everything through or not.

That being said, I was actually pleasantly surprised to see how quickly my renewal app was accepted. :)

Judging by the discussion at WT:TWL, access to newspapers.com is not currently working. I thought it was happening to me only because my annual renewal period had come up (and incidentally, it would be great if users could be notified when their access to a resource is about to expire so we can renew it as promptly as possible), but other users whose renewal isn't for months yet say they're having the same problem. I don't know if this is the best place to ask, but can someone look into this?

I've been having the same issue. I get a message reading "Upgrade to a Publisher Extra Subscription to view this page" any time I try to access an article.

Please see my comment here. I believe most folks running into issues have hit the 12-month limit on access and need to request renewal in the library.

Okay, new problem. I can't get the images in the expanded page. A few months ago, I could see them fine on my Windows Edge and Firefox browsers, but not my Chrome Linux. Now it's not working for either. Also I now get sent to www.newspapers.com/image-view/[long number] instead of www.newspapers.com/image/[long number]. Clearly some things have changed.

This is weird. I was looking through a newspaper, I made a clipping, and then made sure I could access it before putting it in Wikipedia as a source. Then I kept turning pages as I continued reading, and at some point it wanted me to subscribe. I don't know how I got out of the situation but I can see the pages normally again.

Another problem I have is signing in at the library. After I enter my email address and password, it won't let me go to the next step. If I try, it says it couldn't verify my browser. If I sign in again, it works.

Another problem I have is signing in at the library. After I enter my email address and password, it won't let me go to the next step. If I try, it says it couldn't verify my browser. If I sign in again, it works.

It's possible this was related to yesterday's Cloudflare outage?

I should have access to Newspapers.com through the Wikipedia Library, supposedly expiring on January 12 2026. I can sign in via the WP Library, search the site and see clippings that I have saved, but I am unable to see any results of searches, although I see a list of search results. The Subscription Info at My Account says "You're currently a Registered Guest. This free subscription gets you access to search newspapers but you will not be able to view the full page. Get a paid subscription for access." On Nov 10 I was told that I needed to upgrade in order to get Publishers Extra content, which I thought was part of the WP Library access. I commented about that on the WP Library talk page but there was no response. I tried to reapply at that time. I have now tried reapplying again and can get no access through WP Library at all, just a link to my application. Am I doing something wrong? How can I get access?

Per T322916#10168173, Newspapers.com access is now provided directly via an account you have registered on Newspapers.com, not via The Wikipedia Library's proxy service, so please ensure you are logged in to the Wikipedia account for which you provided the email address in your application. If that account isn't working then someone will review your application and should get back to you about getting your account set up again.

I am logged in to the Wikipedia account for which I provided the email address in my application.

Until now I have been accessing Newspapers.com by going to https://wikipedialibrary.wmflabs.org, and choosing Newspapers.com from the list and clicking on Access Collection. This has worked until recently. I don't remember the last time I was able to search and see results, but I have clippings saved from August 26th 2025. Today the result is as I described in my post yesterday.

Today I also tried going directly to https://www.newspapers.com and got access by entering my email and Newspapers.com password. I was already logged in so I signed out of my account and signed back in, with the same result.

Which, if either, of these two methods should I be using? In any case I can only get to a registered guest account status.

Today I also tried going directly to https://www.newspapers.com and got access by entering my email and Newspapers.com password. I was already logged in so I signed out of my account and signed back in, with the same result.

This is the access method you should be using. After your application was approved in January 2025 did you gain access to Newspapers.com? When did it stop working for you?

I've been accessing Newspapers.com via the Wikipedia Library site until at
least the end of August 2025, which is when I last clipped some articles. I
don't know when it stopped working, sometime before November 10, which is
when I commented on the Wikipedia Library discussion page, attempted to
reapply etc.

I just noticed that my attempt to renew in November was reviewed on
November 16th and pronounced "invalid" - probably because I already was
supposed to have (but don't have) access until January 2026. So the
application for renewal I submitted yesterday is likely also to be
rejected. Meanwhile I am recognized as a registered guest at
Newspapers.com, can see clippings I made in August 2025 and can search but
can't see the articles in the search results. I have signed into and out of
my account there repeatedly yesterday and today, with the same result. What
do I need to do to get access to Newspapers.com, please?

This is my situation: I go to https://www.newspapers.com/ and enter my email and newspapers.com password to log in. I click on my account. The subscription info is: "You're currently a Registered Guest. This free subscription gets you access to search newspapers but you will not be able to view the full page. Get a paid subscription for access." The account info contains my email address and the fact that I have been a member since February 20, 2015. I can search but can't see results. Instead I am invited to "Upgrade to a Publisher Extra Subscription to view this page". I can get access to all the clippings I have saved, the most recent of which was done on August 26, 2025. This situation has not changed since at least November 10, 2025.

At the Wikipedia Library (https://wikipedialibrary.wmflabs.org/users/my_library/) under Newspapers.com it says that my access expires on January 12, 2026. On December 1 I reapplied for access. The reapplication is listed as Pending, not yet reviewed. An earlier reapplication that I made on November 10, 2025 (thinking that perhaps my access had been cut off prematurely) was refused on November 16 as Invalid. So I assume this current one will also be unsuccessful when and if it is reviewed.

How can I get access to my Newspapers.com account please?

Hi again,

I posted this on Maniphest this morning and am contacting you again in the
hope that someone can help me regain access to Newspapers.com. Apologies if
this is being worked on without my knowledge... but nothing is changing and
I'm not hearing anything.
This is my situation: I go to https://www.newspapers.com/ and enter my
email and newspapers.com password to log in. I click on my account. The
subscription info is: "You're currently a Registered Guest. This free
subscription gets you access to search newspapers but you will not be able
to view the full page. Get a paid subscription for access." The account
info contains my email address and the fact that I have been a member since
February 20, 2015. I can search but can't see results. Instead I am invited
to "Upgrade to a Publisher Extra Subscription to view this page". I can get
access to all the clippings I have saved, the most recent of which was done
on August 26, 2025. This situation has not changed since at least November
10, 2025.

At the Wikipedia Library (
https://wikipedialibrary.wmflabs.org/users/my_library/) under
Newspapers.com it says that my access expires on January 12, 2026. On
December 1 I reapplied for access. The reapplication is listed as Pending,
not yet reviewed. An earlier reapplication that I made on November 10, 2025
(thinking that perhaps my access had been cut off prematurely) was refused
on November 16 as Invalid. So I assume this current one will also be
unsuccessful when and if it is reviewed.

How can I get access to my Newspapers.com account please?
Thanks!

Good morning,

Nothing has changed, either in my inability to access Newspapers.com or in
my reapplication for access at the Wikipedia Library. I am supposed to have
access until January 12, 2026 and have not had access since at least
November 10, 2025. I posted about this on Maniphest on November 30 and
reapplied for access on December 1. I last heard from you on December 1
when you sent me some troubleshooting questions, which I answered.

Is there anything I can do to get my access restored? Do I have to wait
until January 12, 2026? Is there something wrong with my account?

Good morning,

Nothing has changed, either in my inability to access Newspapers.com or in
my reapplication for access at the Wikipedia Library. I am supposed to have
access until January 12, 2026 and have not had access since at least
November 10, 2025. I posted about this on Maniphest on November 30 and
reapplied for access on December 1. I last heard from you on December 1
when you sent me some troubleshooting questions, which I answered.

Is there anything I can do to get my access restored? Do I have to wait
until January 12, 2026? Is there something wrong with my account?

We'll get back to you on your latest application soon, and work to get you access to Newspapers.com again. We're looking into a few accounts that should be working but aren't at the moment.

If I can also kindly ask - please stop commenting on this unrelated Phabricator ticket, as many users are subscribed and will be receiving notifications for each comment. If you'd like additional direct input from us I recommend either commenting on your TWL application or emailing us at wikipedialibrary@wikimedia.org. Thanks!

Thank you for the update. Sorry, I didn't realize the Phabricator ticket
was unrelated. I won't comment on it again.

Sorry, I was trying to respond to SamWalton9-WMF and didn't notice that
Phabricator was cc'd in the email.