As of the last month, pretty much all of these applets are broken.

Is this that they all broke last month (ie they all broke at the same time)?

Or that as of the last that anyone checked (ie a month ago) they were all seemingly broken?

Also, https://ifttt.com/wikipedia is a 404.

My best guess is that it's the Let's Encrypt root expiry that broke it. It's running on a Python 2 Jessie container.

在T294448#7488230中，@Reedy写道：

As of the last month, pretty much all of these applets are broken.

Is this that they all broke last month (ie they all broke at the same time)?

I have been using IFTTT and it will no longer work after 2021-09-30.

@Maryana would you mind putting me in contact with the IFTTT team with the goal of understanding what the latest usage was on Wikipedia-related IFTTTs?

@Reedy Sorry for the delay – I don't check Phab as much as I used to ;) Not sure when the integrations went down because no one on our end has been monitoring them very closely (but Jon Robson's guess is that, per Legoktm, Let's Encrypt is the issue). And I asked the IFTTT team to move our applets page (https://ifttt.com/wikipedia) into hidden mode while we're figuring out what to do here, so new users wouldn't stumble on it and try to use the broken integrations.

@sdkim will do!

Update from my side, I am reaching out to the IFTTT team to understand the cause of break in integration, what the impact was for users of these and a better understanding of usage.

IFTTT got back with some insights on usage:

• 35k daily active users* and 60k Applet runs per day before the incident
• A breakdown of those 60k applets as of Sept 2021:
  • Articles added to a category: 615
  • Article of the day: 7108
  • Article receives high number of edits 48
  • New article was created: 17,801
  • New edit from specific user: 12
  • New edit to article in category: 0
  • New edit to specific article: 234
  • Picture of the day: 5817
  • Word of the day: 28562
  • Total: 60197
• Wikipedia on IFTTT only supports read-only (no update/creation of articles)

Need to find time with @Slaporte to understand the cost of maintaining the integration and breaking cause.

*Daily active users are defined globally and not specific to any project or wiki

Work required to resolve the breaking change:

1. If the service will continue to run on toolforge, the service must be upgraded from python2 to python3.
2. The service currently leverages the legacy IFTTT API and should migrate to the new IFTTT API
3. The service should move off of toolforge to a more reliable infrastructure
   • Codebase: https://github.com/slaporte/ifttt

Submitted to Community Wishlist Survey 2023: https://meta.wikimedia.org/wiki/Community_Wishlist_Survey_2023/Miscellaneous/Fix_Wikipedia_IFTTT_integration

I have a fork of https://github.com/slaporte/ifttt at https://github.com/bd808/toolforge-ifttt/tree/work/bd808/py3 which has updated the codebase to be compatible with python 3.9. I have this codebase running on Toolforge at https://ifttt-bd808.toolforge.org/. The service answers the curl -d "" -X POST https://ifttt-bd808.toolforge.org/v1/test/setup request.

If @Slaporte, @ori, or anyone else who has poked at the app in the past knows what else I can do to test this version of the app I would love to hear about it.

In T294448#8606822, @bd808 wrote:

If @Slaporte, @ori, or anyone else who has poked at the app in the past knows what else I can do to test this version of the app I would love to hear about it.

I spent some more time with the fork and now have it working as validated by the /v1/test/setup test descriptions. To get things back to a fully functional state I had to remove the IFTTT triggers for "New edit with hashtag" and "Most edited articles". The hashtag triggers require access to a database that has moved from ToolsDB to a separate Cloud VPS project. See T250363#8608260 for my notes on what it would take to restore this feature. The trending articles trigger was built to use https://github.com/jdlrobson/wikipedia-trending-endpoint at the URL https://wikipedia-trending.wmflabs.org. That service is returning a 502 error indicating that the backend service is no longer running. If the service is restored to operation it should be possible to restore this feature.

Things that have been changed in the https://github.com/bd808/toolforge-ifttt/tree/work/bd808/py3 fork:

• Codebase converted from Python2 (sunset upstream since January 1, 2020) to Python3 (3.9 specifically)
• 3rd-party libraries upgraded
• Wiki replica queries updated for T167246: Refactor "user" & "user_text" fields into "actor" reference table
• Wiki replica queries updated for T166732: Refactor comment storage in the database and abstract access in MediaWiki
• Removed "New edit with hashtag" trigger. See T250363#8608260
• Removed "Most edited articles" trigger. See https://www.mediawiki.org/wiki/Reading/Web/Projects/Edit_based_trending_service

With some help from the current maintainers of ifttt on Toolforge this updated codebase could be deployed. Configuration at IFTTT would also need to be updated to remove configuration on their end for the removed triggers.

Thank you for your work on this @bd808! I've updated the CWS 2023 results table to show there's some progress here :-)

Courtesy ping for maintainers:

• @bearND (phab account disabled)
• @Jdlrobson
• @MahmoudHashemi
• @ori
• @Slaporte

I checked out the github for the trending service and it's still working - it would just need to be hosted in a reliable way. Previously I didn't do a good job of this as I've found CloudVPS a bit of a pain for running Node.js sites. Ive updated the README with setup instructions: https://github.com/jdlrobson/wikipedia-trending-endpoint#setting-up-server

In T294448#8696051, @Jdlrobson wrote:

I checked out the github for the trending service and it's still working - it would just need to be hosted in a reliable way. Previously I didn't do a good job of this as I've found CloudVPS a bit of a pain for running Node.js sites. Ive updated the README with setup instructions: https://github.com/jdlrobson/wikipedia-trending-endpoint#setting-up-server

I've created a toolforge tool (and added you/@bd808) to replace the WMCS VPS being used for wikipedia-trending.wmflabs.org — the endpoint is now https://wp-trending.toolforge.org

the endpoint is now https://wp-trending.toolforge.org

Nice. It looks like I need to make a few security fixes... Would this repo autoupdate via cronjob (e.g. git pull && npm install) or would I need to ping you every time I do that @TheresNoTime ?

Ping :) this feels very close to done so would be good to wrap this up! What's next and who needs to do that?

In T294448#8814227, @Jdlrobson wrote:

Ping :) this feels very close to done so would be good to wrap this up! What's next and who needs to do that?

Basically at this point we need someone with existing credentials to either turn them over or to add @bd808, @TheresNoTime, and/or @Jdlrobson to the Toolforge tool and the IFTTT integration so that we can move forward.

I have never heard back from @Slaporte, @ori, or anyone else in response to pings here. In theory any of the folks on https://toolsadmin.wikimedia.org/tools/id/ifttt who were pinged in T294448#8684498 may have the information needed to make updates on the IFTTT side of things.

Maybe @Maryana or @DAbad can help? I'm personally avoiding using WMF internal Slack pings or emails to get attention deliberately, but if someone else wanted to shake other parts of the tree for attention I wouldn't turn it down.

bd808 opened https://gitlab.wikimedia.org/toolforge-repos/ifttt/-/merge_requests/1

Draft: Update codebase for modern Python and Wikimedia needs

bd808 merged https://gitlab.wikimedia.org/toolforge-repos/ifttt/-/merge_requests/1

Update codebase for modern Python and Wikimedia needs

Mentioned in SAL (#wikimedia-cloud) [2023-05-21T08:23:42Z] <wm-bot> <bd808> Update to aad850d (T294448)

Mentioned in SAL (#wikimedia-cloud) [2023-05-21T08:56:49Z] <wm-bot> <bd808> Update to a4c8e30 (T294448)

Mentioned in SAL (#wikimedia-cloud) [2023-05-25T17:01:48Z] <wm-bot> <bd808> Update to 3653758 (T294448)

During and after the hackathon I have continued to work on this project. I have been able to restore both the "New edit with hashtag" and "Most edited articles" triggers that I reported removing in T294448#8608277.

About 20 hours ago I submitted the configuration for the service for review upstream at IFTTT so that it can be re-enabled.

In T294448#8911378, @bd808 wrote:

About 20 hours ago I submitted the configuration for the service for review upstream at IFTTT so that it can be re-enabled.

I am still (slowly) working through some configuration issues with IFTTT support staff. I have just now resubmitted for review following a round of what I hope will be the final changes.

Even after this most recent round of updates there are some unexplained changes that show on the review tool which diffs the old and new configuration. I have asked support for help either fixing these or understanding why the diff tool is showing changes that do not match the configuration screens. The unexplained diffs are missing {{...}} template data markers in the "applet defaults" for both the /new_hashtag and /trending_topics endpoints.