Page MenuHomePhabricator

<Break-Fix> IFTTT integration: fix or sunset?
Closed, ResolvedPublic

Description

Request Status: New Request
Request Type: break-fix

Request Title: Understand why IFTTT integrations are broken and fix OR turn off the service

  • Request Description: in 2015, we soft-partnered with IFTTT to create a Wikipedia IFTTT channel with a bunch of applets to automate content delivery, e.g. get the word of the day sent to me as a text message, automatically update my Android wallpaper to the featured image of the day, etc. (https://ifttt.com/wikipedia). The applets were mostly built by Stephen LaPorte (in his volunteer capacity), a few WMF eng folks who have since left, and some volunteers. As of the last month, pretty much all of these applets are broken. Stephen no longer has the time to maintain these, but the channel is still there and it looks like we’re maintaining it when we’re not, which isn’t a great user experience.

The IFTTT team has some engineering resources available to help us triage and resolve the issues and have offered to put their technical folks in touch with someone on our side, but they don't have the capacity to fully support these integrations themselves.

Usage stats: 3 million users have connected to this service since launch. Though it looks like many have disconnected/stopped using it since then, there are still about 60K applet runs per day. Recent stats are skewed by the fact that most of the applets are broken, but I can try to get some more historical data from the IFTTT team if helpful.

  • Indicate Priority Level: Medium
  • Main Requestors: Maryana Pinchuk
  • Ideal Delivery Date: Since this is live and actively broken, ideally within the next few weeks to at least decide whether to prioritize fixing/maintaining or turn off the service so new people searching for Wikipedia integrations don't see a broken experience.
  • Stakeholders: Platform, Partnerships

Request Documentation

Document TypeRequired?Document/Link
Related PHAB TicketsYesT292376 T250363
Other LinksNoSee this blog post announcing the integrations here: https://diff.wikimedia.org/2015/07/14/wikipedia-recipes-with-ifttt/

Details

ReferenceSource BranchDest BranchAuthorTitle
toolforge-repos/ifttt!1work/bd808/py3mainbd808Update codebase for modern Python and Wikimedia needs
Customize query in GitLab

Event Timeline

As of the last month, pretty much all of these applets are broken.

Is this that they all broke last month (ie they all broke at the same time)?

Or that as of the last that anyone checked (ie a month ago) they were all seemingly broken?

Also, https://ifttt.com/wikipedia is a 404.

My best guess is that it's the Let's Encrypt root expiry that broke it. It's running on a Python 2 Jessie container.

在T294448#7488230中,@Reedy写道:

As of the last month, pretty much all of these applets are broken.

Is this that they all broke last month (ie they all broke at the same time)?

I have been using IFTTT and it will no longer work after 2021-09-30.

@Maryana would you mind putting me in contact with the IFTTT team with the goal of understanding what the latest usage was on Wikipedia-related IFTTTs?

@Reedy Sorry for the delay – I don't check Phab as much as I used to ;) Not sure when the integrations went down because no one on our end has been monitoring them very closely (but Jon Robson's guess is that, per Legoktm, Let's Encrypt is the issue). And I asked the IFTTT team to move our applets page (https://ifttt.com/wikipedia) into hidden mode while we're figuring out what to do here, so new users wouldn't stumble on it and try to use the broken integrations.

@sdkim will do!

Update from my side, I am reaching out to the IFTTT team to understand the cause of break in integration, what the impact was for users of these and a better understanding of usage.

DAbad renamed this task from IFTTT integration: fix or sunset? to <Break-Fix> IFTTT integration: fix or sunset?.Dec 20 2021, 4:19 PM

IFTTT got back with some insights on usage:

  • 35k daily active users* and 60k Applet runs per day before the incident
  • A breakdown of those 60k applets as of Sept 2021:
    • Articles added to a category: 615
    • Article of the day: 7108
    • Article receives high number of edits 48
    • New article was created: 17,801
    • New edit from specific user: 12
    • New edit to article in category: 0
    • New edit to specific article: 234
    • Picture of the day: 5817
    • Word of the day: 28562
    • Total: 60197
  • Wikipedia on IFTTT only supports read-only (no update/creation of articles)

Need to find time with @Slaporte to understand the cost of maintaining the integration and breaking cause.

*Daily active users are defined globally and not specific to any project or wiki

Work required to resolve the breaking change:

  1. If the service will continue to run on toolforge, the service must be upgraded from python2 to python3.
  2. The service currently leverages the legacy IFTTT API and should migrate to the new IFTTT API
  3. The service should move off of toolforge to a more reliable infrastructure

Removing inactive task assignee.

I have a fork of https://github.com/slaporte/ifttt at https://github.com/bd808/toolforge-ifttt/tree/work/bd808/py3 which has updated the codebase to be compatible with python 3.9. I have this codebase running on Toolforge at https://ifttt-bd808.toolforge.org/. The service answers the curl -d "" -X POST https://ifttt-bd808.toolforge.org/v1/test/setup request.

If @Slaporte, @ori, or anyone else who has poked at the app in the past knows what else I can do to test this version of the app I would love to hear about it.

If @Slaporte, @ori, or anyone else who has poked at the app in the past knows what else I can do to test this version of the app I would love to hear about it.

I spent some more time with the fork and now have it working as validated by the /v1/test/setup test descriptions. To get things back to a fully functional state I had to remove the IFTTT triggers for "New edit with hashtag" and "Most edited articles". The hashtag triggers require access to a database that has moved from ToolsDB to a separate Cloud VPS project. See T250363#8608260 for my notes on what it would take to restore this feature. The trending articles trigger was built to use https://github.com/jdlrobson/wikipedia-trending-endpoint at the URL https://wikipedia-trending.wmflabs.org. That service is returning a 502 error indicating that the backend service is no longer running. If the service is restored to operation it should be possible to restore this feature.

Things that have been changed in the https://github.com/bd808/toolforge-ifttt/tree/work/bd808/py3 fork:

With some help from the current maintainers of ifttt on Toolforge this updated codebase could be deployed. Configuration at IFTTT would also need to be updated to remove configuration on their end for the removed triggers.

Thank you for your work on this @bd808! I've updated the CWS 2023 results table to show there's some progress here :-)

Courtesy ping for maintainers:

I checked out the github for the trending service and it's still working - it would just need to be hosted in a reliable way. Previously I didn't do a good job of this as I've found CloudVPS a bit of a pain for running Node.js sites. Ive updated the README with setup instructions: https://github.com/jdlrobson/wikipedia-trending-endpoint#setting-up-server

I checked out the github for the trending service and it's still working - it would just need to be hosted in a reliable way. Previously I didn't do a good job of this as I've found CloudVPS a bit of a pain for running Node.js sites. Ive updated the README with setup instructions: https://github.com/jdlrobson/wikipedia-trending-endpoint#setting-up-server

I've created a toolforge tool (and added you/@bd808) to replace the WMCS VPS being used for wikipedia-trending.wmflabs.org — the endpoint is now https://wp-trending.toolforge.org

the endpoint is now https://wp-trending.toolforge.org

Nice. It looks like I need to make a few security fixes... Would this repo autoupdate via cronjob (e.g. git pull && npm install) or would I need to ping you every time I do that @TheresNoTime ?

Ping :) this feels very close to done so would be good to wrap this up! What's next and who needs to do that?

Ping :) this feels very close to done so would be good to wrap this up! What's next and who needs to do that?

Basically at this point we need someone with existing credentials to either turn them over or to add @bd808, @TheresNoTime, and/or @Jdlrobson to the Toolforge tool and the IFTTT integration so that we can move forward.

I have never heard back from @Slaporte, @ori, or anyone else in response to pings here. In theory any of the folks on https://toolsadmin.wikimedia.org/tools/id/ifttt who were pinged in T294448#8684498 may have the information needed to make updates on the IFTTT side of things.

Maybe @Maryana or @DAbad can help? I'm personally avoiding using WMF internal Slack pings or emails to get attention deliberately, but if someone else wanted to shake other parts of the tree for attention I wouldn't turn it down.

bd808 changed the task status from Open to In Progress.May 8 2023, 11:20 PM
bd808 claimed this task.

Thanks to help from @Jdlrobson, @Slaporte, and @Maryana I now have access to both the Toolforge tool account and the ifttt.com "wikipedia" organization account that will be needed to finish up deployment and configuration of the updated app. I currently plan to continue work on this during Wikimedia-Hackathon-2023.

Mentioned in SAL (#wikimedia-cloud) [2023-05-25T17:01:48Z] <wm-bot> <bd808> Update to 3653758 (T294448)

During and after the hackathon I have continued to work on this project. I have been able to restore both the "New edit with hashtag" and "Most edited articles" triggers that I reported removing in T294448#8608277.

About 20 hours ago I submitted the configuration for the service for review upstream at IFTTT so that it can be re-enabled.

About 20 hours ago I submitted the configuration for the service for review upstream at IFTTT so that it can be re-enabled.

I am still (slowly) working through some configuration issues with IFTTT support staff. I have just now resubmitted for review following a round of what I hope will be the final changes.

Even after this most recent round of updates there are some unexplained changes that show on the review tool which diffs the old and new configuration. I have asked support for help either fixing these or understanding why the diff tool is showing changes that do not match the configuration screens. The unexplained diffs are missing {{...}} template data markers in the "applet defaults" for both the /new_hashtag and /trending_topics endpoints.

https://ifttt.com/wikipedia should be live to the world again!

There is now a Tool-ifttt tag that can be used to report issues with the service. It would be nice to have some more folks who are interested in keeping this running join me as maintainers of the tool. I will try to remember to put a call for help out on wikitech-l and/or the cloud mailing lists.

The error documented at T342014: Frequent "UnicodeError: label empty or too long" errors processing /ifttt/v1/triggers/word_of_the_day is happening quite frequently at the moment. I will try to look into it soon.

@TheresNoTime would you do the honors of updating the Wishlist?