Page MenuHomePhabricator

Deal with donatewiki Thank You page launching in apps
Open, HighPublic1 Estimated Story Points

Description

When donors give us money, we want to hide banners for them on wikiPedia.org subdomains. We previously were doing that on a thank you page at donate.wikiMedia.org by including <img> tags sourcing Special:HideBanners on wikiPedia.org. Browsers now reject third-party cookies so we started showing the Thank You page at donate.wikPedia.org (T251780).

Unfortunately mobile donors with apps installed are now seeing the app launch instead of a Thank You page, and are getting an error instead of the TY page content. This has prompted a few hundred duplicate donation attempts.

Possible fixes:

  • exclude donate.wikipedia.org and thankyou.wikipedia.org (see T259002) from opening in-app
  • (short term) show the TY page in app (needs fixes to get restbase working on donatewiki - T259309)

Event Timeline

Ejegg created this task.Jul 30 2020, 10:26 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Ejegg updated the task description. (Show Details)Jul 30 2020, 10:26 PM
JMinor added a subscriber: JMinor.Jul 30 2020, 11:04 PM

I'd prefer to make the Thanks page work in the app, but besides sounding like the harder alternative, I don't think it will fix your core problem of cookie access, as the cookie would be set on a transient in-app webkit, not Safari.

I think that means its best if we can exclude the sub-domain of donate.wikiPedia.org from the app-site association file on the servers.

If we can't exclude it we can redirect back to Safari, which is generally how we handle requests on non-WMF domains (like citation links). That would require an app release, which could happen in mid-september.

We'll also need to think about what happens with the people who are donating from the app originally, and if we can maybe flag them as donors on the native side, before pushing them to Safari. People often complain about being asked to donate again. I guess they still would get double asked on web and app, but at least we could exclude them from repeat app asks.

JMinor triaged this task as High priority.Jul 30 2020, 11:05 PM
JMinor added subscribers: Tsevener, JoeWalsh.

@JMinor Thanks for the feedback and sorry for the random request here. Does the android app need to implement the redirect too? Should there be individual tasks for iOS and Android?

Dbrant added a subscriber: Dbrant.Jul 31 2020, 4:52 PM

This can serve as the task for Android, too.
On Android we will need to redirect the user back to an external browser in the case of the donate. domain, similar to what we currently do for Special: pages.

Ejegg added a comment.Jul 31 2020, 6:48 PM

Thanks @JMinor ! If we can get the TY page working in the app for now, that would be a big improvement over showing donors an error page, even if it doesn't get them a banner-hide cookie. Hopefully that will just work when RESTbase is enabled for the canonical donate.wikimedia.org and a redirect is added for donate.wikiPedia.org.

Tsevener added a comment.EditedAug 4 2020, 10:09 PM

@Ejegg can we try resolving this via a different app site association file hosted at https://donate.wikipedia.org/apple-app-site-association? I'm hoping this way we can exclude the subdomain and prevent the app hijacking it altogether. We already have one hosted at the various language domains (https://en.wikipedia.org/apple-app-site-association, https://es.wikipedia.org/apple-app-site-association, for example) but this one would need to have different content like this:

{
	"applinks": {
		"details": [{
			"appIDs": ["AKK7J2GV64.org.wikimedia.wikipedia.developer",
					   "AKK7J2GV64.org.wikimedia.wikipedia.tfalpha",
					   "AKK7J2GV64.org.wikimedia.wikipedia.tfbeta",
					   "AKK7J2GV64.org.wikimedia.wikipedia.adhoc",
					   "AKK7J2GV64.org.wikimedia.wikipedia"],
			"components": [{
				"/": "*",
				"exclude": true
			}]
		}]
	}
}

I'm not certain that it will work (I used documentation here and here for referencing) but it's worth a try.

If this doesn't work I also suggest trying to get more specific with the path key value like /wiki/* if the thank you page is hosted behind a wiki path.

{
	"applinks": {
		"details": [{
			"appIDs": ["AKK7J2GV64.org.wikimedia.wikipedia.developer",
					   "AKK7J2GV64.org.wikimedia.wikipedia.tfalpha",
					   "AKK7J2GV64.org.wikimedia.wikipedia.tfbeta",
					   "AKK7J2GV64.org.wikimedia.wikipedia.adhoc",
					   "AKK7J2GV64.org.wikimedia.wikipedia"],
			"components": [{
				"/": "/wiki/*",
				"exclude": true
			}]
		}]
	}
}

I'm not sure where this file is held and how to add a new one. I'm happy to help out and/or do more digging if there are any problems.

Ejegg updated the task description. (Show Details)Aug 5 2020, 6:37 PM

@Ejegg for iOS can you take a look at @Tsevener suggestion, and question about sub-domain config? I think we have a solution but need some help for the deployment.

LGoto assigned this task to Ejegg.Aug 18 2020, 6:20 PM

Just checking up on this. Previous comments noted that some change needed to get out on a new iOS app version. Is this still on track for a specific release?

Hi @DStrine we never received a reply or confirmation of which option to pursue, so currently no change is scheduled. Please see Tsevener comment of Aug 4 for our last action.

Ejegg added a comment.Sep 14 2020, 9:19 PM

Thanks @JMinor ! I think we're going to be fine with opening the TY page in the app (assuming RESTbase is now working). I'll look into updating that file server-side too.

Pcoombe moved this task from Backlog to Apps on the Thank-You-Page board.Oct 9 2020, 6:13 PM
Ejegg added a subscriber: LGoto.Oct 16 2020, 3:51 PM

After a quick chat with @LGoto and @JMinor we decided it would be best to prevent the whole thankyou.wikipedia.org subdomain from opening in the app. @JMinor says for iOS it will be best to try editing the entitlements file first. I guess with Android it would have to be in the manifest.xml?

@Ejegg Cool, you can reference my notes from Aug 4th on preventing hijacking on a particular subdomain for iOS. Hopefully that will work but feel free to reach out if you run into any issues or have any questions.

And... it looks like we're already doing this on Android, too:
https://github.com/wikimedia/apps-android-wikipedia/blob/master/app/src/main/java/org/wikipedia/page/PageActivity.java#L372-L374

i.e. the app does launch momentarily, but then bounces the user right back to the external browser. Unfortunately there's no way to exclude a particular subdomain from our intent filter.

LGoto added a comment.Oct 27 2020, 6:21 PM

@Ejegg Looks like this is over to you, but do you need anything else from the apps teams?

Ejegg added a comment.Oct 27 2020, 6:24 PM

Thanks @LGoto, it looks like the only thing left to do is @Tsevener 's suggestion here: https://phabricator.wikimedia.org/T259312#6361482

I think we'll have to work with the main cluster ops team to get a non-wiki file like that onto donatewiki.

Ejegg added a comment.Oct 27 2020, 8:31 PM

@Dbrant I just noticed in the Android code that it was only bouncing 'donate.' subdomain links back out to the external browser. I just made a pull request in GitHub to do the same with 'thankyou.' subdomain links as well. Perhaps the 'thankyou.' subdomain also needs redirects in iOS?

@Dbrant I just noticed in the Android code that it was only bouncing 'donate.' subdomain links back out to the external browser. I just made a pull request in GitHub to do the same with 'thankyou.' subdomain links as well. Perhaps the 'thankyou.' subdomain also needs redirects in iOS?

Thanks for that! Merged.

DStrine set the point value for this task to 1.
Ejegg added a comment.Oct 28 2020, 6:26 PM

@Tsevener can you point me to the dev / ops person or group you worked with to get the apple-app-site-association file added to the main cluster for the encyclopedias?

@Ejegg this was all in place before my time, unfortunately, so I'm not sure on what team it would be. All I know is that it looks like the configuration file is located at the mediawiki-config repo - I see it under docroot/wikipedia.org after I clone.

I am looking into a way to fake it on my end just to be sure that response would work to hopefully save you some time. I'll let you know if I'm able to.

CDanis added a subscriber: CDanis.Thu, Nov 5, 5:39 PM

Sorry, there's a small mess here -- parts of the relevant behavior are specified in the Puppet repo (where Apache2 config files live), and parts are in the mediawiki-config repo.

As @Tsevener suggests, the right thing to do here is to write a patch against the mediawiki-config repo. Specifically, I think someone needs to copy the docroot/standard-docroot subdirectory to a new docroot/donate subdirectory, and then, create an apple-app-site-association file in that new tree. Then deploy that via the usual config patch deploy process.

Then, once deployed, the only thing left to do is to update the apache configs to point to the new docroot for both donate.wm and donate.wp, in this file in the puppet repo.

@Ejegg if you have the time for the above, I'm happy to review the patches and help with deploys. Otherwise we'll find someone on serviceops who can take this on.

Change 639601 had a related patch set uploaded (by Ejegg; owner: Ejegg):
[operations/mediawiki-config@master] Special docroot for thankyou.wp.org (and donate)

https://gerrit.wikimedia.org/r/639601

Change 639771 had a related patch set uploaded (by Ejegg; owner: Ejegg):
[operations/puppet@production] Special docroot for thankyou.wikipedia.org

https://gerrit.wikimedia.org/r/639771

CDanis added a comment.Fri, Nov 6, 5:59 PM

Patches look ready to go -- as discussed with @Ejegg we'll get this deployed Monday.

Change 640257 had a related patch set uploaded (by CDanis; owner: CDanis):
[operations/puppet@production] Add httpbb tests for apple-app-site-association magic URL

https://gerrit.wikimedia.org/r/640257

Change 640257 merged by CDanis:
[operations/puppet@production] Add httpbb tests for apple-app-site-association magic URL

https://gerrit.wikimedia.org/r/640257

Change 639601 merged by jenkins-bot:
[operations/mediawiki-config@master] Special docroot for thankyou.wp.org (and donate)

https://gerrit.wikimedia.org/r/639601

Mentioned in SAL (#wikimedia-operations) [2020-11-17T14:59:06Z] <cdanis@deploy1001> Synchronized docroot/thankyou: Special docroot for thankyouwiki T259312 d2a20ec57 (duration: 00m 55s)

Change 639771 merged by Giuseppe Lavagetto:
[operations/puppet@production] Special docroot for thankyou.wikipedia.org

https://gerrit.wikimedia.org/r/639771

Change 641438 had a related patch set uploaded (by CDanis; owner: CDanis):
[operations/puppet@production] Enable httpbb tests for thankyouwiki Apple app association

https://gerrit.wikimedia.org/r/641438

Change 641438 merged by Giuseppe Lavagetto:
[operations/puppet@production] Enable httpbb tests for thankyouwiki Apple app association

https://gerrit.wikimedia.org/r/641438

Joe added a subscriber: Joe.Tue, Nov 17, 4:56 PM

the apache change has been merged, and tested to work with the renewed httpbb test suite. It will be deployed everywhere in the next 30 minutes. Please confirm this resolves this issue.

Looks good on Android!

Pcoombe added a subscriber: Ppena.Thu, Nov 19, 7:10 PM

I don't have an iOS device, but @Ppena reports that thankyou.wikipedia.org still offers to open in the iOS app when loaded in Safari. Is that expected?

JMinor added a comment.EditedThu, Nov 19, 7:27 PM

It looks like the server side change wasn't effective for iOS. The thanks page now loads and works/looks correct (at least to my non-fundraiser eyes), but when coming from the app donation flow or following a deep link, it does look like the app is still perferred instead of Safari. This means we'll need to put a harder redirect in the app client. I'm not 100% sure we can have that in place before the fundraiser starts but will let you know asaik.

Okay, I've talked to the team and here are the two options we can try:

  1. We can redirect thankyou.wikipedia.org back to the app by forcing that page back to the users preferred browser (generally Safari). However, the user experience might be a bit wonky. Basically the app will open for a second or two and then pop back to the browser. We're also not 100% confident that you'll be able to set the cookie then, as the browser won't redirect back to the existing tab but open a new tab in the target browser. We're working up an internal build to document the actual experience for y'all (I'll try to upload a video), but the app being "in the flow" even if briefly is likely not something we can ever remove entirely.
  2. We can continue to try modifying the site association file to see if we can exclude subdomains successfully. However, this functionality is not documented by Apple, so there continues to be no guarantees there. Also we have no way to test this without making config changes in production.

@JMinor thanks for continuing to work in solutions. I just sent an email to loop more people in. As long as the users are seeing the page and not an error this isn't a blocker to the campaign. If you have time to help make a better experience that's great. However we don't want to you to feel rushed or that you need to make an app update for this. I just wanted to be clear that this isn't a blocker at this point.

BTW, re #2, I'm happy to deploy more patches to the site association file. The turnaround on that should be much faster going forward (since first we had to create a separate docroot and configure Apache to read from it).

Could we try copying the thankyou site file to https://thankyou.wikipedia.org/.well-known/apple-app-site-association? Per this document[1], it should be in a .well-known folder. If we're putting the file in the wrong place, I'd expect it to fail the other way (the app not swallowing our links), but might be worth trying.

[1] https://developer.apple.com/documentation/safariservices/supporting_associated_domains

Let's try copying the file so that it's accessible via https://thankyou.wikipedia.org/.well-known/apple-app-site-association, as well as https://thankyou.wikipedia.org/apple-app-site-association. While I'm unsure if this will fix things, it seems like that is the canonical spot it is supposed to be at.

Sure -- SRE will get that deployed, but we're going to wait until Monday
Nov 30th, in the interest of not making any changes right before the US
holiday weekend.

I don't really know what I'm talking about here, but I'm also wondering if
the format of the file has changed between iOS 13 and earlier versions,
judging by
https://developer.apple.com/documentation/safariservices/supporting_associated_domains
and also this developer forum thread:
https://developer.apple.com/forums/thread/129460