Page MenuHomePhabricator

Request creation of fr-tech-dev VPS project
Closed, ResolvedPublic

Description

Project Name: fr-tech-dev

Wikitech Usernames of requestors: andyrussg, jgleeson

Purpose: Provide WMF managed internet-facing development server to assist Fundraising Tech in testing and developing payment integrations.

Brief description: Fundraising Tech develops integrations with payment processors. Several of these integrations include processes where the payment processors make requests to our systems.

During the development & testing phases, we need to receive traffic transmitted from the payment processors servers and redirect it to our local development environments to confirm the integrations work as intended. Historically, we have accomplished this using tools such as ngrok, temporarily exposing our local development environments to the internet on free tiers of the product. This approach isn't ideal as it requires us to transmit potentially sensitive integration data via a third party service. Also, with some payment processors, we must have fixed URLs and valid SSL/TLS certificates to receive traffic, which ngrok does not provide on the free tier.

To properly develop and test these integrations, we need a list of fixed WMF managed URLs running over HTTPS, which we can share with payment processors during the setup stages of the integration. Those addresses will point to a single cloud VM on which fundraising-tech developers can connect to and configure ssh tunnels to intercept any traffic received to be processed and tested locally. This project will receive very little traffic as the URLs will only be used during development and testing by engineers on Fundraising Tech.

We have deliberately kept this project outside the fundraising server cluster to allay any PCI concerns and remove the possibility of accidentally exposing donor data stored on the cluster.

How soon you are hoping this can be fulfilled: ASAP. Our latest integration with Apple Pay requires us to specify a list of URLs to generate certificates to so that we can receive payment traffic during testing. We're currently using AWS in place of a proper solution, but we'd like to keep all this traffic on WMF-managed infrastructure for the reasons mentioned above.

Event Timeline

Previous notes on setting something like this up on the Fundraising cluster.

Hello @AndyRussG .

If all you need is DNS records (under any domain) that resolve to something a dev can access, a cloud VM may be overkill for this. Have you already considered a dynamic DNS service? https://en.wikipedia.org/wiki/Dynamic_DNS

We typically discourage use of cloud services for proxying since it gobbles up bandwidth and can generally be accomplished better in other ways. Although this is only sort of proxying, it feels like it's on the line. https://wikitech.wikimedia.org/wiki/Wikitech:Cloud_Services_Terms_of_use#What_uses_of_Cloud_Services_do_we_not_like?

Let me know if I'm misunderstanding what you need.

I'm on WMCS clinic duty this week, but I wont be creating this project, waiting for @AndyRussG to followup on @Andrew clarification request above.

Thanks much @Andrew, @aborrero! We're working on updating the proposal--should be done soon! It's still a bit in flux. In summary, there are some things that are not super easy to set up on the FR cluster, which is why we thought a cloud VPS project would work best. Thanks again!

aborrero added a subscriber: dcaro.

This was approved in the WMCS team meeting! @dcaro will follow up soon.

Great news @aborrero!!! thanks for the update.

Project created and added both @jgleeson and @AndyRussG as admins.

Remember to add yourselves to the https://lists.wikimedia.org/mailman/listinfo/cloud-announce list to get any announcements.
Enjoy!

Change 724816 had a related patch set uploaded (by Jgleeson; author: Jgleeson):

[operations/puppet@production] ssh: Include custom sshd_config files.

https://gerrit.wikimedia.org/r/724816

Change 724816 merged by Muehlenhoff:

[operations/puppet@production] ssh: Puppetize GatewayPorts config option for sshd_config

https://gerrit.wikimedia.org/r/724816