Page MenuHomePhabricator

[M] Implement proxy configuration for kubernetes deployment
Closed, ResolvedPublic

Description

Daily updates in ipoid will require a curl request to the outside world (spur.us) to download a dataset. AIUI, these requests need to go through a proxy. In this task, we'll set up the Kubernetes configuration to proxy requests to outside network and adjust the application to use a proxy, if configured.

acceptance criteria

  • ipoid app uses PROXY_HOST and PROXY_PORT as proxy configuration with axios, when set
  • Kubernetes config has workable values for PROXY_HOST and PROXY_PORT

Details

TitleReferenceAuthorSource BranchDest Branch
Rename HTTP_PROXY to HTTPS_PROXYrepos/mediawiki/services/ipoid!152tchandershttps-proxymain
get-feed: Support using a proxy with axiosrepos/mediawiki/services/ipoid!144kharlanT349171-9634main
Customize query in GitLab

Event Timeline

Change 966867 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[operations/deployment-charts@master] [WIP] ipoid: Set PROXY_HOST and PROXY_PORT

https://gerrit.wikimedia.org/r/966867

Change 966867 abandoned by Kosta Harlan:

[operations/deployment-charts@master] [WIP] ipoid: Set PROXY_HOST and PROXY_PORT

Reason:

https://gerrit.wikimedia.org/r/966867

@Tchanders please use the environmental variable HTTPS_PROXY (vs HTTP_PROXY), as is the convention we use in the repo, along with https_proxy, when applications need to reach a url via https. Additionally, no ENV variable is needed for the proxy's port, as this is included in the URI we provide to the ENV.

This is a bit on me, I wanted to comment on the related patch, but failed to, I am terribly sorry

Thanks @jijiki .

I've put up a patch renaming HTTP_PROXY to HTTPS_PROXY throughout the repo.

Re: the port, do you mean that the port is included in the HTTPS_PROXY variable, or that it's in a separate variable? If the first, then we don't need to update anything. If the second, then I think we need to know the name of that variable so we can update get-feed to use it:

const proxyUrl = new URL( process.env.HTTPS_PROXY );
axiosParams.proxy = {
	host: proxyUrl.hostname,
	port: proxyUrl.port // <---- needs to reference the variable containing the port, if it's a separate variable
};
Tchanders renamed this task from Implement proxy configuration for kubernetes deployment to [M] Implement proxy configuration for kubernetes deployment.Nov 1 2023, 6:51 PM
kostajh added a subscriber: dom_walden.

I think we will QA this by testing it in production, so skipping the QA column and moving to Done. (cc @dom_walden)

Thanks @jijiki .

I've put up a patch renaming HTTP_PROXY to HTTPS_PROXY throughout the repo.

Re: the port, do you mean that the port is included in the HTTPS_PROXY variable, or that it's in a separate variable? If the first, then we don't need to update anything. If the second, then I think we need to know the name of that variable so we can update get-feed to use it:

const proxyUrl = new URL( process.env.HTTPS_PROXY );
axiosParams.proxy = {
	host: proxyUrl.hostname,
	port: proxyUrl.port // <---- needs to reference the variable containing the port, if it's a separate variable
};

@Tchanders looking in operations/deployment-charts repo, the HTTPS_PROXY variable will be http://url-downloader.eqiad.wikimedia.org:8080, so, as you say, we shouldn't need to do anything else here.