Page MenuHomePhabricator

Application Security Review Request : service-runner replacement: @tchin/service-utils
Open, Needs TriagePublic

Description

Project Information

───────────────────────────────────────────────────────────────────────────────
Language                 Files     Lines   Blanks  Comments     Code Complexity
───────────────────────────────────────────────────────────────────────────────
TypeScript                  14      1703      141       215     1347        210
YAML                         1         1        0         0        1          0
───────────────────────────────────────────────────────────────────────────────
Total                       15      1704      141       215     1348        210
───────────────────────────────────────────────────────────────────────────────
Estimated Cost to Develop (organic) $36,963
Estimated Schedule Effort (organic) 3.93 months
Estimated People Required (organic) 0.84
───────────────────────────────────────────────────────────────────────────────
Processed 48989 bytes, 0.049 megabytes (SI)
───────────────────────────────────────────────────────────────────────────────

Description of the tool/project:
service-utils is a replacement for an outdated WMF NodeJS library/framework called service-runner. service-runner makes assumptions on infrastructure that doesn't exist anymore, has features that have been superseded by new infrastructure, and uses libraries that haven't been updated in half a decade.

service-utils extracts parts of the code in service-runner that are still useful with the objective of modernizing it over time.

Description of how the tool will be used at WMF:
It will be used for a NodeJS POC service in the Data Engineering team, and if successful, will be available for everyone at WMF to use if they want to migrate their NodeJS services away from service-runner.

Dependencies

List dependencies, or upstream projects that this project relies on.

From the package.json

"dependencies": {
  "@elastic/ecs-winston-format": "^1.5.3",
  "bunyan": "^1.8.15",
  "bunyan-syslog-udp": "^0.2.0",
  "c12": "^1.10.0",
  "gelf-stream": "^1.1.1",
  "lodash": "^4.17.21",
  "prom-client": "^15.1.0",
  "winston": "^3.13.0",
  "yargs": "^17.7.2"
},
"devDependencies": {
  "@types/bunyan": "^1.8.11",
  "@types/express": "^4.17.21",
  "@types/lodash": "^4.17.0",
  "@types/yargs": "^17.0.32",
  "express": "^4.19.2",
  "tsup": "^8.0.2",
  "vitest": "^1.4.0"
}

But some of these dependencies are for service-runner compatibility and will be removed when possible

"bunyan": "^1.8.15",
"bunyan-syslog-udp": "^0.2.0",
"gelf-stream": "^1.1.1",
"lodash": "^4.17.21"

Has this project been reviewed before?

Please link to tasks or wiki pages of previous reviews.

No

Working test environment

Please link or describe setup process for setting up a test environment.

To install it for use in a NodeJS project:

  1. Connect to the WMF GitLab registry

echo @tchin:registry=https://gitlab.wikimedia.org/api/v4/packages/npm/ >> .npmrc

  1. Install

npm i @tchin/service-utils

Post-deployment

Name of team responsible for tool/project after deployment and primary contact.

Data-Engineering and @tchin

Details

Risk Rating
Low

Event Timeline

tchin renamed this task from Application Security Review Request : service-runner replacement to Application Security Review Request : service-runner replacement: @tchin/service-utils.Wed, Apr 17, 1:10 PM

@tchin - Has this project been discussed across the WMF/Community? Especially with SRE, who would need to support deployments of services that will use this new template? I'm just trying to understand what kind of consensus exists for this being the de facto replacement for service-runner. I know we don't really have a functioning tech-decision-forum or RFC process at the moment (AIUI) but this seems like something that would be a good candidate for wider review.

Has this project been discussed across the WMF/Community?

It would be great if there was a RFC process, but there has at least been discussions about what to do with service-runner and this project is on the radar to the entirety of Data Platform Engineering and some people on the MW engineering team and the language team. It was also posted on slack on #engineering-all to give people a head's up just in case there was another team working on something similar. If there's one thing I'm sure about is that the consensus is that we need a replacement, whether or not this is it.

Especially with SRE, who would need to support deployments of services that will use this new template?

It's not a template, just a library. service-template-node (which uses service-runner) has been abandoned and has no maintainer. Data Products is actually making a new node service and decided against using the template but they still have to use service-runner. This is more like the NodeJS equivalent of servicelib-golang (but the name servicelib-node was already taken by a project that was abandoned when the team was re-orged). Data Platform SREs are helping with deploying the POC that uses our new replacement onto dse-k8s so we can see how it goes.

I'm really looking at this more like "DE is making a new node service and does not want to use service-runner so we will bear the cost of making a replacement for our needs. We coded the library to be easy for people to migrate from service-runner but there's no obligation"