Page MenuHomePhabricator

Refactor shared code out of node template
Closed, DuplicatePublic

Description

Kartotherian by now has became (first?) node-template based service used outside of WMF. In order to help develop it as a service useful to both inside and outside the foundation, it needs to be cleanly separated separated from some of the WMF-specific code, especially inside the service template. I would like to move some of the code out of the node template, but could really use service's team help to figure out how to best partition it. The major side benefit would be a much slimmer node-template that can be easily updated to the newer version of the utility libraries.

First step - lets figure out what should be moved out into separate library(ies). I identified these blocks, but there might be more, or these might need to be broken up further. I propose we move all of them into service-templatelib-node (naming?).

app.js

  • initApp() global functions (metrics, logging, conf, info)
  • initApp() app.conf initialization
  • initApp() express initialization (cors, csp, global flags, compression, parsing, ..
  • loadRoutes() auto-discovery based on files
  • createServer()

api-utils

  • setupApiTemplates
  • basic api functionality

swagger-ui

  • ?

routes

  • root
  • info
  • ex

Event Timeline

Out of your list there's a couple of candidates that I immediately agree with and I propose to start with them and see where that brings us:

  1. Swagger-UI We have a VERY similar file in the hyper switch, so I could imaging that becoming it's own module that's sole purpose is to statically serve swagger-ui files in a way we do that and reusing the module in hyper switch.
  2. API templates Similarly, I could imagine that becoming it's own module with a collection of all the API Wikimedia templates.

Others are a bit more controversial. Let's say I could imagine extracting all the CORS-related parts to a separate module just for handling CORS or reusing some existing solution (then we can reuse it in RESTBase and other places requiring CORS), but following that approach would lead to a collection of tiny modules that I personally don't quite like in the node world..

@Pchelolo I agree that many of these things should not be their own modules. On the other hand, I don't think we should keep it as part of a copy/paste template. Instead, lets move them all into a "template utility" lib, which would allow existing template instants to simply reference it, and would allow you to improve them and migrate to the new versions in a more controlled way.

Out of your list there's a couple of candidates that I immediately agree with and I propose to start with them and see where that brings us:

  1. Swagger-UI We have a VERY similar file in the hyper switch, so I could imaging that becoming it's own module that's sole purpose is to statically serve swagger-ui files in a way we do that and reusing the module in hyper switch.
  2. API templates Similarly, I could imagine that becoming it's own module with a collection of all the API Wikimedia templates.

Agreed for swagger-ui, but the idea for API templates might be a bit too much. It's just one tiny file, so I'm not sure it warrants its own module, really.

Others are a bit more controversial. Let's say I could imagine extracting all the CORS-related parts to a separate module just for handling CORS or reusing some existing solution (then we can reuse it in RESTBase and other places requiring CORS), but following that approach would lead to a collection of tiny modules that I personally don't quite like in the node world..

I completely agree. Having a ton of small moving parts doesn't help the cause.

@mobrovac, I am not proposing to make 100 tiny npm libs. I am suggesting that we make a few "template libraries"., and move all the code there. This way services based on the template can be easily upgraded to the newer versions. Right now, there is no way to merge, without a very tedious line-by-line conflict resolution.

The template is a copy-paste, and as such, it gets heavily customized for the specific usecase, without any expectation of later updates from the "base". Common code should live in library.

FTR, I agree with the general direction of moving the (vast majority of the) code template users are not expected to modify to one or more separate modules. I think this would more clearly document expectations, which in turn will help avoid merge conflicts during upgrades.

Of the listed candidates, routes seem to be the least obvious to fit this description. Users are expected to write similar routes or modify existing ones, so it seems that keeping them in the template might make more sense.

...which in turn will help avoid merge conflicts during upgrades.

I don't think there should ever be a need for merging any service-template changes with the actual services. That's what I would like to avoid with this restructuring.

Of the listed candidates, routes seem to be the least obvious to fit this description. Users are expected to write similar routes or modify existing ones, so it seems that keeping them in the template might make more sense.

Agree for v1 -- it is clearly an example, but I think the _info and robot file generating ones should be part of the lib, as they are mostly boilerplate and do not require any changes.

GWicke triaged this task as Medium priority.Aug 8 2017, 10:24 PM