Between node services, we have developed fairly standard solutions for common service issues:
We currently share these solutions manually, but it would be great if we could extract the common code into a template repository or skeleton initialization tool.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | • GWicke | T88585 Create a standard service template / init / logging / package setup | |||
| Resolved | • mobrovac | T89901 Create a standard puppet module for service-runner services | |||
| Declined | None | T89900 Auto-generate Debian packages for service-runner services | |||
| Resolved | • mobrovac | T90315 Create a vagrant puppet role for service-runner based services | |||
| Resolved | • mobrovac | T100202 Cloning Node.JS service repos into /vagrant/srv/ fail on Windows host |
Can basic testing setup be added to the template/skeleton project as well?
At $DAYJOB-1 we made a skeleton project and used Phing (our PHP build tool of choice) to make it easy to clone the skeleton and run a build command that prompted you for various things to change in the template files. Once it was done, you had a fully functional "hello world" app that included stub config files, logging, and a build script to run tests, create release packages and configure a local deployment.
+1 also I think it'd make sense to capture what metadata we need in a (per-repo?) file so templates and other artifacts can be regenerated at will
That sounds neat.
Lets not go over board with templating though. I think we should try hard to use shared modules with stable interfaces, and keep the actual template bit to an absolute minimum. This is pretty much what we have already been doing, but there are still some utilities that aren't in a convenient package yet.
+1 for metadata and tools over templates that are then hacked. For example, I think that for most node services we should be able to generate Debian packages using just information from package.json and perhaps some other metadata file (port defaults come to mind) as long as the entry points and wrappers are standardized.
We have started to package our existing tools around nodejs clustering, logging, statsd reporting, config loading in https://github.com/wikimedia/supervisoid. By establishing conventions for how a service is started and configured, this will let us build a generic packaging tool & service puppet module.
Change 190708 had a related patch set uploaded (by KartikMistry):
Enable jenkins for operations/debs/contenttranslation
A basic, preliminary v0 of the service template is available at https://github.com/d00rman/service-template-node. We plan to switch to swagger for driving the API route creation process, thus auto-documenting API routes, allowing us to visualise the API intricacies of each service more easily.
Now merged & slightly improved at https://github.com/wikimedia/service-template-node.
I think there is still some room to pare it down further to just the functionality needed for a simple but solid service with less than a handful of end points. Maybe it makes sense to add a 'simple' branch?
Hello,
I wondered if you ever considered using something like yeoman for creating a generic service template? e.g)
http://yeoman.io/authoring/index.html
Then rather than using the current service-template-node you could even go as far as to create new projects with routes etc.
It was just an idea/suggestion
Thanks
@Jcook, we considered it when we started work on this, but decided to go with a repository. So far that has worked well for us. One advantage over yeoman we have seen is better support for updating / rebasing services on top of a newer version of the template. Generators are more of a one-shot operation.
Granted, this is currently more important than it should be because the template includes so much functionality. I would like us to move more of it into libraries & minimizing the template itself.
Since the core of this task is done, I am going to resolve it. Lets continue the discussion about moving out more functionality into libraries in T144954.