Design the technical architecture for MPIC
Open, Needs TriagePublic
Actions

Assigned To

Authored By

	Sfaci
	Wed, Apr 10, 5:06 PM

Description

Goal

At the beginning we decided to start working with service-template-node that is a scaffold for WMF NodeJS services based on service-runner. After starting the development we have found that the initial technical architecture we are working on is not the best regarding both the development and deployment.

At the same time we work on development, we are going to make an effort to find a better architecture (in terms of simplicity) because we think that we can speed up the development and even the deployment.

At this moment we are working using an architecture with two different services frontend and backend that is more complicated to develop, communicate and even deploy to k8s. And we also think that it's something that this tool doesn't need to have. Our proposal for this task is to migrate this architecture to a SSR express application where frontend and backend work as the same service which is easier to develop, containerize and deploy later to k8s.

AC

We have decided what's the more convenient architecture for MPIC
We have explored which libraries we need to work on the decided architecture and our service-template-node based on service-runner
The decided architecture can be containerized according to what we discuss at T361060: Setup k8s configuration and GitLab CI for MPIC
We have a POC that we can run properly

Details

Other Assignee: phuedx

Related Objects

Mentioned Here: T361060: Setup k8s configuration and GitLab CI for MPIC

Event Timeline

Sfaci created this task.Wed, Apr 10, 5:06 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptWed, Apr 10, 5:06 PM

Sfaci renamed this task from Exploring a new architecture for MPIC (a simpler one) to Design the technical architecture for MPIC.Wed, Apr 10, 5:07 PM

Sfaci moved this task from Sprint Backlog to In Process on the Data Products (Data Products Sprint 11) board.

Sfaci updated the task description. (Show Details)Wed, Apr 10, 5:24 PM

VirginiaPoundstone edited projects, added Data Products (Data Products Sprint 12); removed Data Products (Data Products Sprint 11).Fri, Apr 12, 4:20 PM

Sfaci moved this task from Sprint Backlog to In Process on the Data Products (Data Products Sprint 12) board.Mon, Apr 15, 10:06 AM

The following are what we considered before starting exploring for the most appropriate architecture for this project:

We think codex is the most appropriate design system for the project because it's the Wikimedia Design System
Because we are going to use Codex, we will need to work with Vue. Codex is also ready to be use as "CSS-only components" but there are some key components we need that only have Vue representation (Combobox, Dialog, ChipInput, for example)
MPIC will have a "pure" API endpoint (/api/v1/instruments) but we should consider this tool mainly as a web application focused on user interaction. It will have a login, cookie/session and form validation, . . .
Based on the expected number of users and the usage level of this tool, it wouldn't make sense to consider decoupling the backend and frontend for now. Performance is not a requirement at this moment. For the same reason, we should tend to discard rigid/big/opinionated web frameworks and any complicated architecture.
Simplicity for the deployment will be also considered
Once we saw that service-template-node and service-runner are both quite old, we considered upgrading possibilities as another key point.

After exploring several approaches taking the above into consideration, this is the outcome for each one:

Nuxt: This framework seems to be powerful (in fact too much for this project) and is also very opinionated. By choosing it we should learn first how it works and, somehow, we could end up being their prisoners. In addition to that, the webapp we have to develop is not too sophisticated so this kind of framework doesn't seem the best tool for this development.
Vue+SSR: After exploring this approach a while, it seems to be something similar to the previous one. The main reason to use SSR in a Vue project is performance and that’s not our main requirement. There is no reason to use Vue+SSR in this project.
Decoupled backend/frontend + Vue frontend: It's something doable with and without service-template-node and service-runner but more complicated to deploy. And, as we mentioned before, we don't need to decouple backend + frontend at this moment. We could discard it.
service-template-node + service-runner + Vue frontend served as static content: (Available here) Finally it ended up being something technically doable but service-template-node is mainly focused on service development and we are creating a web application. A lot of customization is needed to remove useless components/libraries/features that we don't need. In addition to that, some libraries are quite old and the upgrade process is difficult. It also seems that service-runner makes less sense if we consider that we are going to deploy this tool to Kubernetes. We also took into consideration some comments we found in this thread (https://wikimedia.slack.com/archives/C01R06P8D1B/p1711426702130909?thread_ts=1711374062.017299&cid=C01R06P8D1B) and some other comments where it's mentioned that existing services have been even patched to work as run as standalone process (one per pod) (https://wikimedia.slack.com/archives/C01R06P8D1B/p1711459887829859?thread_ts=1711374062.017299&cid=C01R06P8D1B)
Express NodeJS + Vue frontend served as static content (without service-runner): (Available here) So far this seems to be the best approach. We can have all the advantages and none of these disadvantages that we have mentioned for the service-runner approach. With only a bit of work we already have the features we needed from service-template-node and, for now, we can serve the Vue frontend build statically but we could decouple backend/frontend in the future easily if needed, serving the second one as a separate process (and even in a separate container if needed).

Sfaci moved this task from In Process to Code Review / Tech Input on the Data Products (Data Products Sprint 12) board.Mon, Apr 15, 6:59 PM

In T362268#9714944, @Sfaci wrote:

Express NodeJS + Vue frontend served as static content (without service-runner): (Available here) So far this seems to be the best approach. We can have all the advantages and none of these disadvantages that we have mentioned for the service-runner approach. With only a bit of work we already have the features we needed from service-template-node and, for now, we can serve the Vue frontend build statically but we could decouple backend/frontend in the future easily if needed, serving the second one as a separate process (and even in a separate container if needed).

To add to this ExpressJS and Vue have both already been reviewed by Security (and there should be risk owners for them).

+1, SSR is kind of a pain if done in fancier ways, but in this way you get a lot for free and it helps even reduce code. As a bonus the user gets a great experience.