Page MenuHomePhabricator

Split RESTBase in two services: storage service and API router/proxy
Open, NormalPublic0 Story Points

Description

Problem

RESTBase is a pretty complex piece of software. It currently:

  • serves as the main entry point for the public REST API
  • provides a large storage solution for caching and/or storing responses to REST API calls
  • acts as a storage solution for the back-end services powering the API

Serving all these functions not only makes RESTBase complicated to improve and operate, but it also limits severely the way other entities in the environment can interacts with its storage.

Solution

In order to decrease the complexity and allow the back-end services to use the storage layer directly (and therefore, more efficiently), we are going to separate RESTBase into two different services: one acting as the REST API router and proxy, and the other providing the appropriate storage semantics to both the API router and the existing RESTBase back-end services.

Internally, RESTBase is already pretty modularised, which eases making the split: there are two main routing hierarchies - /v1 and /sys. The former exposes the (public) REST API, while the latter is used internally only for business logic and storage-layer access. The idea is to have /v1 and the business-logic part of /sys in the front-end service, and the rest of /sys to reside in the back-end service and have it exposed to the WMF production environment. The back-end service will remain co-located with Cassandra on the physical nodes, while the front-end service will be running in Kubernetes .

Steps

This is a tracking ticket, so the actual solutions to each of the steps involved will be addressed and discussed on the relevant tickets. Here we are providing just a list of things that need to happen.

Related Objects

Event Timeline

mobrovac triaged this task as Normal priority.Apr 8 2019, 8:54 PM
mobrovac created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 8 2019, 8:54 PM
mobrovac updated the task description. (Show Details)Apr 11 2019, 8:29 PM
mobrovac updated the task description. (Show Details)Apr 12 2019, 9:20 PM
Joe added a subscriber: Joe.Apr 13 2019, 3:59 PM
fsero moved this task from Backlog to Goal tasks on the serviceops board.Jun 20 2019, 2:12 PM
akosiaris moved this task from Goal tasks to Backlog on the serviceops board.Jun 21 2019, 8:26 AM
daniel added a subscriber: daniel.Jul 11 2019, 1:43 PM

We should thin about how the RESTbase storage component relates to the ideas outlined in T227776: Generalize ParserCache into a generic service class for large "current" page-derived data.

WDoranWMF added a subscriber: WDoranWMF.

Note: This needs to be scheduled as work with SRE in order to complete.