Data-Persistence is working on an MVP of a shared service for the persistent caching of outputs which correspond to MediaWiki page content (aka linked artifacts). See the (work-in-progress) proposal for more information. The first use-case of this shared service is T402984: Data Persistence Design Review: Article topic model caching. The rollout plan (copy-and-paste from the proposal document):
Normally a proposal like this is discussed at length with stakeholders and subject matter experts. The process results in feedback, feedback in changes, and consensus is developed. Proposals that move forward are resourced, and planned for accordingly. For this proposal, we would like to fast track an MVP deployment to production, and to do so as the broader discussion unfolds.
This proposal came too late to accommodate tone checking, leaving any eventual migration of it (or other legacy systems) as an article of unresolved debt. There exists however a moment of opportunity for article topics, provided we don’t unnecessarily block the team’s work. Doing this carries some risk that the ensuing discussion will result in changes that require double-handling or rework. At the extreme end of this, discussion could lead to consensus that we should not do this at all, in any form. In that case, what we do here risks becoming a bespoke one-off for article topics —a dead-end— but that is precisely what happens if we do nothing. Provided that we do not proceed with any additional use-cases until the discussion concludes, and consensus is reached, these risks seem manageable.
TL;DR We’re trying to get a foot in the metaphorical door before another bespoke implementation adds to our mounting support costs, while putting in place guardrails to minimize risk, and ensure a full review and discussion.
The software is called Hoarde, and its source repo is: https://gitlab.wikimedia.org/repos/sre/hoarde.
Ideally we'd keep the software & service names disjoint, as we did for Kask (software), and sessionstore and echostore (services). Possible ideas for the service name are: linked-artifacts, linked-cache, linked-artifact-cache, artifact-cache, etc; Bikeshedding of service names welcome!
The service endpoint will need to be accessible from within the production network, and should presumably use the service mesh (it will be accessed from MW extensions). It will need to connect to Cassandra (the RESTBase cluster) for storage, and to make connections to configured "lambdas" (services running in k8s, including MLs k8s cluster).
See also: T402984: Data Persistence Design Review: Article topic model caching
