Page MenuHomePhabricator

Request creation of futureaudiences VPS project
Closed, DeclinedPublic

Description

Project Name: futureaudiences

Developer account usernames of requestors:

Purpose: Future Audiences game server

Brief description:

The FA team has a long history of hosting its tools on toolforge (we have 11 tools hosted there now). This has often worked well but several times we have run into limitations which have limited our development velocity. The largest limitation has been our inability to host/utilize open source software such as 1) neo4j 2) posthog 3) ffmpeg 4) headless chrome. Sometimes we've been able to overcome these limitations after much effort.

We are currently working on a new project that would benefit from a live queryable neo4j service (with a DB size > 50GB). Currently we are hosting this service on developer laptops but this is untenable long term.

The project in question is a Roblox game in development as an experiment. Several people have requested features for this game which require us to host a persistent graph db.

I anticipate that near term only FA team members will have access to this cluster.

How soon you are hoping this can be fulfilled: next couple weeks?

Event Timeline

Hi!

CloudVPS project should be project-based not team-based, maybe the name should be something like fagameserver?

A couple of side notes (just fyi., you still need cloudvps for neo4j):

local.tf-test@lima-kilo:~$ toolforge build start https://gitlab.wikimedia.org/dcaro/toolforge-chromium-headless-test

local.tf-test@lima-kilo:~$ toolforge jobs run --command run --image tool-tf-test/tool-tf-test:latest test1

local.tf-test@lima-kilo:~$ toolforge jobs logs -f test1
2025-03-18T17:08:15+00:00 [test1-2thks] 
2025-03-18T17:08:15+00:00 [test1-2thks] > toolforge-chromium-headless-test@1.0.0 run
2025-03-18T17:08:15+00:00 [test1-2thks] > node test.js
2025-03-18T17:08:15+00:00 [test1-2thks] 
2025-03-18T17:08:20+00:00 [test1-2thks] Portal:Toolforge - Wikitech       <- this was fetched from the chromium browser page navigating to toolforge.org
2025-03-18T17:08:21+00:00 [test1-2thks] npm notice 
2025-03-18T17:08:21+00:00 [test1-2thks] npm notice New major version of npm available! 10.2.3 -> 11.2.0
2025-03-18T17:08:21+00:00 [test1-2thks] npm notice Changelog: <https://github.com/npm/cli/releases/tag/v11.2.0>
2025-03-18T17:08:21+00:00 [test1-2thks] npm notice Run `npm install -g npm@11.2.0` to update!
2025-03-18T17:08:21+00:00 [test1-2thks] npm notice
dcaro changed the task status from Open to In Progress.Mar 18 2025, 6:07 PM
dcaro claimed this task.

Hey thanks for the response.

I was thinking we could keep the name generic in case this server becomes useful for multiple projects.

Yeah we did eventually get ffmpeg to work. And getting that to work saved our bacon. But the performance of it on toolforge remains much worse than on our dev laptops and projects that use it take a very long time to build. We have very similar issues with chrome headless. It runs but it takes a long time to build and then stutters when in use (this caused a bunch of pain as things would work fine on our laptops but then not work in toolforge).

More generally we expect to regularly need these kinds of unsupported/demanding tools.

I was thinking we could keep the name generic in case this server becomes useful for multiple projects.
...
I anticipate that near term only FA team members will have access to this cluster.

https://wikitech.wikimedia.org/wiki/Help:Cloud_VPS_project#Reviews_of_Cloud_VPS_Project_requests states:

The majority of project requests are approved, but if your request involves any of the following items, it may require further discussion:
Intentionally limiting membership to a closed group.

There are valid reasons for a project to limit participants (sensitive data stored/transmitted, infrastructure for larger initiatives, etc). Because Cloud VPS administrators prefer to dedicate resources to inclusive projects, projects that limit membership require additional scrutiny.

"Umbrella" projects with a broad scope, such as all the work to be done by an engineering team or a large problem space.

"Umbrella" projects with broad scopes are difficult to track over time because of organizational changes and lack of continuity in ownership.

Open projects with a clear scope are much more likely to either succeed or to be able to reason about shutting down when things are not working out as hoped (or the work is complete). They are also generally more likely to be responsive to requests from Cloud VPS admins for special work like operating system upgrades and annual usage audits. Once you have a Developer account it is straightforward to join multiple Cloud VPS projects. It is also typically relatively quick to go through the process of requesting a new project; the main thing that causes slow downs is asking for things that are outside of the typical project parameters thus triggering discussion about why this work is special and how we might create compensating controls.

Yes I had read that before making this application but it sounded those were not necessarily hard rules.

Given FA's velocity and diverse set of projects it seems unlikely anyone outside would be able to effectively collaborate (at least in the near term). Though this isn't a hard rule.

It was my understanding that requesting a new project could take several weeks.

I hoped that the fact that we have multiple contactable full-time PMs/Engineers that this would ameliorate the concerns regarding responsiveness.

If this can't be made to happen we could rename the service and restrict its scope as you suggest.

Yes I had read that before making this application but it sounded those were not necessarily hard rules.

Given FA's velocity and diverse set of projects it seems unlikely anyone outside would be able to effectively collaborate (at least in the near term). Though this isn't a hard rule.

It was my understanding that requesting a new project could take several weeks.

This will depend, if you request a very specific project without many extra resources should be fairly easy (<1 week worst case, if you ping on irc you might get it reviewed faster).

I hoped that the fact that we have multiple contactable full-time PMs/Engineers that this would ameliorate the concerns regarding responsiveness.

If this can't be made to happen we could rename the service and restrict its scope as you suggest.

I would go for the more specific scope yes, even inside the foundation team names change, reorgs happen, etc.

So what would be the name you want for the project then?

@derenrich are you still interested in a Cloud VPS project with a more restricted scope?

hey sorry for the delay this slipped between the cracks. yeah given the restrictions on using Cloud VPS I think we'll pass on using it for this project. we'll revisit the question for our next project though many of the same issues will remain.