Page MenuHomePhabricator

Lightweight preview environment for gerrit changes
Closed, ResolvedPublic

Description

roughly, the idea would be:

  • Have an application that lives on WMCS or on Toolforge
  • Listen for comments (e.g. "stage this") from whitelisted contributors in gerrit
    • If on WMCS, find a free port or use nginx-proxy/traefik, use docker run to run a Quibble image, without running tests, and probably just with a SQLite DB, and with an entrypoint command that will keep the container running for 30 minutes, then post a comment to gerrit with the URL where the staged changes can be reviewed. We probably don't want to provide the admin password or allow file uploads, but maybe it doesn't matter if a malicious user gets access to the shell since quibble image doesn't use a root user?
    • If on Toolforge, run quibble without Docker, using SQLite as the backend DB; find a free port, and create a directory based on the gerrit patch short number, so the URL for accessing the staged site would be something like tools.wmflabs.org/{name-of-the-project}/{gerrit-short-change-id}:{port}. We would need some scripts to shut down the PHP built-in web server and clean up the directory after 30 minutes.

We'd probably also want a tool that can generate a .env file for quibble's use based on a gerrit change. That could probably be a command baked into quibble (quibble --generate-env {url-to-gerrit-change})

The advantage of building this around quibble is that it's the same tool already used for CI, so the people doing QA can know that the web environment they are looking at is the same one that had tests which passed.

We'd probably also want to be able to somehow surface the logs from browsing the site, especially to look for PHP notices / warnings / errors. Conceivably the clean up script could grep the log file for that particular instance and then report back to gerrit with any notices/warnings/errors.

Because of how we do configuration management and the lack of content, this isn't going to be a perfect solution by any means, but it would probably help with the code review and QA process.

See also T234643#5558658 and related discussion / notes.

From T76245 see:

Event Timeline

kostajh renamed this task from Lightweight staging for gerrit changes to Lightweight preview environment for gerrit changes.

@Bstorm @bd808 do you have any insights about WCMS vs Toolforge for this? It seems like an application on WCMS that creates docker containers would probably be the right way to go rather than a Toolforge application, yeah?

@Bstorm @bd808 do you have any insights about WCMS vs Toolforge for this? It seems like an application on WCMS that creates docker containers would probably be the right way to go rather than a Toolforge application, yeah?

Yes, this idea is probably better suited to a dedicated Cloud VPS project than Toolforge. Running multiple webservices from a single tool account would require some infrastructure changes, and the NFS $HOME is a poor place to manage multiple large git clones and sqlite databases.

find a free port or use nginx-proxy/traefik

Putting the MediaWiki deploys behind the Cloud VPS HTTPS proxy would be a good idea. The benefits of this are TLS termination and removal of visitor IP address to prevent exposure in logs. We could probably find a safe way for your listener/launcher application to create new proxies so that you could end up with something like mwgerrit-{gerrit-short-change-id}.wmflabs.org as the exposed proxy endpoint for each Docker container. Toolforge tools do something similar when they register webservices running on the job grid.

We'd probably also want to be able to somehow surface the logs from browsing the site

An ELK stack hosted within the same Cloud VPS project could be used for this potentially. That might be more complication than needed though.

cc @hashar and @awight with regards to some of the upcoming Quibble work.

I think that has been addressed by patchdemo T76245. Long term the project would be Catalyst https://wikitech.wikimedia.org/wiki/Catalyst which should have the ability to spin up additional backend services and is most probably not based on Quibble.

I am thus not sure what this task is about, but I feel its need is addressed by Patchdemo and Catalyst and thus we can decline it?

kostajh claimed this task.

yeah I think this was a duplicate of T76245, although, this task had proposed to use Quibble for the heavy lifting instead of having another set of developer environment tools for things like cloning, installing, and serving MW.