Page MenuHomePhabricator

Investigate deploying an instance of the docs site per-patch
Closed, ResolvedPublic

Description

We'd like to automatically deploy a version of the Vitepress site for each open Codex patch to facilitate design and product manager review in particular. We might be able to use patchdemo for this, or consider something like Netlify.

Event Timeline

We made this happen for the Wikidata Query Builder. There is a job template that should be ready-to-use: https://github.com/wikimedia/integration-config/commit/3dcbf270b814d70e49b47fbf105ec68839474f36

That job essentially executes npm run branch-deploy on the repo of you choice and you can do with it whatever you like. Query Builder uses this to deploy every patch to Netlify: https://github.com/wikimedia/wikidata-query-builder/blob/1f4f1078aa0cbb98be9962fc8d6676adaa2fddc2/package.json#L26

You will probably need to set and use the secrets:

- credentials-binding:
     - text:
         credential-id: BRANCHDEPLOY_AUTH_TOKEN_{key}
         variable: BRANCHDEPLOY_AUTH_TOKEN
     - text:
         credential-id: BRANCHDEPLOY_SITE_ID_{key}
         variable: BRANCHDEPLOY_SITE_ID

Things that can be improved upon the status quo here:

  • the link to Netlify is currently buried in some Jenkins logs, it would be nice if it were commented or otherwise accessible on the Gerrit change directly
  • there will be a new Netlify subdomain for every patchset. It would be nice if it would always push/deploy to the same Netlify subdomain for each change. (Wikit has this somehow figured out)
  • Not sure how well this works with a monorepo? Might not be an issue at all, but haven't tried it yet.

Change 763641 had a related patch set uploaded (by Catrope; author: Catrope):

[design/codex@main] build: Bump .nvmrc to 16.9.1

https://gerrit.wikimedia.org/r/763641

Change 763642 had a related patch set uploaded (by Catrope; author: Catrope):

[design/codex@main] docs: Make Vitepress base URL configurable as an environment variable

https://gerrit.wikimedia.org/r/763642

Change 763644 had a related patch set uploaded (by Catrope; author: Catrope):

[integration/config@master] Codex: Pass the custom docroot to the npm command as an env variable

https://gerrit.wikimedia.org/r/763644

Change 763645 had a related patch set uploaded (by Catrope; author: Catrope):

[design/codex@main] docs: Make CODEX_DOC_ROOT default to / instead of /codex/main

https://gerrit.wikimedia.org/r/763645

We made this happen for the Wikidata Query Builder. There is a job template that should be ready-to-use: https://github.com/wikimedia/integration-config/commit/3dcbf270b814d70e49b47fbf105ec68839474f36

That job essentially executes npm run branch-deploy on the repo of you choice and you can do with it whatever you like. Query Builder uses this to deploy every patch to Netlify: https://github.com/wikimedia/wikidata-query-builder/blob/1f4f1078aa0cbb98be9962fc8d6676adaa2fddc2/package.json#L26

Thanks, this is very helpful! We should be able to set up something similar for Codex pretty easily.

You will probably need to set and use the secrets:

- credentials-binding:
     - text:
         credential-id: BRANCHDEPLOY_AUTH_TOKEN_{key}
         variable: BRANCHDEPLOY_AUTH_TOKEN
     - text:
         credential-id: BRANCHDEPLOY_SITE_ID_{key}
         variable: BRANCHDEPLOY_SITE_ID

How do I set the values of these secrets? I can't find any documentation on how to do that.

Things that can be improved upon the status quo here:

  • the link to Netlify is currently buried in some Jenkins logs, it would be nice if it were commented or otherwise accessible on the Gerrit change directly
  • there will be a new Netlify subdomain for every patchset. It would be nice if it would always push/deploy to the same Netlify subdomain for each change. (Wikit has this somehow figured out)

As I understand it this should be doable by passing --alias $ZUUL_CHANGE to the netlify deploy command. That would result in the URL of the deployed instance being something like https://123456--wikimedia-codex.netlify.app/

  • Not sure how well this works with a monorepo? Might not be an issue at all, but haven't tried it yet.

Based on my own testing with Netlify, this isn't much of an issue. The root package.json file needs to support the npm run doc command and delegate it to the sub-packages, but we already needed to support that for the doc.wikimedia.org build.

You will probably need to set and use the secrets:

- credentials-binding:
     - text:
         credential-id: BRANCHDEPLOY_AUTH_TOKEN_{key}
         variable: BRANCHDEPLOY_AUTH_TOKEN
     - text:
         credential-id: BRANCHDEPLOY_SITE_ID_{key}
         variable: BRANCHDEPLOY_SITE_ID

How do I set the values of these secrets? I can't find any documentation on how to do that.

https://integration.wikimedia.org/ci/credentials/ – documentation at https://www.jenkins.io/doc/book/using/using-credentials/ but it's pretty self-explanatory.

Change 763840 had a related patch set uploaded (by Catrope; author: Catrope):

[design/codex@main] build: Add 'branch-deploy' npm script, for WMF CI to call

https://gerrit.wikimedia.org/r/763840

Change 763841 had a related patch set uploaded (by Catrope; author: Catrope):

[integration/config@master] Codex: Add branchdeploy job as experimental

https://gerrit.wikimedia.org/r/763841

Change 763641 merged by jenkins-bot:

[design/codex@main] build: Bump .nvmrc to 16.9.1

https://gerrit.wikimedia.org/r/763641

Change 763644 merged by jenkins-bot:

[integration/config@master] jjb: [codex-docker-doc-publish] Pass the custom docroot to the npm command

https://gerrit.wikimedia.org/r/763644

Change 763642 merged by jenkins-bot:

[design/codex@main] docs: Make Vitepress base URL configurable as an environment variable

https://gerrit.wikimedia.org/r/763642

Change 763645 merged by jenkins-bot:

[design/codex@main] docs: Make CODEX_DOC_ROOT default to / instead of /codex/main

https://gerrit.wikimedia.org/r/763645

Mentioned in SAL (#wikimedia-releng) [2022-02-18T22:14:25Z] <James_F> Jenkins: Defined BRANCHDEPLOY_AUTH_TOKEN_codex and BRANCHDEPLOY_SITE_ID_codex secrets for T293705

Change 763840 merged by jenkins-bot:

[design/codex@main] build: Add 'branch-deploy' npm script, for WMF CI to call

https://gerrit.wikimedia.org/r/763840

Change 763841 merged by jenkins-bot:

[integration/config@master] jjb, Zuul: [branchdeploy-codex-node14-npm-docker] Create as experimental

https://gerrit.wikimedia.org/r/763841

Mentioned in SAL (#wikimedia-releng) [2022-02-18T22:54:05Z] <James_F> Zuul: [branchdeploy-codex-node14-npm-docker] Create as experimental for T293705

Change 763866 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] jjb: Allow passing of environment variables to {name}-docker-doc-publish

https://gerrit.wikimedia.org/r/763866

Mentioned in SAL (#wikimedia-releng) [2022-02-19T00:35:24Z] <James_F> Manually re-triggered a build of the docs of Codex (via zuul-test-repo design/codex postmerge) now that we actually set the environment vars for T293705

Change 763866 merged by jenkins-bot:

[integration/config@master] jjb: Allow passing environment variables to {name}-docker-doc-publish

https://gerrit.wikimedia.org/r/763866

Change 763874 had a related patch set uploaded (by Jforrester; author: Jforrester):

[design/codex@main] docs: Set CODEX_DOC_ROOT default to '/' not ''

https://gerrit.wikimedia.org/r/763874

Change 763874 merged by jenkins-bot:

[design/codex@main] docs: Set CODEX_DOC_ROOT default to '/' not ''

https://gerrit.wikimedia.org/r/763874

Change 763877 had a related patch set uploaded (by Catrope; author: Catrope):

[integration/config@master] codex: Fix CODEX_DOC_ROOT, add leading and trailing slash

https://gerrit.wikimedia.org/r/763877

Change 763877 merged by jenkins-bot:

[integration/config@master] jjb: [codex-docker-doc-publish] Add /s to CODEX_DOC_ROOT

https://gerrit.wikimedia.org/r/763877

This is now working! Big thanks to @Jdforrester-WMF for his quick help with this.

In the (near) future we may enable this for every patch by default, but for now you have to trigger it manually. Here's how you do that:

Change 763881 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] Zuul: [design/codex] Publish the Netlify preview on every patch

https://gerrit.wikimedia.org/r/763881

Change 763881 merged by jenkins-bot:

[integration/config@master] Zuul: [design/codex] Publish the Netlify preview on every patch

https://gerrit.wikimedia.org/r/763881

Mentioned in SAL (#wikimedia-releng) [2022-02-19T02:15:18Z] <James_F> Zuul: [design/codex] Publish the Netlify preview on every patch for T293705

Catrope closed this task as Resolved.EditedFeb 19 2022, 2:53 AM
Catrope claimed this task.

Now enabled for every patch without needing to trigger manually. Thanks James for your help deploying this, and Michael for doing all the hard work that I could just reuse.