Page MenuHomePhabricator
Create Task
Maniphest T315541

scap documentation is no more generated
Closed, ResolvedPublic
Actions

Description

Scap documentation is published at https://doc.wikimedia.org/mw-tools-scap/ . It is done by the mw-tools-scap-tox-doc-publish Jenkins job which invokes tox -e doc.

Since we have removed tox.ini in Feburary 2021 (5bc9d9dd55565440b381bc87cf17594f3e97ac01), the job has been failing and the documentation is no more generated.

I don't think PipelineLib / Blubber has a mechanism to capture artifacts and publish them. The easiest is probably to reintroduce tox at least for doc generation and while at it I would like to align the doc publishing to no more use Jenkins git polling but use the postmerge zuul pipeline and we can probably relocate the doc to a more friendly name than mw-tools-scap.

Details

SubjectRepoBranchLines +/-
Update URL for online documentationmediawiki/tools/scapmaster+1 -1
doc: properly redirect back compat URLsoperations/puppetproduction+7 -3
doc: redirect /mw-tools-scap/ to /scap/operations/puppetproduction+3 -0
doc: update links from /mw-tools-scap/ to /scap/integration/docrootmaster+3 -3
Sphinx program output must use a relative pythonmediawiki/tools/scapmaster+10 -10
Move scap doc from /mw-tools-scap/ to /scap/integration/configmaster+1 -1
[scap] check doc generation in test and gateintegration/configmaster+6 -0
Restore tox for CI documentation generationmediawiki/tools/scapmaster+13 -0
Publish scap documentation on postmergeintegration/configmaster+10 -48
Customize query in gerrit

Related Objects

Event Timeline

hashar created this task.Aug 18 2022, 10:53 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 18 2022, 10:53 AM
gerritbot added a comment.Aug 18 2022, 11:06 AM

Change 824456 had a related patch set uploaded (by Hashar; author: Hashar):

[mediawiki/tools/scap@master] Restore tox for CI documentation generation

https://gerrit.wikimedia.org/r/824456

gerritbot added a project: Patch-For-Review.Aug 18 2022, 11:06 AM
gerritbot added a comment.Aug 18 2022, 11:15 AM

Change 824463 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] Publish scap documentation on postmerge

https://gerrit.wikimedia.org/r/824463

hashar claimed this task.Aug 18 2022, 11:15 AM
gerritbot added a comment.Aug 18 2022, 1:29 PM

Change 824495 had a related patch set uploaded (by Hashar; author: Hashar):

[operations/puppet@production] doc: redirect /mw-tools-scap/ to /scap/

https://gerrit.wikimedia.org/r/824495

gerritbot added a comment.Aug 18 2022, 4:01 PM

Change 824463 merged by jenkins-bot:

[integration/config@master] Publish scap documentation on postmerge

https://gerrit.wikimedia.org/r/824463

gerritbot added a comment.Aug 18 2022, 4:11 PM

Change 824456 merged by jenkins-bot:

[mediawiki/tools/scap@master] Restore tox for CI documentation generation

https://gerrit.wikimedia.org/r/824456

gerritbot added a comment.Aug 18 2022, 4:30 PM

Change 824531 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] [scap] check doc generation in test and gate

https://gerrit.wikimedia.org/r/824531

gerritbot added a comment.Aug 18 2022, 4:38 PM

Change 824532 had a related patch set uploaded (by Hashar; author: Hashar):

[mediawiki/tools/scap@master] Sphinx program output must use a relative python

https://gerrit.wikimedia.org/r/824532

gerritbot added a comment.Aug 18 2022, 4:41 PM

Change 824531 merged by jenkins-bot:

[integration/config@master] [scap] check doc generation in test and gate

https://gerrit.wikimedia.org/r/824531

gerritbot added a comment.Aug 18 2022, 4:43 PM

Change 824532 merged by jenkins-bot:

[mediawiki/tools/scap@master] Sphinx program output must use a relative python

https://gerrit.wikimedia.org/r/824532

gerritbot added a comment.Aug 18 2022, 4:47 PM

Change 824535 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] Move scap doc from /mw-tools-scap/ to /scap/

https://gerrit.wikimedia.org/r/824535

gerritbot added a comment.Aug 18 2022, 4:53 PM

Change 824535 merged by jenkins-bot:

[integration/config@master] Move scap doc from /mw-tools-scap/ to /scap/

https://gerrit.wikimedia.org/r/824535

gerritbot added a comment.Aug 18 2022, 5:05 PM

Change 824538 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/docroot@master] doc: update links from /mw-tools-scap/ to /scap/

https://gerrit.wikimedia.org/r/824538

gerritbot added a comment.Aug 18 2022, 5:07 PM

Change 824538 merged by jenkins-bot:

[integration/docroot@master] doc: update links from /mw-tools-scap/ to /scap/

https://gerrit.wikimedia.org/r/824538

Stashbot added a comment.Aug 18 2022, 5:08 PM

Mentioned in SAL (#wikimedia-operations) [2022-08-18T17:08:26Z] <hashar@deploy1002> Started deploy [integration/docroot@1aca57b]: doc: update links from /mw-tools-scap/ to /scap/ - T315541

Stashbot added a comment.Aug 18 2022, 5:08 PM

Mentioned in SAL (#wikimedia-operations) [2022-08-18T17:08:35Z] <hashar@deploy1002> Finished deploy [integration/docroot@1aca57b]: doc: update links from /mw-tools-scap/ to /scap/ - T315541 (duration: 00m 09s)

gerritbot added a comment.Aug 18 2022, 5:10 PM

Change 824495 merged by Cathal Mooney:

[operations/puppet@production] doc: redirect /mw-tools-scap/ to /scap/

https://gerrit.wikimedia.org/r/824495

hashar added a comment.Aug 18 2022, 5:30 PM

Cathal from SRE kindly deployed the Apache config, however the redirect does not work. We have:

# Back-compat for T246042
Redirect 302 /mediawiki-extensions-CollaborationKit/ /CollaborationKit/
Redirect 302 /mediawiki-extensions-Wikibase/ /Wikibase/

# Back-compat for scap T315541
Redirect 302 /mw-tools-scap/ /scap/

https://doc.wikimedia.org/mediawiki-extensions-CollaborationKit/ and https://doc.wikimedia.org/mediawiki-extensions-Wikibase/ are 404.
https://doc.wikimedia.org/mw-tools-scap/ does not redirect but serves the old content.

Cathal did restart Apache.

Why the Redirect does not work is a mystery, it might because of our rewrite rules taking precedence over the mod_alias Redirect

# When the requested URI is not available in our webapp:
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !-f
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} !-d
# Then internally redirect to CI published material under WMF_DOC_PATH
RewriteRule ^(.+) ${WMF_DOC_PATH}$1
gerritbot added a comment.Aug 18 2022, 6:00 PM

Change 824542 had a related patch set uploaded (by Hashar; author: Hashar):

[operations/puppet@production] doc: properly redirect back compat URLs

https://gerrit.wikimedia.org/r/824542

Krinkle awarded a token.Aug 18 2022, 10:53 PM
hashar moved this task from INBOX to Doing on the Release-Engineering-Team board.Aug 19 2022, 11:06 AM
hashar edited projects, added Release-Engineering-Team (Doing); removed Release-Engineering-Team.
hashar updated the task description. (Show Details)Aug 22 2022, 2:00 PM
gerritbot added a comment.Aug 29 2022, 6:14 PM

Change 824542 merged by Dzahn:

[operations/puppet@production] doc: properly redirect back compat URLs

https://gerrit.wikimedia.org/r/824542

Dzahn added a subscriber: Dzahn.Aug 29 2022, 6:39 PM

Before the change was deployed, new test for scap3 redirect fails (as expected, test the test):

[deploy1002:~] $ httpbb --hosts doc2001.codfw.wmnet /srv/deployment/httpbb-tests/doc/test_doc.yaml
Sending to doc2001.codfw.wmnet...
http://doc.wikimedia.org/mw-tools-scap/scap3/index.html (/srv/deployment/httpbb-tests/doc/test_doc.yaml:26)
    Status code: expected 302, got 200.
    Location header: expected 'https://doc.wikimedia.org/scap/scap3/index.html', was missing.
===
FAIL: 10 requests sent to doc2001.codfw.wmnet. 1 request with failed assertions.

Change was applied:

-    Redirect 302 /mediawiki-extensions-CollaborationKit/ /CollaborationKit/
-    Redirect 302 /mediawiki-extensions-Wikibase/ /Wikibase/
+    RewriteRule ^/mediawiki-extensions-CollaborationKit/(.*)$ /CollaborationKit/$1 [L,R]
+    RewriteRule ^/mediawiki-extensions-Wikibase/(.*)$ /Wikibase/$1 [L,R]
 
     # Back-compat for scap T315541
-    Redirect 302 /mw-tools-scap/ /scap/
+    RewriteRule ^/mw-tools-scap/(.*)$ /scap/$1 [L,R]

Now all tests including the new one pass:

[deploy1002:~] $ httpbb --hosts doc2001.codfw.wmnet /srv/deployment/httpbb-tests/doc/test_doc.yaml
Sending to doc2001.codfw.wmnet...
PASS: 10 requests sent to doc2001.codfw.wmnet. All assertions passed.
hashar updated the task description. (Show Details)Aug 30 2022, 9:31 AM
Stashbot added a comment.Aug 30 2022, 9:32 AM

Mentioned in SAL (#wikimedia-releng) [2022-08-30T09:32:07Z] <hashar> doc: on doc1002: sudo -u doc-uploader rm -fR /srv/doc/mw-tools-scap/ That got moved to /srv/doc and a redirect has been set. # T315541

hashar added a comment.EditedAug 30 2022, 9:35 AM

@Dzahn thank you for the httpbb test suite!

I have deleted the old/stalled artifacts and updated a few links on wikitech and mediawiki.org.

The canonical link is now https://doc.wikimedia.org/scap/

gerritbot added a comment.Aug 30 2022, 9:35 AM

Change 827967 had a related patch set uploaded (by Hashar; author: Hashar):

[mediawiki/tools/scap@master] Update URL for online documentation

https://gerrit.wikimedia.org/r/827967

hashar closed this task as Resolved.Aug 30 2022, 9:39 AM
gerritbot added a comment.Aug 30 2022, 9:43 AM

Change 827967 merged by jenkins-bot:

[mediawiki/tools/scap@master] Update URL for online documentation

https://gerrit.wikimedia.org/r/827967

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL