@hashar spinning this ticket out from the ping on T138401#9339536.

As discussed in DMs, the approach you suggested of having jsduck and jsdoc executed in the same repository is not possible because the markup languages are incompatible. Is there a way we could update https://doc.wikimedia.org/mediawiki-core/master/js/ to redirect to another domain that can host static and non-disposable content?

In T351764#9350674, @Jdlrobson wrote:

@hashar spinning this ticket out from the ping on T138401#9339536.

As discussed in DMs, the approach you suggested of having jsduck and jsdoc executed in the same repository is not possible because the markup languages are incompatible. Is there a way we could update https://doc.wikimedia.org/mediawiki-core/master/js/ to redirect to another domain that can host static and non-disposable content?

you could make a new miscweb domain https://wikitech.wikimedia.org/wiki/Miscweb but might be able to get away with doc.wm.o just a different directory which should be rather easily doableTM unless I'm missing something super obvious. Also you can create a new directory in mw-config repo under docroot/noc/ but it's not really a good idea.

There is new and old "miscweb". New is on kubernetes. If you actually need a new microsite or a rewrite rule, please tag my team (sre-collab) and we will help you with it, no problem. Please don't add stuff to the old one though.

If you can get way with just copying files and don't need any of that, that's even easier of course and I think Antoine can help.

@Dzahn copying and pasting the existing folder https://doc.wikimedia.org/mediawiki-core/master/js/ to a new domain would be a great start. I can add a static page that redirects to the new documentation once that's done if a redirect rule is a pain. Is this the right tag? I can't find collaboration-services

@Jdlrobson The tag is correct. We got renamed, my bad.

To clarify, are we talking about something like doc.wikimedia.org/mediawiki-core-NEW or doc-NEW.wikimedia.org/mediawiki-core?

(where NEW is of course just a placeholder for something nicer)

I think a new vhost is a bad idea. There are already a large number of separate generated document collections on doc.wikimedia.org. Just move the thing you want to preserve to a new directory. https://doc.wikimedia.org/mediawiki-core-js-legacy/ or whatever.

https://doc.wikimedia.org/mediawiki-core-js-legacy/ sounds great

Could also consider https://doc.wikimedia.org/archive/mediawiki-core/master/js/ to mirror the existing URL structure and maybe create a generic place for this sort of thing in future?

In T351764#9351145, @Jdlrobson wrote:

maybe create a generic place for this sort of thing in future?

YAGNI. "This sort of thing" is working on a 7 year old task to replace an obsolete documentation generator. If it happens twice more in the next 5 years maybe think about making a repeatable pattern then.

Fair. Please host at whatever URL makes sense.

If we end up copying a static snapshot that will no longer be generated afterwards to a folder on a host, let's make sure a backup of it exists as well.

The doc files are rsynced by the "doc-uploader" user to /srv/doc/ on the actual doc* webservers.

Everything like mediawiki-core/FOO is just under that structure.

And the document root of the webserver is /srv/deployment/integration/docroot/org/wikimedia/doc where /srv/deployment hints at scap deployments. scap is installed on doc hosts.

The hosts allowed to rsync to it are gitlab_runners and jenkins_releases_hosts.

So as far as I can tell this change needs to happen in CI and there is nothing to be done on the doc hosts. Just change it at the source and then it's rsynced.

Looking at the way this is set up in CI, it seems to me that we could do the following:

• Change what running npm run doc in MW core does, from building the jsduck output to building the jsdoc output (and update/rename npm run postdoc and Grunt's copy:jsduck jobs accordingly). @Jdlrobson already has a WIP patch for this at https://gerrit.wikimedia.org/r/c/mediawiki/core/+/975921
• Once this patch is merged, it will automatically cause jsdoc docs to be published to https://doc.wikimedia.org/mediawiki-core/master/js (where the jsduck docs currently are). This is because the CI job is tool-agnostic, it just runs npm run doc and expects the output to appear in a certain directory. The jsduck docs would no longer be published there.
• To archive the jsduck docs, we could use this hack:
  • Create a Git tag called jsduck-legacy at the commit right before the jsduck->jsdoc switch commit (the last commit where npm run doc still points to jsduck)
  • Push that tag. This should cause CI to run a job that builds the jsduck docs for that tag, and publish them to https://doc.wikimedia.org/mediawiki-core/jsduck-legacy/js (not exactly the URL @Jdlrobson asked for but I think it's close enough
  • Delete the tag. This cleans up the stray tag in the Git repo, but leaves the published docs in place

Preserve existing JSDuck documentation on docs.wikimedia.org

https://doc.wikimedia.org/mediawiki-core/master/js/ is autogenerated whenever a change is merged to mediawiki/core. That ensures the documentation is in sync with the code it documents. Back in the early days, we considered hosting the source code documentation on the mediawiki.org wiki and eventually I went to introduce Doxygen to keep the code documentation in sync with the code (while the wiki has the user documentation).

Taking a snapshot means the documentation will diverge as the code evolve and eventually the documentation becomes obsolete or at least misleading. Another issue I have is doc.wikimedia.org hosts artifacts generated by CI which should be considered disposable and regeneratable, I am not willing to have it support hosting archives. I have sympathy for the hack to create a jsduck-legacy tag, but that does not feel like a sustainable solution long term.

Since improvements to the JSDoc documentation will be incremental, we'd like to preserve the existing documentation found at https://doc.wikimedia.org/mediawiki-core/master/js/.

Requirements

• https://doc.wikimedia.org/mediawiki-core/master/js/ should redirect to a static copy of the documentation

This propose a narrow technical solution and I'd like to first get a problem statement about the jsduc/jsdoc conversion, do an analysis of the various solutions on the table and eventually. I think it is too early to come up with the proposed solution to keep a static copy, and I am willing to decline that task for now.

Since improvements to the JSDoc documentation will be incremental

That seems to be the problem statement.

From the parent task T138401 it seems the jsduck to JSDoc migration is a lot about converting tags which sounds like we can write a converter. Then maybe JSDoc generates a good enough documentation which is sufficient to entirely replace the jsduck one and get incrementally enhanced. dfea3e8e5afff0dd94106f69a3421012e610ffda went to exclude most modules and a result https://doc.wikimedia.org/mediawiki-core/master/js/ is mostly
empty, maybe the exclude can be removed so we at least got a more or less complete doc.

Ideally the master branch would solely be documented with JSDoc by the time we cut the RE1_42 branch in June 2024? Meanwhile the documentation will be a bit flappy but I think that is acceptable for master.

Side note:

• REL1_41 got cut already and would stick with jsduck
• CI will have to keep supporting jsduck to generate the documentation for the REL1_39 branch which is supported until November 2025 (two years from now).