Page MenuHomePhabricator

Convert epytext docstrings to reST (sphinx)
Closed, ResolvedPublic

Description

Pywikibot has been using the epytext format for its documentation. epytext is outdated and no longer maintained since 2008 (see: http://epydoc.sourceforge.net/whatsnew.html). We should convert the documentation in Pywikibot source to the reST (also known as sphinx) format. We can use sphinx-epytext package for this purpose. Requirements for this package can be seen in requirements-py3.txt file in docs subfolder. A difference of the epytext and reST formats can be found here

Event Timeline

Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald Transcript

@Xqt for my own education as well as improving the task definition for future reference, can you put some links that describe the history of epytext or indicate that it is not maintained (since when) and about the sphinx-epytext conversion?

Xqt added a subscriber: jayvdb.

@Xqt for my own education as well as improving the task definition for future reference, can you put some links that describe the history of epytext or indicate that it is not maintained (since when) and about the sphinx-epytext conversion?

See the answer at https://github.com/terrencepreilly/darglint/issues/159:

Thanks. Updating task accordingly.

Change 673836 had a related patch set uploaded (by Xqt; owner: Xqt):
[pywikibot/core@master] [doc] convert epytext docstrings to sphinx format

https://gerrit.wikimedia.org/r/673836

I am not happy with darglint docstring checker. It introduces a new undocumented yield(s) Sphinx token and does not handle *args and **kwargs parameter correctly (needs the the stars for the parameter name) and finally it is not possible to keyword token (probably ivar, cvar too). Therefore there are thousands issues to fix: https://integration.wikimedia.org/ci/job/pywikibot-core-tox-docker/14706/console

Xqt removed Xqt as the assignee of this task.Apr 3 2021, 2:49 PM
Xqt lowered the priority of this task from Medium to Lowest.

I think @epytext is more readable and the conversion is done by our doc/conf.py, see rPWBC574a0218d7be. No need to change the system then except we find a better lint checker than darglint.

Change 673836 abandoned by Xqt:

[pywikibot/core@master] [doc] convert epytext docstrings to sphinx format

Reason:

see task

https://gerrit.wikimedia.org/r/673836

Hi Xqt. I really like your CR 673836 and would love to move this along. Unfortunately your CI link no longer works - what kind of errors did you experience and can we reproduce them locally?

I am not happy with darglint docstring checker.

Do we presently perform some sort of epytext check? What kind of validation do we need to move forward with your commit?

It introduces a new undocumented yield(s)

Does darglint or the ReST conversion cause the problem, and where are you seeing the yields?

Sphinx token and does not handle *args and **kwargs parameter correctly

What kind of problem are you seeing? I use Sphinx and haven't experienced a problem documenting kwargs. For example: documentation (code)

and finally it is not possible to keyword token

What do you mean by 'keyword token' and what sort of problem are you experiencing?

Sorry I haven't documented all the reasons why I discarded this change. The basic idea behind this is listed inT277396. I've searched for a linter and found darglint but there where to many problems with it and so much work that all of the errors have must be hidden. The conversion was prepared because darglint does not support epytext. reST is the preferred format because we use sphinx for our generated documention. The conversions are made in docs/conf.py.

Do we presently perform some sort of epytext check? What kind of validation do we need to move forward with your commit?

Currently the only check is made by sphinx but there is no code validation. See also T277396

Does darglint or the ReST conversion cause the problem, and where are you seeing the yields?

Thats about DAR301 and DAR302 I think. See https://github.com/terrencepreilly/darglint/issues/165. Sphinx does not support a yield field but darglint supposes it.

What kind of problem are you seeing? I use Sphinx and haven't experienced a problem documenting kwargs

Sphinx handled *args and *+kwargs correctly but darglint doesn't. You cannot use :param kwargs: or keyword or ivar with darglint but with Sphinx

What do you mean by 'keyword token' and what sort of problem are you experiencing?

same as above. Keyword token is for keywords passed through **kwargs which may be documented but is not allowed in darglint. See https://doc.wikimedia.org/pywikibot/master/scripts/scripts.html#module-scripts.harvest_template for example

Currently the only check is made by sphinx but there is no code validation.

If we don't have existing epytext lint checks then why is reST linting a requirement to proceed with your change? Or did we discard the change for reasons aside from darglint?

Currently the only check is made by sphinx but there is no code validation.

If we don't have existing epytext lint checks then why is reST linting a requirement to proceed with your change? Or did we discard the change for reasons aside from darglint?

rstcheck validates the reSt syntax. It does not validate the content.

If we don't have existing epytext lint checks then why is reST linting a requirement to proceed with your change? Or did we discard the change for reasons aside from darglint?

rstcheck validates the reSt syntax. It does not validate the content.

Hi Xqt. Sorry, what I was trying to ask about was why your CR 673836 was discarded. If darglint is problematic then can we simply drop that for now and merge your reSt conversion?

I have quite a bit of experience with Sphinx so I'd be delighted to help get this conversion across the finish line if I can.

If we don't have existing epytext lint checks then why is reST linting a requirement to proceed with your change? Or did we discard the change for reasons aside from darglint?

rstcheck validates the reSt syntax. It does not validate the content.

Hi Xqt. Sorry, what I was trying to ask about was why your CR 673836 was discarded. If darglint is problematic then can we simply drop that for now and merge your reSt conversion?

I have quite a bit of experience with Sphinx so I'd be delighted to help get this conversion across the finish line if I can.

Don't see any advantage. Sphinx tags can be used already bot epytext tags can be used to. @param foo: is more readable to for developers than :param: because it focuses a bit more. On the other hand I prefer bold and *italic* in favour of B{bold} and I{italic}. We have the best of two worlds.

Don't see any advantage.

Isn't the advantage that epytext is unmaintained as of 2008 and reSt is the documentation format most python projects use? For my part I've extensively worked with Sphinx's reSt whereas pywikibot is the first time I've ever seen epytext. It would be nice for pywikibot newcomers not to need to learn an obsolete documentation format unless we have significant reason to do so.

@param foo: is more readable to for developers than :param:

Per chance could this be familiarity rather than readability? @param and :param: are identically readable to me, and if anything I find the former more jarring because I haven't seen it within documentation before.

Don't see any advantage.

Isn't the advantage that epytext is unmaintained as of 2008 and reSt is the documentation format most python projects use? For my part I've extensively worked with Sphinx's reSt whereas pywikibot is the first time I've ever seen epytext. It would be nice for pywikibot newcomers not to need to learn an obsolete documentation format unless we have significant reason to do so.

Ok that might be a good reason to switch completey. I am fine with it. Sphinx is already supported and there is only the Link shortener which can be useful. The conversion is made in` conf.pywikibot_epytext_to_sphinx()` function.

I made a new conversion run.

Change 673836 restored by Xqt:

[pywikibot/core@master] [doc] convert epytext docstrings to sphinx format

https://gerrit.wikimedia.org/r/673836

Change 673836 merged by jenkins-bot:

[pywikibot/core@master] [doc] convert epytext docstrings to sphinx format

https://gerrit.wikimedia.org/r/673836

Xqt claimed this task.