Documenting Guile code

1

Hello,

How do you document Guile lisp code? I’ve also asked in the guile-users mailing list but I’m still at an impasse.

I’m noticing guile-hoot has the usual

;;; Commentary:
;;; ...
;;; Code:

style, but how does it get extracted to be put in texinfo/HTML pages? Similarly, a procedure with a docstring

(define (f x)
    "The procedure @code{f} with variable @var{x}."
    x)

How do these make it in a manual?

2

Hi @spacebarista,

I will try to explain how I view documentation in Lisp/Scheme and how it contrasts the conventions used by many other programming languages. This is complicated by the fact that there isn’t really one agreed upon way to do things. Different communities do their own thing. What I’m going to describe is what I think is typical for modern Guile projects based on my own observations.

There are 3 categories of documentation that have different purposes:

  1. Source code comments. These are for people hacking on the codebase. That’s what the ;;; Commentary: stuff is, along with any other comments elsewhere.
  2. Procedure/macro docstrings. These are for online user help when hacking at the REPL. In Guile, you’d use the ,describe metacommand to print procedure documentation, for example. There’s an under-advertised preference in Guile to use Texinfo markup for these docstrings so that REPL integrations such as Geiser could generate prettier output vs. the old-school Lisp style of CAPITALIZING variable names. (I’m still trying to make the mental switch to write docstrings this way.)
  3. The manual. This is for offline reference and includes a mix of prose and detailed API specification. The manual text exists separately from the source code.

The distinction between 2 and 3 is due to Lisp’s REPL-driven, live hacking workflow. This is rare outside of Lisp, so the notion of attaching documentation to live objects doesn’t apply. Comments are all you have in those other languages.

We can contrast the Lisp style with the javadoc style commonly found in Java, JavaScript, C++, and elsewhere where documentation is embedded as comments in the source code and then extracted by a tool to produce the documentation. This convention isn’t really used in the Lisp world. The closest analog are docstrings, but it’s important to stress that they are not comments but strings that are attached to live objects in the running program. User documentation in comments alone would prevent this useful runtime introspection.

When it comes to offline documentation (the manual) it is possible to generate some of it using automation to extract docstrings, though there is no standard tool for this. However, I think generating a manual in a completely automated manner produces a rather weak result. For one thing, docstrings are meant to be brief, but manuals are meant to be thorough. Sometimes there’s not much to say about a particularly procedure and the text in the docstring and the manual are the same and thus there can be some duplication. However, we often want to expand upon things in the manual by including examples and inserting prose before, after, or within the documentation for any particular procedure. Thus, a little bit of duplication of the simpler things is offset by giving the author a lot of freedom in how the information is presented. I think I’ve used a custom script to bootstrap a manual from docstrings only once, when I had written a lot of docstrings but didn’t yet start the manual. That was a one-off job, though, and afterwards the manual was maintained as its own thing.

The tl;dr is: I rarely extract documentation from source code files for use in manuals. Writing a manual is a mostly manual (no pun intended) exercise and is a different kind of technical writing than writing source comments.

Hope this helps somehow!

1 Like
3

Thanks for the explanation. I will add that you can extract the file commentary with (guile)texinfo reflection.

What you’re saying makes sense. I would rate myself as newbie to REPL-driven programming, and so understandably these points did not come to me naturally. However, I will also say that the Guile situation is not 100% satisfactory; e.g. GNU Emacs has a lot more features for the integration of documentation…

4

That’s true! Emacs’ documentation system is quite a wonderful thing. I don’t know if there’s any actively used Lisp system out there that matches it. Geiser has a documentation browser, and most Guile programmers use the REPL through Geiser, so improving its abilities would be the way to close the gap somewhat. One nice feature that’s coming in the next release of geiser-guile is texinfo docstring support: Allow processing of texinfo in docstrings. (2cccde1e) · Commits · emacs-geiser / guile · GitLab