Back to the schedule
Previous: Using Org-Mode For Recording Continuous Professional Development
Next: Org as an executable format
Creating technical API documentation and presentations using org-babel, restclient, and org-treeslide
Jan Ypma
Q&A: live Q&A or IRC
Duration: 10:22
This talk was also streamed at an alternate time for APAC hours: https://libreau.org/past.html#emacsconf21
If you have questions and the speaker has not indicated public contact information on this page, please feel free to e-mail us at emacsconf-submit@gnu.org and we'll forward your question to the speaker.
Talk
Q&A
Description
The emacs org-babel package is often mentioned in conjunction with literate programming. The ability to mix code segments with prose indeed offers an intuitive way to augment semantic code pieces with textual descriptions.
In recent projects, I've started to turn to org-mode as the primary format to maintain technical documentation, as well as slides for a technical language course. By using org-babel to pull in "live" code for REST requests, language examples, and shell scripts, one can be sure that the documentation and slides are never out of date.
The session will show how leverage org-babel, restclient and org-treeslide to write and present technical documentation with style.
Discussion
Pad:
- Q1: Sorry if you already answered this somewhere (but if not, can someone with a reddit account copy this over? thx). Hi, I would love to move my team over to using something org-based, but that'll never happen because, well... (wait for it) Emacs! By the way, I'm currently using heavily customized Sphinx setup, mostly internal, sometimes shared with data partners; lots of schema-gen from message protocols defined in code, etc. Anyhow: questions. Do you work with non-Emacs users? If so, how did you get them to accept this workflow? And if it's just you DJ'ing, how do they weigh in when they want an update, open a formal ticket?
- https://github.com/jypma/emacsconf2021/blob/master/presentation.org
BBB:
- Have you encountered any push-back from people requesting the documentation who are of the opinion that only a Word document will do?
- Jan: Not really, I tend to deliver the PDFs only. If more is needed, I expect I'd make a docker container with emacs in it, so people can export the org-file to PDF themselves.
- The main problem I have is that effectively they end up branching it in order to use Word's change tracking, and there is no reverse path.
- Jan: That sounds more like a team thing. Most people I interact with are developers, and are OK reading/interpreting change tracking as a git diff, e.g. on github or azure devops.
- OK, thanks. I think that is the root of my problem I just thought I would ask in-case you had encountered the same. Thank you for the talk.
- Jan: I think in your case I'd either introduce them to a nice web-based git interface (org-mode looks fine usually), so they see it can do similar things as word change tracking.
- Jan: Thanks for your feedback
feel free to reach out if you want to chat more. - Unfortunately in this case they refuse to use anything that isn't a WYSIWYG interface. So even a Markdown editor in a split screen with a preview window is not accepted.
- Thanks!
- Jan: Hold their hands, be kind, small baby steps
IRC: (nick: jan-ypma)
- I use restclient everyday, but never thought about using it from code blocks, duh! Very interesting talk!
- This is a good demo, I've found org-babel to be a really amazing glue language for stuff that's sort of annoying to automate otherwise.
- Thanks! So the fonts of the current talk are: Fixed pitch (serif): New Heterodox Mono, Variable pitch (serif): ETBembo
- for live coding presentations, demo-it is also pretty cool
- indeed, i have been trying to work out literate devops through org documents. Very cool and useful in specific contexts atleast I guess. like finding the status of a service quickly right within a structured org document.
Outline
- Introduction
- Demo: Developer guide
- Demo: REST API guide
- Demo: Presentations
- Used packages and configuration
Transcript
Hi! My name is Jan, and I'll be talking about using Emacs for technical writing. Let's first define what we mean by technical writing. At least, I mean with that, any kind of writing that involves computer systems. So, maybe a developer guide for a system, or a library you've been creating, maybe reference documentation or a user guide for a REST API that you offer as a cloud service, or doing a technical presentation exactly like this one that may actually include some live coding as well that you may want to do while you're showing the presentation without too much context switching. I've been doing a variety of these things in my professional life for a while now, and I found Emacs to be a really nice tool to help out with that, since it actually pulls in different languages. The ones I work with is Scala, Java, C++, and things like that, and everything works in the same way within Emacs, so you don't have to learn different tools to do the same thing. Doing all of this against Java looks the same as it would but with C++ except the language is different.
[00:01:07.119] A little refresher for people that might be viewing this out of context. Emacs is a very customizable text editor environment, and Org mode is a part of Emacs that allows you to deal with structured text. So, a plain text file containing headings, lists, tables, and even code blocks formatted in a particular way, so Org mode can help out with that. And Org babel is the particular part of Org mode that deals with executing those code blocks and actually interacting with, say, a Java or a Python environment underneath, and showing the results of that right inside the same Org file.
[00:01:43.840] Let's look at what are a couple of scenarios using this might actually look like. Let's start with imagining that we are writing a developer guide for a service or a library, or a computer program that we might be writing. And, imagine that we have some dependencies that the program requires that are configured using docker-compose, for those who don't know Docker, docker-compose, it's a way to quickly describe some Linux programs that can be immediately run without installing too much dependencies. You define these using a YAML file called the docker-compose file. Now, here inside Emacs we have a block that defines a YAML file, and we're actually saying this is called docker-compose.yaml with some content here, and you can see that even though we are in Org mode, Org mode knows that it can highlight this according to YAML and Org mode doesn't directly know about YAML, we just said hey this block has to do with YAML. Because there's a yaml-mode in Emacs, it will borrow from that mode to actually highlight this block.
[00:02:55.920] Now, the fun thing is that there's a feature in Org called tangling that allows you to take these kinds of blocks and actually export them to separate files. So, if we look at the directory that we're in right now, we see that we just got the presentation, there is no docker-compose file yet. If I say Control c Control v t (C-c C-v t) and I go back and refresh this directory, now we have a docker-compose file as well, which has the content in it that we just created here. That's very nice because conceptually we don't actually need to leave Org mode, we can say something about this file and have the contents of the file in the same descriptive document while also having some actual side effect of the file existing on disk and us being able to interact with it. For example, we could…, now that the file is there, invoke docker-compose and actually create the nginx web server that we're defining here. Let's do that. We have a little block here that runs the shell script if I invoke that from Org mode, we get the results here, we see that now we have a web server running on port 8080. That's, by the way, serving up the contents of the directory that we're in here on port 8080. So, that's already quite nice.
[00:04:06.000] Let's look at another scenario where we may be documenting a REST API. REST APIs use a lot of HTTP interactions typically describing an XML or JSON structure and which HTTP verb GET or PUT to use with that and the URL. There's actually a nice extension to Org babel called rest client that you can install, that allows you to describe these kind of requests right inside Emacs. First, let's make sure that our HTTP server has something to respond with, that's a little interesting, for example, an XML file. We already know how to do that. So, let's create a code block type xml that we can tangle to file called test.xml, Control c Control v t (C-c C-v t). Now, if we look at the directory again, we have a test.xml file. And, now we can have a new type of block called the restclient, which will invoke REST client, and anything you type into here will be sent as an HTTP request to the server that you specify. Right now it goes to localhost on 8080 and let's see if we can get our test.xml file back. I've just invoked this, and you can see we got the spec and the content type of the server, if we scroll down a little bit I think we see the headers here, yeah, so the server said it's text/xml and restclient is smart enough to actually invoke Emacs's sgml-mode to highlight it. I'm not exactly sure what's the difference is between sxml and xml-mode and there's a nxml-mode, they all pretty much know how to deal with XML. In this case sgml was chosen, which is fine. But you can see we just served up that test.xml file, and we can have some actual text here describing "Hey, if you do this request you might get a response like that," and the server will actually serve that up and insert it right into the Org mode document.
[00:05:56.233] By the way, we're looking at this now inside Emacs rendered somewhat interestingly, but obviously you can export this to a PDF, or HTML, or in all sorts of nice and different ways as well depending on what your particular needs are. Of course, we can't just send GET requests, we can send PUT requests as well, and just like in plain HTTP you have the PUT method on the first line then your headers, and a blank line, and then the body.
[00:06:22.000] If we try and invoke this then nginx will say "405 Not Allowed" because, obviously, just running a plain web server will not allow you to actually upload any files, but this of course could have been any other response as well. Now, let's look at doing presentations themselves, like the one you're looking at. There's a package that I like to use a lot, which is called org-tree-slide. That's the one that's active right now, which takes an Org document and allows you to show one heading at a time. It doesn't matter whether it's the first level, second level, third level heading, they sort of fold into nice things at the top, where you can sort of go through a document one piece at a time. I actually do like to use Org babel at the same time to do some live coding in it as well. Actually there are two ways to go to a PDF, you can just use the normal Org export option to go to a PDF, which is Control c Control e, and then l p (C-c C-e l p), but if you use restclient, the LaTeX file underneath sometimes gets a little wonky because those things don't directly work together. I wrote a little bit of Lisp to help out with that, which you can look at if you check out my presentation later.
[00:07:40.960] There's another package for Org babel called beamer, or ox-beamer it's called, which uses a LaTeX style called beamer to create a PDF, and that one looks sort of…, that one tries to actually create one page per slide which you would actually have a PDF with the slides, but that one is a lot more picky on what your Org file is structured like, so you need to have all your leaf headings at the same level, which I typically don't do. So, I can show you what this one looks like. For this presentation you get a nice title slide, and then you get…, it tries to make an outline, which is the one level above. The slides sort of look okay, but as you go further they sort of start to run into, you know, things not flowing as they should. I'm sure with a lot more LaTeX knowledge you could make this look a lot nicer, but personally I tend to just create a normal PDF document that's just, you know, text with all the actual content of the document. Inside the text you can see the highlighting of especially restclient stuff that works just fine, and it's enough for my needs, so I just tend to make plain PDFs. Since we only have 10 minutes, I will not go into the detailed configuration, you can check out the presentation online to see how all these packages are configured and how I use them, but for now that's all I have. I do recommend you try this out yourself. If you have any kind of documentation or textual things to do, just pick one of these packages at a time, integrate them into your configuration if you haven't already. That's really the best way to go about this, and you know, Google is your friend, if you think "Hey how I would do this with these packages," definitely do that.
[00:09:37.839] More things I will be looking at is using this concept to write unit or integration tests, you can imagine if you have a documentation in Org mode that describes your service as a function of its REST API, you may want to actually run all those commands as part of your build and check if all the documentation is still in order. I'm not doing that yet, but I'm definitely looking into that. I'm also writing some extensions to use Java and Scala in a somewhat higher level with Org mode. But that's not entirely working yet, and we don't have time to go into that today. That's it. Thanks a lot for your attention, and I'll be there for questions later.
Back to the schedule
Previous: Using Org-Mode For Recording Continuous Professional Development
Next: Org as an executable format
Speaker profile
Jan Ypma is an independent software architect and developer, specializing on the Java platform, functional programming and distributed systems.
Name pronunciation: Jan EEP-mah Pronouns: he/his Preferred contact info: jan@ypmania.net