Back to the talks Previous by track: New in hyperdrive.el: quick install, peer graph, transclusion! Next by track: Emacs 30 Highlights Track: General

Emacs Writing Studio

Peter Prevos (he) - peter@prevos.net , https://peterprevos.com , @danderzei@aus.social

Format: 14-min talk ; Q&A: BigBlueButton conference room
Etherpad: https://pad.emacsconf.org/2024-writing
Status: TO_FOLLOW_UP

00:00.000 Introduction
00:57.649 Why?
02:02.452 EWS configuration
02:50.072 How did I develop EWS?
03:21.954 Overall workflow
04:29.320 Inspiration
05:54.761 Ideation
07:39.113 denote-explore
08:54.581 Writing with Org
10:05.920 The project file
12:18.720 Conclusions
Duration: 13:31 minutes

Description

Emacs Writing Studio is a comprehensive guide and configuration for authors using Emacs. The book covers everything from organising ideas and writing distraction-free to publishing in multiple formats. The book itself was fully written and published with the configuration it describes. This talk outlines the proces of developing this configuration and the book and some perspectives on using Emacs as a tool to write for humans.

About the speaker:

Peter Prevos is a water engineer and social scientist. In this talk Peter introduces Emacs Writing Studio, a starter kit and associated manual for authors with no Emacs experience.

Discussion

Technical issues connecting to the BigBlueButton room, so no live Q&A

  • Q: What's Emacs composition proposition above focused/opinionated packages but below full config bases (Doom/Spacemacs) -- say for "js/ts devel with the kitchen sink", or "writer studio" like this? Akin to Doom/Spacemacs "modules" and such. Just other packages? If so, why haven't these caught up, and convenient libraries of those emerge? We're typically composing at the lowest levels of packaging, and proliferation of base configs could be a symptom(?)
    • A: (not yet answered)
  • Q: For writers who procrastinate, it's very tempting to fall into the time sink of configuring Emacs rather than simply writing, even with the optimal Emacs writing setup. Is this something you've encountered, and do you have any suggestions for how to combat it?
    • A: As probably all beginning Emacs users I spent a lot of time tinkering. I decided that I should work with rather than on Emacs and thus EWS was born. My prod config is 99% EWS with some enhancements. My config developed by just starting with vanilla Emacs and only add what I needed, as the need arose. 
    • Don't try to develop the ideal system in your mind and then build it, let it grow organically.
  • Q: How much success have you had getting writers to use Emacs -- and stick with it -- using  EWS?  I've had people ask me about using Emacs for technical writing and/or coding, and start off well, but move to another editor in two months.  The main reason is when they want to do something with Emacs, can't figure out how, and find a plugin (for Obsidian or VSCode, say) that does the thing they want with no further setup or tweaking required.
    • A: I have had some good feedback from readers, but I cannot attest to their personal success. The aim of EWS is to get things working without the need for much config. However, Emacs will never bny an easy point and click system such as the ones you mention. 
  • Q:When I was learining Emacs I bounced off it the first couple of times, after reading Mastering Emacs and hearing it was the tinkerers editor I got the right mental model to learn Emacs. Did you have that? and what made it worth using and teaching others?
    • A: The best way to learn anything is to teach other people. So I used EWS as a project to imporve my understanding of Emacs. Not only did it force me to fully understand how things works, but also develop new fucntions and packages where I saw a need.
  • Q: I love using org to write prose. For me, I like writing the final version of the text in org, and I prefer paper to brainstorm ideas and draft things. How was the brainstorming experience and the process to organize your thoughts to write the book using org-mode?
    • A: I do my brainstorning in a paper notebook. The first EWS idea is a sketch in my notebook. In my weekly review I transfer notes worth keeping to Denote (scan graphic notes). Writing by hand has many advantages for idea generation. Denote is my repository, not a thinking tool.

Notes

  • Thanks for the great questions. Apologies for the technical glitch.
  • Very interesting talk
  • YouTube comment: ❤

Transcript

[00:00:00.000] Introduction
Hi, I'm Peter Prevos, and I'll be talking about Emacs Writing Studio. I'm a water engineer, a social scientist, and I dabble in theatrical magic. For each of those endeavors, I do a lot of writing. I take notes, I write reports, articles, books, and websites. I use Emacs for all my research, my writing, and my publishing. Emacs Writing Studio is an opinionated starter kit for authors who write for humans, not for programmers who write for computers. It consists of a configuration, some bespoke functions, but are not yet a package, a website, and a book. The book is completely written with EWS itself. The target audience are authors who are frustrated with using commercial software and hopping from application to application to achieve an objective. These are people without any Emacs experience or coding skills. The objective is to teach these people how to use Emacs, not how to configure Emacs to manage a complete writing project.
[00:00:57.649] Why?
So why did I write a book about Emacs when the documentation is already so extensive? Most Emacs documentation focuses on configuration, burying potential new users with choices. It forces people to work on Emacs instead of with Emacs, and it quickly can become a productivity sink. Emacs is the ultimate free software platform, but with this freedom also comes a price. Barry Schwartz wrote about the paradox of choice, which is about the dramatic explosion in choices in the modern world. Just think about the amount of effort it can take to decide what cereal to buy in a supermarket that you haven't been before. So many different choices. Paradoxically, that's become a problem instead of a solution. Emacs can perhaps suffer from the same issue. Another objective, using a COVID-19 trope: EWS is about flattening the curve--that is, the learning curve--by making these choices for the new Emacs user and avoiding the paradox.
[00:02:02.452] EWS configuration
What about this EWS configuration? I decided to stay as close as humanly bearable to vanilla Emacs. I say that tongue-in-cheek, but also with some realism, centered around Org mode and Denote for note tagging, and citar for accessing bibliographies, and other convenience packages such as vertico for minibuffer completion. There's a dictionary, a thesaurus, and some other packages that are of interest to authors. EWS uses the standard keyboard shortcuts. Writing is much more about thinking than about maximizing the amount of words per minute. Just think about that the most used function for authors is self-insert. There's no need to use fancy keyboard shortcut systems when you write prose.
[00:02:50.072] How did I develop EWS?
How did I go about developing Emacs? I declared Emacs bankruptcy, like we all have every now and then. I decided to write this book with vanilla Emacs and only configure the system as was required. The first thing I did, I hooked visual-line-mode to text-mode and off I went. The configuration grew as the need arose. I was actually surprised with how far you can get with just using Vanilla Emacs. I also developed two packages to enhance how I can use Denote.
[00:03:21.954] Overall workflow
Well, let's get to a demo. EWS is based on a typical research and writing workflow. First step is you need to get some inspiration. Usually we do that by reading, by watching, by listening. Then the next step is ideation. You develop your ideas, summarize ideas from other people, write down your own ideas. For that, you need a note-taking system. EWS also talks about managing bibliographies. The third step is production. This is the actual writing process where, in this case, the EWS book. Once that is finished, we convert this Org mode file into something that can be published. That might be an e-book in an EPUB format, or a PDF for the interior of a a paperback book or perhaps as an e-book as well. There's also configuration in Emacs to produce MS Word documents if you need to collaborate with other people. Let's jump in to Emacs and walk through this workflow, give you a very quick demonstration of what's in the box.
[00:04:29.320] Inspiration
So the first step in the EWS workflow is inspiration. We all stand on the shoulders of each other. Some giants; most of them are normal people. We get inspiration by reading, listening, and watching, and Emacs can help you access text, sound, and video. It depends on external software. Emacs acts as a beautiful interface, and EWS helps you with the configuration. But we also need a tool to manage our electronic library, and BibTeX and citar package by Bruce D'Arcus provides a perfect interface to manage this literature. If I open my bibliography here, you see that I've got 1864 references in my BibTeX files. There's a whole bunch of stuff about, I guess, the weird things that I read. Let's look for Emacs, right, because that's a joint interest that we have. Let's open here this paper by Marcus Birkenkrahe, who did some research using Emacs to teach data science. Data science is my day job, so that had my interest. We have here a Denote file, which I'll talk about in a second. There's a library file, which is a PDF, but it could be a whole collection of different files in different formats, even video or sound. We can create a new Denote file, or there's a link to the document object identifier. Let's go to the PDF. Here we can now do our reading. We can get our inspiration by what Marcus has written here about teaching data science with literary programming tools.
[00:05:54.761] Ideation
Next step is ideation. Taking notes is the core of all creativity. When I read that paper, I might want to copy some ideas, I might get some of my own ideas, and I need to write that down. I basically write everything in a paper notebook, but then I transfer the things that are worth keeping to the Denote note-taking system. Now, there are many systems that exist out there that promise you note-taking heaven when you just follow a certain process. My advice: don't worry about Zettelkasten, PARA, or whatever is out there. Just write your notes and worry about structure and all that later. Even Niklas Luhmann, the inventor of Zettelkasten, called his system a septic tank of ideas. I call my collection of notes a primordial soup from which my books emerge. Now, Denote by Prot Stavrou is a flexible note-taking system that can implement any of the popular methods. I transferred thousands of files to this format, so all my information is at my fingertips. We can jump to the Denote directory. In my case, that's ~/documents/notes. We see here the marvel that is Denote, which is its ingenious file naming convention that has a timestamp, a title, and some (what do we call them) tags or categories, whatever you please. This is a very quick way to find things. Then there is an ews-dired-narrow function. For example, I can find anything, _ews means anything that has the EWS tag, and there we go, it's narrowed down in my EWS notes.
[00:07:39.113] denote-explore
Now, this is all very well, but I also decided to write a package called denote-explore, which helps us sort of explore these collections of notes. One of the fancy things we can do is do some visualization. Let's go to the network function and create a community of notes. A community is a collection of notes that match a regular expression, _ews. All the notes here that are on this list, the ones with the EWS tag. In a second, my browser pops up. I shall move that to my other window in a second. There we go. denote-explore creates a SVG file and it creates an arrow between all the nodes that are linked. It puts a title up there when the node has more than two links going in or out. We can also click on the note to read it in the browser, if you choose. If you set up Firefox properly, I'm sure it can also go into Emacs. This is the note tagging that's available within EWS.
[00:08:54.581] Writing with Org
Now, we've taken all these notes. Now we need to start writing stuff. Org mode, for me, is the ideal tool. Org mode is what you see is what you mean. The text and the syntax instruct the computer on how to produce the final result. This means that one file can become many different formats, an e-book, a printed book, or a website. It doesn't matter. Now, a lot of people talk about what you see, what you get, and that we should have a what you see, what you get mode in Emacs. I think that's irrelevant, because showing the final result while you're writing is actually a distraction. Traditional writing, producing the content, and designing the layout and typesetting are separate processes done by different professionals. In Emacs Writing Studio, the writing is still done by a human. There are no provisions for large language models in EWS. But layout and typesetting is done by external software, be it CSS, LaTeX, or XML, all mediated by Org mode. EWS also has some other tools for case conversion, thesaurus, a dictionary that I haven't got time to show, but what I will show you is how the book functions, and then how we create these publications.
[00:10:05.920] The project file
Okay. Let's go to my project file. So these are the EWS chapters, and each chapter is an Org mode file. Let's go into the main document, which is set up with olivetti-mode just to make it easier to read. What we see here is all the fluff from Org mode. These are all the various bits of metadata that I used to create the final result. Then for each chapter, I have an inclusion. This references another file. I've got some properties. In this case, the forward is unnumbered. There's some notes. In this case, Prot actually wrote this for me. Then we can jump into that text. Then what we see at the bottom here is some inclusions for HTML. This is the EPUB version, but it's different in LaTeX. Again, the same file can serve different purposes. So now let's, as the final part of this demonstration, actually create the book, because the book's freely available. You can download the source files from GitHub, and you can roll your own. Let's open the dispatcher. We'll export the LaTeX and we want to open the file. Now this takes a minute because there's a lot happening within all this code, so I'll shorten this video. I'll be silent now. There we are. This is the Emacs Writing Studio PDF version, which will eventually become the interior for the paperback version. So the EWS book is available in all major e-book shops. I'm also working on a paperback, which I hope to finish when Emacs 30 comes out. But the Org Mode files that I use to produce the book, they're available in the GitHub repository, so you can also roll your own, because the EWS configuration is the one that I use to produce the book.
[00:12:18.720] Conclusions
Now, some conclusions from this journey is that the best way to learn is to teach. That was my personal objective. I learned a lot from systematically working out how to implement the EWS workflow. One of the surprising things that I learned is to have reliance on external software throughout the writing process. I spent a lot of time in the book about explaining these connections that Emacs is not just a text processor, it is also an interface with other software. Now, future developments, as I mentioned, there'll be a paperback version of the book when Emacs 30 comes out, and I might reconfigure things a little bit. I'm only planning to update the configuration when packages break or with another major Emacs release, because I want to work with Emacs, not work on Emacs. A big thank you to Prot Stavrou, who helped me with Denote packages, and he also wrote the forward to the book. There's also several test readers that I met through Mastodon that helped me out quite a bit. Of course, all the Emacs and package developers without which none of this would exist. Thank you all for your attention, and I look forward to your questions and suggestions.

Captioner: sachac

Questions or comments? Please e-mail peter@prevos.net

Back to the talks Previous by track: New in hyperdrive.el: quick install, peer graph, transclusion! Next by track: Emacs 30 Highlights Track: General