Back to the talks Previous by track: How I play TTRPGs in Emacs Next by track: (Un)entangling projects and repos Track: General - Watch

Org-Mode Workflow: Informal Reference Tracking

Christopher Howard (he/him) - IRC: lispmacs, christopher@librehacker.com

Format: 16-min talk ; Q&A: IRC
Status: All done

00:00.000 Introduction
02:06.040 Tip about completion frameworks
03:14.920 References file overview
05:39.320 The Emacs Lisp code
08:02.720 Example reference to Elfeed article
11:41.540 Searching the references
Duration: 15:04 minutes

Description

This talk describes my Emacs workflow for keep track of references to papers, book, and Web articles, and other things I might want to look-up or quote later. I go over a few code snippets and some basic org commands which make the process quick, easy, and practical for my needs.

This talk is targetted for the beginner to intermediate Emacs user, or those who do not have much experience with org-mode. Also, those who do a lot of advanced technical writing should look instead to Vidianos Giannitsis' talk Writing and organizing literature notes for scientific writing where they will find better tools for reference tracking and scraping.

About the speaker:

Christopher Howard is an electronics technician who studies various scholarly and scientific subjects in his (very limited) spare time. He needed a quick and easy system for keeping track of various references to books, Web articles, and so forth. He found that a few code snippets and some basic tools from Org-Mode have worked well for him, while requiring a minimal setup and learning curve. He hopes that sharing his workflow will be useful to some others with similar needs.

Discussion

  • Q: Is the emacs config for the system in the last talk published?
  • example code: https://bpa.st/UXBQ
  • I didn't mention it in the video, but I like also to use org-mode's attach feature to sometimes attach documents to the references.
  • Try this for inserting link: https://xenodium.com/emacs-dwim-do-what-i-mean/
  • I use (ivy-bibtex &optional ARG LOCAL-BIB) but never hacking tag system, this something I can improve.
  • The current talk shows that most of the time, the already included tools in Emacs allow for custom workflows without needing to use external packages (org-roam, etc.). Of course, the latter are great for richer workflows, but core Emacs is often enough.
  • do you know about the org-insert-link command? saves you from having to type brackets :)
  • You'll likely find helm-org-rifle useful for searching, it handles, e.g. tag inheritance and so forth
  • I used to use a similar approach: https://dindi.garjola.net/zettelkustom.html but I finally moved to protesilaos denote
  • if you search properties infrequently, you can take advantage of quite specific literal pattern ":name" (starting with ":") using swiper or its help equivalent. You should have limited no of candidates if you know prop name. This way you do not need to remember dedicated command for searching org properties :).
  • See also org-ql-find for property searches, e.g. "property:NAME=FOO" syntax
    • that's honestly so verbose that I never managed to use it
      • Well you could alias "property" to "p" and then type "p:foo" or "p:foo=bar". or you could just search for "foo bar" as plain text regardless of their being properties
      • If you frequently search for certain properties you could use org-ql-defpred to define a predicate that matches them
  • Great talk.
  • Great talk, clap clap clap

Transcript

[00:00:00.000] Introduction
Hello, this is Christopher Howard, and welcome to my talk, "Informal Reference Tracking." This is a workflow talk, so I need to explain a little bit about what my needs were. I am not a professional scholar or academic, but there are a number of subjects that I'm interested in, and I occasionally like to write gemlog posts about them. So I needed some way to keep track of references. References to webpage articles, references to books, pages in books, and notes about them. Something that was searchable, but also something that was quick and easy to use, and something that I could set up quickly. And the approach I took, it only took me about an hour or two to figure out how to put it together. I do want to emphasize that there are better ways to do this. I'm not recommending you use my code or follow my exact approach. In particular, what I'm doing was meant to be done with Org's built-in capture and templates functionality, so that's something that's more flexible, programmable, and there's also a lot of add-ins that can be tied into that. For example, tools that allow you to search for, you know, feed in a URL, and it automatically pulls all the reference data for you. And there's tools out there that are really meant for scientific writing, so if you do this professionally, you may need to keep track of dozens of details for each reference and then have some fancy system to generate that into your, or output that into your paper. So there are better systems, but this is what worked for me and what was easy.
[00:02:06.040] Tip about completion frameworks
I do want to emphasize that if you haven't, you really want to learn how to use helm-mode H-E-L-M, or one of the similar systems in Emacs that does fuzzy search on Emacs commands. For example, in Helm here, I input one keychord, and then I just have to remember a few characters of some command, and they don't even have to be right next to each other, like H-O-C will bring up helm-occur. That's based on its algorithms of what I most likely meant and the ones that I've used in the past. So it usually brings up the command that I want, or the one that I want is one or two spots away in the entry. That just saves me a lot of time [and] a lot of memorization. So if you haven't learned Helm or a similar system for Emacs, you really want to.
[00:03:14.920] References file overview
So what is my approach? Well, basically, what it comes down to is really fundamentally nothing more than just a list of Org entries in a file. And there's one entry per reference. Fundamentally, that's all it is. But I'll go over the parts. You can see there's the title for the entry, and that's not necessarily the title of the book or the article, but that's my perspective on it, that's what I want to remember about it, and what I'll be looking for later when I do a search on my references. There's also in here the use of Org's tags here to the right of the title, very handy for searching for entries later. I use some Org properties attached to each entry. I automatically add in here an ID that can be useful if you want to link entries together later. I automatically add in here the date that the entry was created, which can be useful to me if things got sorted in a different order at some point, I could still look through the most recent entries that I had made if I wanted to do that for some reason. And sometimes I add in this publication year field with the idea that one day I might want to do a search for entries based on the publication year of the book or the article, say, only to use recent references or something like that. And then down here below the properties is where I paste in the URL to the webpage, or type in the title and author of the book on the pages, maybe the pages that were relevant, the pages of the periodical, or something like that. And I could put anything that I want down here, some other notes about what's important about this article to me. So fundamentally, that's all it is. Of course, I've added in a bit of convenience code to make this go a lot faster rather than typing all this out.
[00:05:39.320] The Emacs Lisp code
For that, I'll switch back to my init.el file. There's really just five functions. The first two here are ones that I've adapted off the Internet. Honestly, I can't remember exactly where that I got them from, but basically, they're just some functions for making a block of text writable or readable. Writable or not writable, I should say. The idea there is that when I'm creating a new entry, I don't want to accidentally delete or write over some earlier entries that I've made. So I use a little bit of Emacs functionality for that. And then here are the three reference functions that I've actually written. Really trivial, basic stuff here. The core of it is the new-reference function. Basically, what that does is it opens up the references file, jumps to the end of the reference file, starts a new entry, inserts the asterisk. It jumps back to the previous text, and whatever previous text there is, it makes that read-only. Again, so that I don't accidentally delete that, or cut, or type over it, or something when I'm making a new reference. Then it goes back to the new reference, automatically adds in a unique ID for that, and then automatically stamps it with the date the entry was created — today's date. Now, I've got two other functions here. One is view-references, which does nothing but open up the reference file and switch to that buffer if you're not already on it. And then there's one other here, edit-references, which does the exact same thing except for it also goes over all the text in the buffer and makes it writable. So if I really do want to edit those other references, I've got a function to quickly make that possible.
[00:08:02.720] Example reference to Elfeed article
Let me give an example of this. I type in here, new reference. Now I've jumped to the end of my references file. See, it's ready to take the title. Well, I guess I need to have something, some content, to put in here. Let's say I was looking through Elfeed, and let's say I found this interesting article about Mars earthquakes. Let's say I open it up [and] I read through the article. First, I'd figure out what it is that I find interesting about this, what it is that I'm going to want to remember and look up later. So I come up with a quick title based on that. Let's go back to the references with view-reference. And, let's just call it "Study of Mars Earthquake." Now I'm going to also want to put in some tags. On my system, that's done with Control C, Control Q (C-c C-q). And I can put in some tags. I like to go ahead and insert the colons. You can leave those out, but they're going to get added anyway, so I'm in the habit of using them. Let's say we'll call this 'Astronomy' as one tag, and the next tag could be 'Planets'. If I wanted to use a tag that was more than one word in the tag, I'd need to use underscores or something like that. If I wanted a tag that was 'Mars Earthquakes', I could do it like that, but that's kind of silly. Now I try not to be too clever with the tags. I don't spend a lot of time thinking about them. I just come up with some general buckets to throw things in. You can see the tags were added there, to the right of the title. Now you can see down here under PROPERTIES, the ID has already been added, the Date_Created has been added. Sometimes, I'll like to put in the publication year, and for that, I use the org-set-property command. Publication_Year, this year in this case. And then I just need to paste in the URL. I do that manually. I use Org's bracket format for that. So I start that, go back to the article, copy the URL, paste that in. If I want, I can add it in the title with the second pair of brackets here. Don't have to, but often like to. Close that off, and there it is. That was really it. I add a return on the end here, just so the next entry comes out with the right spacing. But really, that's it, and typically, when I'm not explaining it, that only takes 20 seconds or so, or 30 seconds. Pretty quick. Pretty easy.
[00:11:41.540] Searching the references
What about searching later? Well, often the easiest thing is just do a simple, boring incremental search. I usually know roughly what it is that I'm looking for already. If I was looking for that wildflower article, I could just do an incremental search for wildflowers and jump through that. It's pretty simple. Not very impressive, but honestly, most of the time that gets me there pretty quick. Sometimes I find it useful to do an Occur search, more specifically a Helm Occur search. If I use the helm-occur command, then I like to use this to search by tag. That's where it really becomes handy. Let's say I want to narrow it down to all my astronomy references and then narrow it down a little bit more to planets. I can put spaces in between and it still works. You can see here in one window, it gives me the bottom window there. It's giving…, just because of the way the tags are formatted with the title, it gives me a list of all the titles that have those tags. And I usually find what I want pretty quick by just tapping through here. Once I find the one that I think I want, I press enter, and now I'm focused on just that entry. There is some advanced functionality, I believe, that I used in the past where you could search based on the property fields. So do something like search for publication — the most recent publications in the last 10 years. There's some kind of advanced syntax for that, which I used once or twice. Honestly, I use that so infrequently that I have to go back to the Emacs manual and figure it out each time, and figure out again how I did that the last time. But since I do it only once every three or four months, it's not a problem. So I'm not going to go over that today. That's pretty much it in a nutshell. Again, the code that I wrote, this specific approach is not really what I'm recommending. But here it is if you really do want to use it. Maybe I can make a link to the URL and share that in the chat room or something. But I consider this to be trivial code. So just use that if you want to use it. I should be signing off here now. I should be in the chat room, in the IRC chat room, or you can reach out to me by email if you'd like. Thank you very much.

Captioner: bhavin192

Questions or comments? Please e-mail christopher@librehacker.com

Back to the talks Previous by track: How I play TTRPGs in Emacs Next by track: (Un)entangling projects and repos Track: General - Watch