Back to the schedule
Previous: Org-roam: Presentation, Demonstration, and What's on the Horizon
Next: Org-roam: Technical Presentation

CategoryOrgRoam

Org-mode and Org-Roam for Scholars and Researchers

Noorah Alhasan

Download video
Download captions

Download compressed .webm video (50.8M)
Download compressed .webm video (22.2M, highly compressed) View transcript

Org-mode improved so much over the years, and the use-cases in org-mode are vast and highly technical. There is something for everyone in org-mode, and it's important to sift through all of these features and figure out what's best for a given situation or specific users. Therefore, I will be targeting academics and scholars that are engaging with literature in the early stages of a project or their academic careers.

Academics and scholars engage with complex ideas and unstructured research workflows. I believe that org-mode can add more structure to the madness, and I will use this talk to clarify a possible solution to reduce such complexity. I propose a research workflow framework that utilizes org-mode, its raw form, and its many associated packages. However, the main package I will be mostly talking about is Org-Roam, and the way its underlying principles will revolutionize the research workflow.

This presentation will help researchers organize and build their knowledge database in a streamlined and effective way. The research workflow is presented in three phases: planning, note-taking, and reference management. I will talk briefly about the packages and special-use cases for each stage and learned lessons along the way. Finally, the presentation concludes with future considerations and possible org-mode features.

  • Actual start and end time (EST): Start: 2020-11-28T15.17.33; Q&A: 2020-11-28T15.32.18; End: 2020-11-28T15.39.00

Questions

I use org-roam-bibtex to take notes on particular academic papers in conjuction with org-noter. This means all notes for a given paper are in one Org file. However while it is possible to link to headings within a file, there is no functionality to easily search through and link to these subheadings. What do you do to overcome this? I've only superficially looked at org-rifle as a possible method.

Whats this presentation software? Looks really cool.

beamer (LaTeX)

https://github.com/nalhasan/emacsconf2020 for the slides/presentation.

How does the view for time blocking works?

have you seen the project Papis? https://github.com/papis/papis I think the author is working on an Emacs package, what would be your thoughts? (it's a Zotero alternative)

"Powerful and highly extensible command-line based document and bibliography manager."

Did you try using ebib instead of Zotero? If so, is Zotero better in some way?

Zotero has a lot of plugins you can play with and so far it's been great.

Some people have been using a connector between Emacs & Zotero….

You can create groups for collaborative projects in Zotero and this is a plus. (thanks for the answers! I'll give it a try!). - https://github.com/papis/papis-zotero maybe useful ^^

Do you have any suggestions on what subjects/things should be tags/separate org-roam files for cross-linking? I've been struggling with whether making almost every term be a link or only using links for broader subjects.

"Should I be combining ideas together into one…?" So far I've been using the Org Roam default way.

Meta question: is there a place where people are collaborating on research "about" Emacs?

Definitely interested, but there is no place (yet!).

Notes

Transcript

Good afternoon or good evening, everyone. Today, my talk is going to be on Org Mode and org-roam for scholars and researchers. Leo has talked about the overall picture of org-roam and org-roam-bibtex. I will be talking more about the research process itself using these tools.

(00:00:20) All right. So, just to introduce that the research process is really messy. You're always working in piecemeal tasks and things move around all the time. There needs to be a system where you can organize all these tasks, all these ideas in a way that is flexible and effective. So my motivation is that research is hard and writing about it is even more difficult. My goal is to add some structure to this whole madness.

(00:00:51) Here's a list of some of the stuff that I've been using since I first learned about Emacs in 2019 and what I've found useful within my research process. I've organized Org Mode for Researchers and Scholars Within the Writing Process into three modules. First, there's the planning aspect of it, then you've got the writing and the reference management, which I will join together by looking at the example of doing your literature review.

(00:01:21) When we're talking about planning, we're talking about either task management or time management with task management. You've got Org Mode's TODOs, tags, and categories. These are really powerful tools that you could use in your Org files to organize your tasks and your appointments. There are different types of TODOs that you can either set globally in your init file or they can be file-/buffer-specific. That means, based on context, based on the type of manuscript you're working on, whether it's a literate programming report or your actual thesis/dissertation.

(00:02:03) Also, these TODOs are either created as a subtree, like think of them as headings and sections if you use LaTeX, or inline tasks, which are like Org inline tasks. I like Org inline tasks because I can add TODOs between two paragraphs. That way, it doesn't show up in the table of contents when I export into PDF, HTML, or anything else. So this is an example of buffer-specific TODOs, and this is an example of a little programming report that I was working on where I was dealing with data and analysis and all of that stuff. I needed context-specific TODOs to use them within this buffer. That's how I would organize it. There's also another example of an Org inline task where you could see it in the middle between the two headings. That way, it wouldn't show up in the table of contents, and it would look neater within the text when you export it. I also added a tag of :noexport: so it won't show up at all when I export it into either PDF, which I use all the time.

(00:03:19) Another useful tool for the research and just general planning is the org-capture. When I first started with Emacs, actually, it was for org-agenda. I went crazy with my capture template. I created a template for everything because I was just so excited. But with time, I was using less and less of them, so I kept taking them out. Now this is my simplified capture templates that I use, either for a general TODO, for a regular appointment, a fleeting note, research tasks (because those are what I focus on, like my bread and butter), and then finally with meetings, which I find sometimes I don't use it as much because I would just have the Org file ready instead of needing to capture, you know, open a capture template.

(00:04:07) Right. Org Agenda. That's how I got into Emacs. I needed to organize my life. I found Emacs and it's been great ever since. It populates all your TODOs and appointments into a singular view. So the default view, I think, is a week view. However, I use org-super-agenda. Love this package. I set up my agenda as a daily view with appointments, deadlines, and a habit tracker. A side note, you guys: I'm still struggling with organizing the perfect agenda, so it's a process. Take it easy, all right? So this is just an overview of my daily agenda. As you can see they're just appointments that I import from Gmail using org-gcal; a simple habit tracker of daily free writing-- as you can see, there are a lot of times where I'm skipping, and the asterisk is the one where I've completed that day, so, you know, it's a process--and then regular deadlines. So what happens is that I have other TODOs that I have not scheduled or not added a deadline but they're just tasks that keep piling up.

(00:05:19) When I first started with Emacs and org-agenda, I had everything in there, and it got overwhelming. Then I decided, no, I'm not gonna even let them show up. So what I would do at the beginning of each week or the night before, I would sit down, look at all my TODOs that I have not assigned yet to a deadline or a schedule or just a simple timestamp, and I would organize them throughout the week. So, here's an example of what I did. On that Wednesday, from my Gmail, I had all these appointments, but one of them is I have a writing group session. So I looked at my tasks and I thought, okay, then I will just assign, for example, my Emacs slides or the framework diagram into that writing session. All I did was just add an active timestamp. That is all I needed to do, and it went straight into my appointment. Now, if I miss that, it won't show up on the next day. So if you put in a deadline, it will show up as an overdue, but if you have no deadline or schedule, it will not show up in your daily org agenda. So, just a star.

(00:06:32) All right. Another way of accessing your TODOs is that if it's Org file-specific, buffer-specific, and so... Like when we talked about like whether to have a big-ass Org file or like tiny files, it all depends. This isn't the... you know, the way this depends, because if you're working on a dissertation, it's a huge manuscript. You need to work on that Org file all the time. Then yes, my TODOs should be in that file specifically, because every time, if I'm visiting this Org file all the time, I should be able to just look at my tasks from within that buffer. And so I use org-sidebar to keep all these specific TODOs within that Org file. I find it helpful. Okay.

(00:07:26) Now that we're going into the writing and reference management... We'll call it a literature review. This is something I've built as a schema. I think that it works for now. It requires one outside software, which is Zotero, what I use. It's an open source reference management software. It's great. But the thing to keep in mind is that I use two plugins that are really needed for when we work with org-roam-bibtex, org-roam, Org Mode, and the ZotFile. Better BibTeX organizes your reference keys in a way, in a fashion that works for you. For me, all my reference keys are last author and year. With ZotFile, I let it rename all the PDF files the same way that I have for my Bib keys, which is last name of author and year. All right. Once you export your entire library as a Bib file, then you can work on it within Org Mode and Emacs using the following packages. So with org-roam-bibtex, it creates an Org file for each Bib entry. You have the option of templating and doing other stuff with it.

(00:08:52) Then finally, there's this orb-pdf-scrapper. I've used it briefly but I think the potential with orb-pdf-scrapper is if you're going to do a bibliometric study or a systematic literature review, there's something there, but I have to look through it. Anyway, so once you create your reference file of reference X and you're writing your notes, you can either go... Like, with going through Org Mode, you're writing your ideas, you're writing your notes, you're assigning tasks, and then there's org-transclusion, which I will mention briefly at the end, and ways to extract. If you're going to go through the org-roam... Things that you're going to use within org-roam... It's a great way to build your database. You start making the connections. You can visualize your notes and how these references are linked to each other through the org-roam server or org-roam graph. All right. This is just notes for later.

(00:09:53) So this is an example of an org-roam file that I have. For example, if I'm working on adaptation policy, I have these hyperlinks that are linked to other concepts and ideas such as either climate security, changing global environment, so on and so forth. The backlinks are other references that talk about this specific concept. This is really helpful. Then, when you visualize it, the picture on the left (which I'm sure looks really small), you can see the connections that it's making with other references. Of course, this is just like a buffer network. When you look at the entire database network... It's growing. Okay.

(00:10:38) So going into org-roam-bibtex... It utilizes a combination of the org-ref package, helm-bibtex, bibtex-completion. It works with org-roam functionalities and other good stuff. This is an example of my org-roam-bibtex file. All right. So I've created the template which I pretty much use, what Leo has produced in his tutorial. I think it's great. It works well for me. What it does is that it works with your bib file. So if you're in your bib file, you have a sub entry that's called keywords, and usually that's within a journal article. The author would specify these keywords. When it gets imported into Zotero, it extracts those keywords and then it gets populated as an Org file with org-roam-bibtex. I always start with the meta information first, and then I would write my notes after that. This is an example, though, for reference of a physical book, so I don't have a pdf file for it. So when I've figured out a new idea for it, if I'm writing notes on it, I would create a property that says pages. That way, it's easier for you when you go back to citing certain ideas or something, that you have the pages prepared there. It's easier that way. Okay.

(00:12:07) org-noter which is something I use a lot, especially with journal articles that have PDFs and stuff like that. They're really helpful if you are going to... If you've just started using Emacs and org-roam, and you have all these PDFs that have all the annotations and highlighting and all that stuff, with org-noter you can just use the org-noter-create-skeleton command and it will populate all your notes that have already been entered within the PDF file if you're using an outside software, and creates them as a neat Org file. I highly recommend.

(00:12:52) Finally, org-transclusion. I think this is still in its beta phase, but I've been enjoying it so far. I'm guessing people know what transclusion means, which is like copy-pasting text from one Org file to another. This is helpful. I think I peeked at a question that was talking about linking to other Org files. I think org-transclusion could really work. It's equivalent to the include function within Org Mode, but I think... So if you have other files that you know which region that you need in another file, you could use the #+INCLUDE, but with org-transclusion... It's great. I mean you just have... you're just linking one part to the other. Sort of... Like, not refiling, but you know hyperlinking. So this is an example of what org-transclusion looks like. The highlighted problem statement is from another Org file. Then what I would do is just link it to there, and there was a transclusion command. I wish I made another screenshot of it. When you invoke org-transclusion-mode, it turns... It prints it out like that. It's in view mode. Then when you want to edit, it will take you back to that buffer and you can edit the text however you want. All right. So, thank you so much.

(00:14:22) I wanted to leave room for questions, but special thanks to all the folks that work on org-roam, org-roam-bibtex, org-roam-server, org-transclusion, and of course alphapapa on org-super-agenda and org-sidebar. That's how I got into Emacs. Thank you.

(00:14:43) (Leo: All right. Well, thank you. So yeah, this time I'll be the one asking the question and not Amin. I'm filling big shoes right now, so you'll have to bear with me folks. So thank you so much, Noorah, for your presentation that is incredibly interesting. Would you mind if I fed you questions from the charts?)

(00:15:00) Noorah: Go ahead. (Leo: Okay. so the first one I've picked on my end was "Did you try using ebib instead of Zotero, and if so, is it better than Zotero in some ways?) Noorah: No, I have not used Ebib. I've only used Mendeley and then they got bought by Elsevier, and so I was, like, okay I'm done, I'm going to Zotero. There are a lot of plugins with Zotero that you can play around with. I can't speak for Ebib, but definitely Zotero has been a good experience so far. (Leo: Yes, same. I also do research on the side; as I told you, English major, and yeah I also do Zotero. Some people have been using uh a connector between Zotero and Emacs which has... they've had great success with them but personally I haven't touched it already, so yeah. Oh, go ahead. Sorry.) Noorah: All right. So far, I don't have any problems with Zotero, but maybe if I run into something, I might check out Ebib in the future. (Leo: yeah, definitely. I think zotero is a very solid project. You know, the fact that it's being used by people outside of Emacs also ensures that there's quite a lot of backing behind the software, which is reassuring when your livelihood depends on your research.) Noorah: Right. And then I think one more thing with Zotero is that you can create groups, so if you're in a collaborative project, you can create a reference, a library just for your group, and I think that could help. I'm going to be in a project next semester that requires that.

(00:16:27) (Leo: Yeah, definitely. I believe the ability to have folders inside Zotero makes it incredibly useful to manage your different projects, concurrent projects. So moving on to other questions, do you have any suggestion on what subjects or things should be tags or separate org-roam files for cross-linking?)

(00:16:46) Right. So far, now, I'm having trouble with "should I be combining certain concepts together as one?" This is where the thought process starts coming to fruit, is that when you start combining ideas together so you won't need a specific tag and another one that are like similar in ideas... I'm not sure if that answers the question, but so far I've been using the org-roam the default way, which is many small files and then just linking them to my... Like, either if I have a report to write, or if I have an essay to write... I think you're muted.

(00:17:35) (Leo: I did two stupid things. The first one was spilling out my water. The second one was speaking without actually turning on my microphone. Let's just hope that nothing is going to fry in the near vicinity of me right now. But yeah, I believe you've answered the question, so don't worry about it. I'm slightly wet right now, which is not a very agreeable feeling, but we'll have to carry on, I suppose. Another question: "is there a place where people are collaborating on research about Emacs?" So do you want to try to take this one?)

(00:18:06) Noorah: I don't know, but I'm definitely interested in the user experience of Emacs, so if anyone wants to work on that, I'm happy.

(00:18:16) (Leo: Well you do have a a pretty good candidate in front of you, if I should say so myself. I'm incredibly interested about the ability to do research in Emacs and about the ability to preach the FLOSS way to academia and to the academe, especially because I believe there's really something great to be done. Sorry, I'm just looking at the puddle of water on the side which is slightly oozing my way, which is not a very good feeling, really. I believe some work could be done, and if people are interested in the chat right now, do get in touch with us. Both Noorah and I are on our Slack channel. Yes, I know, Slack, the corporate hive mind that is Slack. But we've decided with org-roam to use Slack. You can find us very easily. If you want to talk about these topics, by all means, do, and we'll be very interested to answer your questions.)

(00:19:10) Noorah: I have a question here that says, "How does the view for time blocking works?"

(00:19:15) I use org-super-agenda, so what happens is that my active timestamps are only in my Gmail Org file. If you use org-gcal, you have to specify a certain Org file. When it imports them, it imports them as active timestamps. I make sure whenever I create a TODO or even a research task that it doesn't have a timestamp on it, because what I want to do is go back and then move around these tasks according to my either weekly schedule, or monthly, or however long you want to do it. So yeah, only active timestamps or deadline um appear in your time grid. So that could work. (Leo: That's very good. Just to interject for a second about this, you know with org-roam right now, we're mostly focused on optimization, but we're hoping to move on to UX very soon. So all those matters about having TODOs in your files, it is something that we've been thinking about with Jethro Kuan, who is my main co-maintainer for org-roam. We'll be working on this in the coming months, so don't worry too much about it and stay tuned.)

(00:20:26) Noorah: Yeah. So I've got the ebib and what else... What subjects... I think... Okay! What is this question? "Have you seen the project Papis?"

(00:20:37) I'm not sure what... oh it's a Zotero alternative. Okay. I'll look into it. Thank you. (Leo: I don't know about it either, so please look into it and let me know.) Have we covered all the questions?

(00:20:53) (Leo: I believe we have. We have about two-minute leeway for me to move into the next talk, so we're right on time.) All right. Thank you so much. Really appreciate it. Good luck everyone! (Leo: well thank you, and thank you so much for coming, and allowing me not to be the only one talking about org-roam today.) Noorah: Sounds good. All right. (Amin: Thank you both very much.)

Saturday, Nov 28 2020, ~ 3:17 PM - 3:37 PM EST
Saturday, Nov 28 2020, ~12:17 PM - 12:37 PM PST
Saturday, Nov 28 2020, ~ 8:17 PM - 8:37 PM UTC
Saturday, Nov 28 2020, ~ 9:17 PM - 9:37 PM CET
Sunday, Nov 29 2020, ~ 4:17 AM - 4:37 AM +08

Back to the schedule
Previous: Org-roam: Presentation, Demonstration, and What's on the Horizon
Next: Org-roam: Technical Presentation