Back to the schedule
Previous: Beyond Vim and Emacs: A Scalable UI Paradigm
Next: On why most of the best features in eev look like 5-minute hacks

Building reproducible Emacs

Andrew Tropin

Download compressed .webm video (29.4M)
Download compressed .webm video (18.4M, highly compressed)
View transcript

It's not always easy to take part of someone's configuration and make it work, it's almost never easy to move your configuration to fresh OS installation or hardware. Not sure that this snippet is enough to make package work? Forgot to install ripgrep in your system for rg.el? Got a broken version of package on package-install?

There is a way to make an Emacs configuration reliable, composable and self-contained. It's possible to freeze package versions, create systemd unit for emacs daemon, maintain system dependencies and package subconfigurations in one place with one tool.

The talk explains how to leverage the power of nix package manager and use-package to make pretty good emacs configuration.

There is a stream record on the same topic: https://youtu.be/2_e3kPJQ93s. It lacks few interesting points about composability of such configuration approach, but already have enough interesting information. The talk will be a little more structured and more Emacs-users oriented.

• Actual start and end time (EST): Start: 2020-11-28T11.26.34; Q&A: 2020-11-28T11.40.48; End 2020-11-28T11.43.33

Questions

Do you deal with config files such as emacs-custom.el, some which have sensitive data?

Sensitive data is in other directories that aren't shared, and emacs-custom.el is completely avoided, as it prevents reproducible/system independent behaviour.

How did you learn Nix language basics? Just from the the manual?

He referred to the Nix IRC channel.

What are the main advantages besides switching computers (which most people rarely do)?

Make parts of config available for projects - sharing with other people.

Have you tried Guix in place of Nix? (more parens!

Currently trying it, and also in-process of switching from Nix to Guix.

Notes

• Emacs configuration is entangled with the system configuration (dired uses ls, grep.el uses grep).
  • Reproducible behaviour is therefore not only dependent of Emacs compilation/configuration, but also system configuration.
  • "config.el" files configure Emacs, and accompanying "default.nix" files make sure that the correct packages/fonts/libraries/etc are installed.
• Reproducible development environment: https://github.com/abcdw/rde
• Using Org-roam to demo how to config a Nix layer(?)
  • custom.el conflicts with Nix(?)

Related talks

• rde Emacs introduction - Andrew's 2022 talk

Transcript

00:00:00.000 --> 00:00:24.056 Hello, everyone. I am Andrew Tropin. I am a professional software engineer I was playing with NixOS It's an operating system based on the Nix package manager. I came up with this interesting approach for configuring Emacs. I want to share it with you.

00:00:24.056 --> 00:01:31.389 I will start with the bold statement that Emacs configuration is almost the same as system configuration. It's not related to that Emacs joke about Emacs being an operating system. It's more about Emacs being integrated with so many tools inside the environment. For example, if you don't even use any fancy workflows, you use only plain Emacs without any configuration, dired uses ls, grep.el uses grep, and info files placed somewhere in your system. Also Emacs can interact with gpg, git, make, and other stuff. When you grow your Emacs Lisp init.el file or other files in your .emacs.d directory, you get much more integration with underlying operating system.

00:01:31.389 --> 00:02:08.622 The question is: how to manage such configuration? Because you can't just take a bunch of .el files and move to a different machine and be sure that everything will work. Because you didn't move your executables. You didn't move configuration of other programs. You didn't move your service configurations. And you can't even just create dotfiles for each program and move it with your .el files. The approach would be a little broader.

00:02:08.622 --> 00:02:23.722 Everything that I am showing today is available on Github. Any source code, you can find here. but my copy of the repository is on my local machine.

00:02:23.722 --> 00:03:45.889 As you can see, the font is a little small. And also, my terminal font is also a little small. I can do a quick fix and increase the font. But imagine how cool it will be if you can have a file which contains the configuration for a system. You change some value. Here, for example, fontSize = 16 and run some command and based on this file and some other includes your operating system is built and all your environment is set up and ready for use. For example here, we already built the new operating system, and everything is already installed in my SSD. Now I can run the program and you can see that my alacrity terminal has much bigger font and also if I restart my Emacs instance it by default uses a much bigger font for any buffer. Practical, and as you can see, it's already working, thanks to Nix and NixOS.

00:03:45.889 --> 00:04:44.556 I will explain a little later how it works inside, but for now, let's specify a little more what happened right now. I fed my... Oh. It doesn't work. Sorry. I want... I have my whole operating system defined in a few Nix files. For example, here you saw the file which defines some variables for my environment and then a few more files for different programs. There is a folder which contains all Emacs-related configuration. Also, there are package definitions defined in Nix package repositories which is also included for the function which generates the operating system.

00:04:44.556 --> 00:05:11.689 Getting all my configurations written in Nix language and a few firewalls in ?? languages, everything is gathered together, and from that input and only from that input, the new operating system is built. Emacs now is a part of this operating system. I can distribute this Emacs configuration with all the environment that I want.

00:05:11.689 --> 00:05:50.789 Practical so far. Let's clarify which problems does it solve. First of all, the integration problem. For example, a few minutes ago, you saw that I changed one variable. That was to update... The first one, for my terminal, and the second one, for my Emacs. It's pretty good that a few different programs can share some data. For example, you can have one of them for every application, or something like that and you change only one value in one place and the whole operating system is updated.

00:05:50.789 --> 00:06:20.856 Also, another problem is reproducibility. For example, when you install your new instance of Emacs on your laptop or something like that, you can be sure that you will get the same package versions and you can be sure that the configuration of your work results in newly-updated or newly-installed packages.

00:06:20.856 --> 00:06:43.256 Also, if you update packages, sometimes it's hard to revert, because it's the way your package manager almost every time works. You're just getting the latest available packages. If they are broken, you need to wait for the maintainer to update them.

00:06:43.256 --> 00:07:39.656 And also, your basic configuration almost always doesn't contain any native dependencies, like executables or something else. Recently, I saw some attempts to make it possible to use use-package for those needs, like ensuring native dependencies or something like that. It's obviously... If your configuration isn't reproducible and it doesn't have your whole environment, placed in one repository, it's very hard to share such configuration. You can share part of your configuration and some instruction how to get a similar environment, but it doesn't always work. Let's go closer to actually Emacs configuration itself.

00:07:39.656 --> 00:08:10.839 I had some experience with Spacemacs and Doom Emacs distributions. I also watched a lot of videos and articles by Protesilaos and a lot of other custom configurations of many different cool people. And also I was inspired by use-package and decided that I will create a folding structure for my Emacs configuration.

00:08:10.839 --> 00:09:01.306 I will be using subconfigs. It's almost the same as layers in Spacemacs, or modules in Doom Emacs, which are self-contained. They contain Emacs Lisp code which configures all packages necessary for this part of configuration. It contains all Emacs dependencies like Emacs packages. It contains all native dependencies like binaries or maybe info pages or something like that. It also contains variables that can be shared between Emacs and other applications, and it can contain service or system definitions which configure your systemd service or something like that that you use in your workflow. For example, for synchronizing your e-mails.

00:09:01.306 --> 00:10:06.922 Let's start from just the example that I already am... I have a folding structure for my configuration. I have some files here. early-init just has this. Nothing changes. It will be copied to that .emacs.d directory later with some exceptions that it will replace the Nix dir and a symlink will be created to it. I have use-package-init.el. It's part of configuration that will be on top of everything to be able to use use-package in my subconfigurations. And actually some Nix code to glue everything up and config dirs which contain all my subconfigs.

00:10:06.922 --> 00:11:10.105 Let's start from faces subconfig. Let's start from config.el which can be familiar for many people. Just use-package definition for faces package and some configuration for it which are setting some attributes. It reads some variables. Those variables are actually defined in a different place. If I open default.nix file, you can see that it contains the definition or subconfig, and it should contain a definition of variables that it uses by... I forgot to move it from my original default.nix file somewhere here. You probably can find definition of those variables just right here.

00:11:10.105 --> 00:11:38.422 I took values from my Nix expressions. Those values will be shared across my alacrity, Emacs, and other applications. Later, they will be placed in generated Emacs configuration. They will be available for faces config. Here I will be referencing them just like Emacs variables.

00:11:38.422 --> 00:12:39.222 Let's take a look at another more complicated example. For example, org-roam package. Just a basic use-package configuration which uses a variable and the definition. It's a little more complex than the previous one. Elisp configuration in the same file. emacsPackages specified here. Those two packages: org-roam and company-org-roam. systemPackages: it's something that should be available on your host operating system. And for emacsPackages, you need sqlite package, and also the definition of the variable which will be passed in my Emacs configuration later. It's equal to my workDir, which is defined in my environment, and a subdirectory of it.

00:12:39.222 --> 00:12:43.222 ([Amin:] Andrew, you have about five minutes including questions.)

00:12:43.222 --> 00:13:26.222 Oh, okay. I'm almost finished. It was last example. Let me open my Org file. Okay. Right here. I won't give you an introduction to Nix itself and the underlying mechanism, but I can say that there's already a proof of concept framework for utilizing Nix and NixOS for configuring Emacs and making a very complex workflow reproducible on other machines. It gives everything that we saw right now.

00:13:26.222 --> 00:14:05.389 For the future work, I plan to reimplement it in Guile, which is a Scheme dialect, which is another Lisp language, for the GNU Guix operating system, because I like Lisp languages a little more than Nix languages and I want to make this project from proof of concept to some state which will be user-friendly and available for other people. If I will have a lot of time, I will make an operating system which will be inspired by Lisp machines to make the whole experience very Lispy.

00:14:05.389 --> 00:14:22.622 Thank you for your attention and now I will answer questions. Oh. There is a lot of... Okay. I see some questions.

00:14:22.622 --> 00:14:29.222 Did you release some config files such as Emacs custom.el, some of which have sensitive data?

00:14:29.222 --> 00:14:59.456 Ideally, in the folding way, I create a separate directory called local/share/emacs, and I place custom el files here. It's not synchronized in any way, and it will be just lost in case you move to a separate machine. I do it for a purpose, because I don't use custom.el. It's hard to make it reproducible if you're using such mechanism as custom.el.

00:14:59.456 --> 00:15:06.656 How do you learn the Nix language basics? Just from the manual?

00:15:06.656 --> 00:15:32.989 I read a lot of documentation. Also, I saw the course like Learn Nix in 15 minutes. And also there was another resource. Better to ask this question in Nix or NixOS channel in IRC, which will be treated in more details.

00:15:32.989 --> 00:15:38.909 What are the main advantages besides switching computers, which most people rarely do?

00:15:38.909 --> 00:16:10.556 For example, the original idea was to make part of configurations available for projects. For example, you have some project, you made the setup, and want other developers to use the same setup on their machine, but you implement only the part of stuff, like one subconfig especially for this language for this project. With such approach, you can easily share such subconfig with other people.

00:16:10.556 --> 00:16:15.239 Have you tried Guix in place of Nix?

00:16:15.239 --> 00:16:41.239 Yes, I tried it, and currently I am in the state of switching from Nix to Guix. You can follow my Youtube channel, I think, I do streams twice in a month talking about reproducibility and related stuff. Probably soon I will be talking about installation of Guix and configuration of it.

00:16:41.239 --> 00:16:50.406 In case you're watching this video later, you can find me somewhere on the network using those contacts. It's my nickname and my e-mail address.

00:16:54.072 --> 00:17:04.622 ([Amin:] Awesome. I think we're wrapping up just on time. Thank you so much, Andrew, for your great talk, and for hanging out to answer the questions live.)

00:17:04.622 --> 00:17:18.000 [Andrew:] Thank you for organizing the conference and thank you all participants for questions and participation. See you soon!

Saturday, Nov 28 2020, ~11:24 AM - 11:44 AM EST
Saturday, Nov 28 2020, ~ 8:24 AM - 8:44 AM PST
Saturday, Nov 28 2020, ~ 4:24 PM - 4:44 PM UTC
Saturday, Nov 28 2020, ~ 5:24 PM - 5:44 PM CET
Sunday, Nov 29 2020, ~12:24 AM - 12:44 AM +08

Back to the schedule
Previous: Beyond Vim and Emacs: A Scalable UI Paradigm
Next: On why most of the best features in eev look like 5-minute hacks