Gypsum: my clone of Emacs and ELisp written in Scheme
Ramin Honary (he/him)
- Source code: https://codeberg.org/ramin_hal9001/gypsum
- E-mail: ramin.honary@gmail.com
- ActivityPub: @ramin_hal9001@fe.disroot.org
- Website: https://tilde.town/~ramin_hal9001
The following image shows where the talk is in the schedule for Sat 2024-12-07. Solid lines show talks with Q&A via BigBlueButton. Dashed lines show talks with Q&A via IRC or Etherpad.
Format: 25-min talk ; Q&A: BigBlueButton conference room https://media.emacsconf.org/2024/current/bbb-gypsum.html Etherpad: https://pad.emacsconf.org/2024-gypsum
Etherpad: https://pad.emacsconf.org/2024-gypsum
Discuss on IRC: #emacsconf-dev
Status: Q&A open for participation
Saturday, Dec 7 2024, ~8:00 AM - 8:20 AM MST (US/Mountain)
Saturday, Dec 7 2024, ~7:00 AM - 7:20 AM PST (US/Pacific)
Saturday, Dec 7 2024, ~3:00 PM - 3:20 PM UTC
Saturday, Dec 7 2024, ~4:00 PM - 4:20 PM CET (Europe/Paris)
Saturday, Dec 7 2024, ~5:00 PM - 5:20 PM EET (Europe/Athens)
Saturday, Dec 7 2024, ~8:30 PM - 8:50 PM IST (Asia/Kolkata)
Saturday, Dec 7 2024, ~11:00 PM - 11:20 PM +08 (Asia/Singapore)
Sunday, Dec 8 2024, ~12:00 AM - 12:20 AM JST (Asia/Tokyo)
Description
Slides
Introduction
Ramin Honary
Emacs enthusiast since 2017
Software developer (full stack)
I love Haskell, Scheme, functional programming
Started learning Scheme about 2 years ago
My project: an Emacs Clone
- Tentative name: "Gypsum"
- Its not a great name, open to suggestions.
Goal: to Clone Emacs Lisp
Many clones already:
- Edwin, Jed, jEdit, Jove, Lem, MG, Yi, Zile
These only clone the key bindings, not Elisp
Only XEmacs (a fork of GNU Emacs) provided an alternative Emacs Lisp
Most people don't use Emacs for the keybindings
Anecodtal, but yes really.
Use Emacs because of the power of Emacs Lisp
Emacs is as powerful as a system shell
A good language is what makes it powerful
Goal: use R7RS Standard Scheme
I want it to work on a many Scheme implementations
Guile is the reference implementation
(more about this later)
Goal: able to run any init.el
Should be able to use
init.el
without significant changesMany invest significant time in their configs
Suddenly not having your config is disruptive
Such an Emacs clone would be more useful
Why do this?
I personally like Scheme's minimalism.
Use Scheme as more than just an academic language.
Seems to be a lot of interest in a project like this.
Talk of "Guile Emacs" for about 30 years
A long history of Guile Emacs (1/3)
Early 90s: Initial discussion between RMS, Tom Lord, Aubrey Jaffer, begin work on replacing Emacs Lisp with Scheme.
1999--2009: Ken Raeburn's Guile-Based Emacs. (My project is similar.)
"This project that I (Ken Raeburn) have started is for converting GNU Emacs to use Guile as its programming language. Support for Emacs Lisp will continue to exist, of course, but it may be through translation and/or interpretation; the Lisp engine itself may no longer be the core of the program."
A long history of Guile Emacs (2/3)
2010: Andy Wingo and Ludovic Courtes take maintainership of Guile project.
2009--2011: Emacs Lisp interpreter implemented in Guile. Still ships with Guile.
2011: Guile 2.0 is released
2011--2015: Robin Templeton's GSoC project. (Is presenting later today!)
A long history of Guile Emacs (3/3)
2020: Vasilij Schneidermann published an overview called "The State of Emacs Lisp on Guile".
2020 to present: Guile Emacs is dead? Andrea Corallo, GCC Emacs, JIT-compiler for Emacs Lisp based on "libgccjit", brings into question any need for combining Guile with Emacs.
Demo
GUI is barely working
I have almost no experience with Gtk or GObject Introspection
Hard to debug, crashes at C-level produce no stack traces
Using GDB requires rebuilding all of Gtk, GIO, GLib, etc.
Emacs Lisp parser based on Guile Emacs Lisp
Foked the Guile Emacs Lisp implementation for easier development
Have already submitted a patch to the parser upstream
Emacs Lisp interpter is barely working
Implementing my own interpreter in portable Scheme
Monadic pattern matcher
Can parse but not interpret "subr.el
"
"
subr.el
" is the first ELisp file run by EmacsA good way to determine what to work on first
A call for help
Latest Emacs has 1,393 built-in functions
I could never implement that many functions alone
Probably not all are required to create a useful editor
My job is to make contributing easy
Document the build and test process
Document the system architecture
Prioritize which built-in functions are most essential
Find low-hanging fruit, use as means to teach others
The work for which I will take responsibility
Clone enough Elisp to be able to run ERT tests
Then use GNU Emacs's own regression tests to test patches
Make sure there is a usable GUI
(Someday?) be able to contribute a patch from within
Quick architectural overview
The editor is based in Scheme, not Emacs Lisp
Config, scripting, packages all done in Scheme
Use of Emacs Lisp for scripting not encouraged
Should still be able to run your
init.el
Ideally should be able to run ELPA packages
Difference with Robin Templeton's project
Guile-Emacs links Guile runtime into Emacs
Not a Scheme application
An IDE for Schemers
Emacs Lisp is an "environment"
"Environments" are a feature of Scheme
Scheme procedures can be called from Emacs Lisp
Scheme state can be mutated by Emacs Lisp
(See "
./gypsum/elisp-eval.scm
", "new-env
")
"Functional Lenses"
Because R7RS does not standardize MOP (not even in "large")
Inspired by Haskell
Composes getters and setters
Single source file, easy to port
Ported to 3 other Schemes
A lot of work went into keymaps data structure
Keybindings are an important part of Emacs
Had to do this well from very beginning
Keybindings work correctly in demo
A lot of work went into separating GUI from Editor logic
"Parameters" are a feature of Scheme
Platform-specific APIs are always parameterized
Windowing and widgets
Translate key events to bindings
Evaluating Scheme expressions
Text buffering and rendering
(See "
./gypsum/editor-impl.scm
")
Monadic pattern matching
Simpler, more portable
(Not as feature-rich)
Easier than porting SRFI-241 ("Match") to Guile
No relation to SRFI-247 ("Syntatic Monads")
You can still use pattern matching
Monad pattern matching
Example program
(define push-stack (put-with cons))
(define collatz
(many
push-stack
(either
(try (check (λ (n) (<= n 1)))
(success))
(try (check odd?)
(next (λ (n) (+ 1 (* 3 n)))))
(try (check even?)
(next (λ (n) (quotient n 2))))
(fail "not an integer")
)))
Conclusion
I am just getting the ball rolling
Helping others contribute is my top priority
ActivityPub ::
ramin_hal9001@fe.disroot.org
E-mail :: ramin.honary@gmail.com
Homepage :: https://tilde.town/~ramin_hal9001
Codeberg :: https://codeberg.org/ramin_hal9001/gypsum
This presentation :: https://emacsconf.org/2024/talks/gypsum/
Original presentation proposal
I would like to demonstrate an Emacs clone I have been writing in Guile Scheme for the past year, which I am tentatively calling "Gypsum". Unlike other editors which only clone the Emacs keybindings (Edwin, Jed, jEdit, Jove, Lem, MG, Yi, Zile), I hope my Emacs clone will also fully clone the Emacs Lisp programming language well enough that many of the packages in ELPA, Non-GNU ELPA, and perhaps even MELPA, can be used in "Gypsum" without any modification. I would also like to talk a little bit about the how I am implementing it (the software architecture), and invite others to contribute.
I think my project is of interest to many Emacs users because, firstly, I have personally spoken with a relatively large number of people who have expressed interest in making Emacs programmable in Scheme. Secondly, there is a good amount of prior art for Scheme implementations of Emacs. There are even builds of Emacs that link to Guile which provides a "scheme-eval" built-in function that translates between Elisp data types and Scheme data types. The Guile compiler itself ships with an Emacs Lisp compiler as well, although it does not provide enough of Emacs's built-in functions to be of much use.
So by using Guile, we can make use of a lot of the prior art, in fact I am currently using the tokenizer and reader used in Guile's built-in Elisp interpreter to implement "Gypsum's" Elisp interpreter. That said, I have gone out of my way to make my code fully R7RS compliant, so I hope I can port it to other Scheme implementations like MIT Scheme, Gambit, Stklos, and perhaps Chez Scheme with Gwen Weinholt's R7-to-R6RS translator. I consider the Guile version of Gypsum to be the reference implementation of what I hope will become a fully cross-platform programming language and text editor written in portable R7RS Scheme.
The reference implementation of "Gypsum" is a GUI application based on Gtk using a library called "Guile-GI". Guile-GI uses the GObject Introspection framework to automatically generate Scheme language bindings to libraries like Gtk and Glib which are written in the C programming language. There is not yet any terminal-emulator version of "Gypsum."
The next step of the project will be to implement enough of Elisp that we can run tests written in the Emacs Regression Testing (ERT) framework. We can then incorporate the original GNU Emacs regression test suite into Gypsum. Any new API added to Gypsum Elisp will most likely already have regression tests we can use to make sure it is working in a way that is compatible with GNU Emacs Lisp. I would like to make it as easy as possible for people to contribute to this project, and having a list of APIs to be implemented each with a set of regression tests the APIs are expected to pass, is a very good way to do that.
About the speaker:
My name is Ramin Honary, I have been a professional software engineer of 16 years, lately mostly doing full-stack software development. I have always been fascinated with programming languages, and especially functional languages like Lisp and Haskell. I have been using Emacs since 2017. But lately it is with Scheme that I have been spending most of my free time. I am only a Scheme programming enthusiast, I am not involved with Scheme professionally.
You may also like another talk by this speaker: EmacsConf - 2022 - talks - Build a Zettelkasten with the Hyperbole Rolodex
Transcript
Hi, my name is Ramin Honary, and I'm here to talk to you today about my clone of Emacs and Emacs Lisp that I've written in Scheme so far. So I am an Emacs enthusiast since 2017, currently employed as a full stack developer, mostly working with Python and JavaScript, although my true love is functional programming, especially Haskell, and Scheme. I started learning Scheme about two years ago. And for the past year, I've been working on a project that I'm tentatively calling Gypsum. Naming things is hard. It's not a great name. I'm open to suggestions. But yes, this is the project in which I am trying to write an Emacs Lisp interpreter in Scheme. There are many clones already of Emacs. You've probably heard of Edwin, Jed, Jedit, Jove, Lem, MG, Yi, Zile. Edwin itself is also written in Scheme--MIT Scheme. These only clone the key bindings of Emacs and not Emacs Lisp itself. The only alternative to GNU Emacs that I'm aware of is XEmacs, which is a fork of GNU Emacs. Most people don't use Emacs for the key bindings. I mean, this is anecdotally speaking, but the people who I've talked to, I would say don't use Emacs for the key bindings. They use it really more because of the power of Emacs Lisp. Emacs is as powerful as any system shell, perhaps even more powerful than system shells like Bash. The reason why it's so powerful is because there's a good programming language which you can use to control everything on your system. You can control processes. You can load and save files. You can create files. You can configure things. You can capture the output of processes in buffers. You can filter text through buffers. And a good programming language is what you need in order to do all of this. So one big goal of this project is to try to stick as closely as possible to the R7RS standard Scheme definition. That is the latest Scheme standard: R7. And this is just because I want my project to work on many scheme implementations, not just Guile. Although Guile certainly is the reference implementation. So another goal is to be able to run any "init.el". So you can take your existing "init.el" and run it in my program without significant changes. That's one of my goals in the end. I should be able to do that. A lot of people invest significant time in their configs, and it's kind of disruptive if you want to change editors, not be able to use your Emacs Lisp config. And so I think a useful Emacs clone would be able to clone Emacs Lisp well enough that you can run your "init.el". And so overall, why am I doing this? It's just because I like the Scheme programming language. I love its simplicity and its power. It's an extremely well thought-out language. It's one of those languages where you can understand the entire language from top to bottom. You can read the entire specification and understand it yourself. It's like the computers I grew up with when I was a kid. They were all very simple computers in the late 80s, early 90s. And back then, theoretically, an engineer could understand the entire system at the software level all the way down to the circuit level. You can't do that nowadays. And so nowadays, my computer is not really a physical computer anymore. It's the Scheme language standard itself. That is the core of computation, of all of computation for me. And I would like to use it as more than just an academic curiosity. It was originally designed for teaching at MIT, but it's found use in industry. And the R7RS standard is still relatively new. It's over 10 years old at this point, but hasn't, I mean, the Scheme ecosystem itself is already fairly small. There still, I don't think, has been a whole lot of adoption of R7RS quite yet. Kind of a shame. So I'd like a project like this, a very large scale, kind of a killer-app-like project where you're developing a text editor and perhaps even an integrated development environment in Scheme, I think would be very useful just even as a study of, you know, what can this language do? And just overall, there seems to be a lot of interest in Guile-based Emacs and well, maybe a Scheme-based Emacs, but Guile in particular. There has been talk of changing Emacs Lisp or the core of the Emacs Lisp over to Guile for about 30 years or so, talks originally in the early mid 90s. There were discussions between Richard Stallman, Tom Lord, and Aubrey Jaffer. They considered actually replacing Emacs Lisp with Scheme. In 1999, and going for about 10 years, someone named Ken Raeburn actually started a project where he started writing Emacs in Guile. My project is very similar to this. Here's a quote from his webpage, which is still up, even though it hasn't been updated in 15 years. This project that I have started is for converting GNU Emacs to Guile as its programming language. Support for Emacs Lisp will continue to exist, of course, but it may be through translation and/or interpretation. The Lisp engine itself may no longer be the core of the program. And this is my goal as well. In 2010, Andy Wingo and Ludovic Courtes took maintainership of the Guile project. From 2009, so while Andy... 2009 to 2011, the first Emacs Lisp interpreter was already being implemented in Guile. And even to this day, this Emacs Lisp interpreter ships with Guile. And so this was happening while Andy Wingo took control of the project. In 2011, so shortly after Andy Wingo took control of the project, Guile 2.0 was released. And also in 2011, in the summertime, someone named Robin Templeton, I believe it was a Google Summer of Code project, started actually trying to incorporate libguile, that's the guile interpreter, as a linkable or loadable library, linking it to the Emacs executable, and then providing some built-in functions in Emacs that allows you to call the scheme interpreter, the Guile Scheme interpreter, from Emacs. And so it's not like a wrapper around the REPL like Geiser or SLIME. It's actually the whole Scheme interpreter loaded into your Emacs process. And that means your Emacs will have the ability to actually load compiled Scheme programs and actually run them and share memory with Emacs Lisp processes. And, well, Robin Templeton will explain all of this. They're presenting today, and I'm very excited to actually see their presentation. They'll explain everything. So, let's see. Moving on. 2020, someone named Vasilij Schneidermann, I'm not sure how you pronounce that, published an overview called The State of Emacs Lisp on Guile. Let's see if I have that here. Yep, it's this page right here. He goes into detail about who has done what so far, and what can you do in Guile with Emacs Lisp so far, and so on. Like, what is the state of the project overall? And so (speak of the devil) (Andy Wingo on social media). So, 2020 to present. Guile Emacs is dead? So there's GCC Emacs now. Emacs Lisp now has its own JIT compiler. And it seems like over the past few years, Emacs Lisp has kind of moved off into the direction of becoming its own programming language in its own right, and it is decidedly Common Lisp-flavored. It is very similar to Common Lisp, and that seems to be the direction that it's headed now, and I don't know if there's really any interest anymore amongst the Emacs maintainers of continuing with a Guile-based Emacs. But as far as I know, there's still a lot of interest in the community amongst Scheme and Lisp and Emacs users who are interested in maybe continuing to try to get Guile to become the core of Emacs, or if not, you know, what Robin Templeton has been doing, at least trying to get Guile a language, a first class supported language in Emacs. So that's enough talking. Let me just show you what I have so far. The GUI is barely working, because I have very little experience with GTK or GObject Introspection. It's very difficult to debug, so it's very slow to develop. Any crash at C level produces no stack traces. So far, most of the crashes that I've experienced are due to simple mistakes like passing the wrong data type. So, so far, no, not a whole lot of need for GDB or rebuilding all GTK, glib, and so on with the debugging symbols. But yes, still development's been very slow. I'm learning as I go. I've chosen to use Guile GI as the foundation for the GUI. Let me just load it up quick here. "load main-guile.scm". And this will launch the GUI. I also happen to have a REPL that runs in a separate thread and submits any form that you type to be evaluated inside of the running GUI environment. But you can just type stuff. So "hello world." And of course there is... as you can see, it's not quite rendering correctly. This "Messages" thing here, that should be over here, obviously. I haven't been able to figure out how to get those little details down. But yeah, you can do M-:, and you get your eval, and you can just evaluate, like (what's an emacs,) (or what's a Scheme-specific thing?) Like "(import (srfi 1))", and let's see, do "(iota 20)", for example. And so that is the procedure that iterates and produces some 20 elements of a list. Or you can do something like, let's see, string-append "hello" with space "world". And you get the result and so on. And, you know, scheme allows you to return multiple values. So what I have done here is just every value is captured in a list and it prints all of the return values in the list. So if a procedure returns no values, you get an empty list. And that's that. It's still quite buggy. So like, here's a bug that I can reproduce fairly consistently. I can, yeah, if you do... there seems to be a problem with a widget being freed too soon, so it will crash. I'm going to try and solve that, hopefully, before this presentation goes live. Let's see here. The Emacs Lisp parser is based on Guile Emacs Lisp. So the Guile Emacs Lisp interpreter that ships with Guile, that is what I am using. I've actually copied and pasted the source code from the Guile source base into my own project so that I can iterate on it more quickly. And I've already had to make some modifications to the Emacs Lisp interpreter in Guile. So here's the evaluator. I've actually already modified the parser and the lexer a little bit. And it's at least able to parse all of the "subr.el" program, the Emacs Lisp program. It can actually load that, but not evaluate it, or parse it, but not evaluate it... Read, not eval. By the time this goes live, I will have submitted a patch upstream. And that's another goal of this project, incidentally, is that anything that we can contribute to Guile and any built-in functions that we can implement I would like to, for this project, I would like to try and contribute upstream to Guile. The Emacs Lisp interpreter is not working well, unfortunately. So this copy, this is the copy of the code base (from this commit in particular) and well, I can't get it working. I can't actually get the non-copy, the actual built-in version of the Emacs Lisp interpreter to work properly quite yet. So let me quick go to, (what is this here?) Guile Elisp. So suppose you have this "eval-elisp" procedure here and it takes an Elisp environment and then it evaluates an expression in that environment. And evaluates to a value. So this is the standard way of doing it in Guile. If you can see here, you've got this expression, "compile" expression. This is like "eval". And so actually trying to load this. So let's do "load gypsum". (Let's see here. This is, no), I wanted to "import gypsum backend guile Elisp". And if I actually want to do this... So elisp eval, first of all, it says it failed because there's an unbound variable "elisp-eval". Don't know what it's talking about. There's no such variable in any of my programs. I have no idea what's going on here. You can try to run eval elisp on some simple form like (+ 1 2). And it gives you this exception. This works. This is the same issue that I have with all of the, every version of the Emacs Lisp Interpreter in Guile. I can get it to work with this big ",L" mode. So I can actually do (+ 1 2) here. I can do "princ" like here. That all works fine. It gives me, for some reason, a stack trace here. And yeah, so it's a bit, it's not well-documented. The code base is fairly old. As I said, it was developed around 2011, and it's fairly opaque, and I have not been able to figure out how to get Emacs Lisp in Guile working smoothly. So I have started writing my own Emacs Lisp interpreter. And, uh, "gypsum/elisp/eval-tests.scm". It's, uh, not entirely ready. I can show you some of the tests at least. Here is a simple Emacs Lisp program that you can evaluate. You got "progn", "setq" a to 3, "setq" b to 5, "setq" c to the sum of a and b, return c. And this at least works correctly. As you can see here, the result is eight. Um, but the "let*" semantics are not completed yet. Lots of work left to do there. So in the time I have left, I guess I can just, talk a little bit about what my plans are for the future. I would like to begin by evaluating or actually loading the "subr.el" into my Emacs Lisp interpreter. I actually have tests set up for that as well, so I can actually select any form I want from "subr.el". I can just run this through my interpreter and test to see if everything is working once I get that far. And yeah, let me just say that this is my formal appeal to the community for help on this project. Emacs Lisp has 1,393 built-in functions. I could never implement that many functions on my own, so if this project is going to be useful to anybody in any reasonable amount of time, I'm going to need help. And I know that there are people out there who are very interested in a Guile-based Emacs, and so if you're watching this, please feel free to contact me on social media or over e-mail. My job, the way I see it, is if there's enough interest, and I do get a lot of people interested in starting to contribute, my job will be to document the building and testing process and make sure that it is as easy as possible to contribute code to this project. I want to document the system architecture. I'll write blog posts. I'll do videos on PeerTube explaining how everything works. And I will prioritize which built-in functions I think are probably going to be the most necessary, the most essential to get the interpreter running, and then find low-hanging fruit, functions that are easy for people to implement as a good introduction to getting them started on contributing to the project. And then, of course, I will take responsibility myself of making sure that we can get the Elisp interpreter to the point where it can run the Emacs regression tests. These are the test suites that are used to test Emacs Lisp itself in the GNU Emacs code base. And so ERT is itself written in Emacs Lisp. And so I think if we implement enough of the built-in functions to be able to run ERT, then we can actually start using the GNU Emacs regression tests to test our own interpreter, our own Emacs clone. And of course, I'll make sure that there's at least one usable GUI. I'm currently working on Guile GI and GTK. It would be great to have an... ANSI terminal based... something that works in your terminal emulator. And yeah, it would be great if someday soon, hopefully, we get enough done that you can actually contribute a patch to this project from within the Gypsum editor itself. I was going to do an overview, but that would be for more of an hour-long presentation. So I'm out of time. I guess the last thing I should quickly say is there's no meta object protocol in this project. I think that's a little bit too difficult to port to various scheme implementations. So I've created a substitute, which I'm calling "functional lenses", which is inspired by the Haskell project of the same name. Everything in this project is based on functional lenses. Yeah, also a lot a work went into the keymaps data structure. The point being that I think I have a pretty good foundation here upon which we can build, even though there isn't an actual, there isn't a lot done in the actual prototype itself, not yet anyway, but I made sure to get the fundamentals down from the beginning. And so I think we have something like a solid foundation on which to build. So, I'm going to conclude it there. And here's my contact details. Like I said, this is a project, I'm appealing to the community of all people who are interested in Guile and Emacs to help contribute to this project. I see myself as just getting the ball rolling. Again, taking-off from the work that Ken Raeburn left behind, with my own from-the-ground-up implementation. So yeah, contact me: e-mail, you can take a look at my blog where I talk about what I have done. My source code, the code for this project, is up on Codeberg... The presentation... this presentation, the home page for this presentation, you can find more details there. Oh, I'm on ActivityPub as well, so my handle is @ramin_hal9001@fe.disroot.org, and I'm on everyday. So yeah, please feel free to contact me if you're interested, and thank you for your attention.
Captioner: ramin
Questions or comments? Please e-mail ramin.honary@gmail.com