Back to the talks Previous by track: Transducers: finally, ergonomic data processing for Emacs! Next by track: An experimental Emacs core in Rust Track: Development

Gypsum: my clone of Emacs and ELisp written in Scheme

Ramin Honary (he/him)

Format: 25-min talk ; Q&A: BigBlueButton conference room
Status: TO_CAPTION_QA

Talk

Duration: 24:36 minutes

Q&A

03:09.370 Q: I'm curious to know how the hell guile-emacs deals with all of the dynamically scoped modules out there. Is there any effort to automatically modularize and namespace stuff?
05:23.646 Q: Would it be possible to support a GUI toolkit other than GTK?
06:45.257 Q: Do you plan to provide improvements to Elisp as a language, or is the focus on a compatibility layer to facilitate doing all new extensions, etc. in Scheme?
08:29.673 Q: Can we consider a translator like utility to convert elisp to scheme, once guile-emacs becomes a reality?
10:54.390 Q: Why is being able to interpret all of `init.el` an useful goal?
12:08.539 Q: What is the plan to handle elisp packages that depend on 3rd party/external libraries? (libgit/magit or rg/ripgrep)?
15:21.112 Q: Not really a question, but how about Schemacs as a name?
16:45.931 Q: Why is it not feasible for the Emacs layer that interprets Emacs Lisp (the core in C) ot have a Scheme interpreter, instead of using Guile?
Listen to just the audio:
Duration: 23:38 minutes

Description

Slides

Introduction

  1. Ramin Honary

    • Emacs enthusiast since 2017

    • Software developer (full stack)

    • I love Haskell, Scheme, functional programming

    • Started learning Scheme about 2 years ago

  2. My project: an Emacs Clone

    • Tentative name: "Gypsum"
    • Its not a great name, open to suggestions.

Goal: to Clone Emacs Lisp

  • Many clones already:

    • Edwin, Jed, jEdit, Jove, Lem, MG, Yi, Zile

  • These only clone the key bindings, not Elisp

  • Only XEmacs (a fork of GNU Emacs) provided an alternative Emacs Lisp

Most people don't use Emacs for the keybindings

  • Anecodtal, but yes really.

  • Use Emacs because of the power of Emacs Lisp

  • Emacs is as powerful as a system shell

  • A good language is what makes it powerful

Goal: use R7RS Standard Scheme

  • I want it to work on a many Scheme implementations

  • Guile is the reference implementation

  • (more about this later)

Goal: able to run any init.el

  • Should be able to use init.el without significant changes

  • Many invest significant time in their configs

  • Suddenly not having your config is disruptive

  • Such an Emacs clone would be more useful

Why do this?

  • I personally like Scheme's minimalism.

  • Use Scheme as more than just an academic language.

  • Seems to be a lot of interest in a project like this.

  • Talk of "Guile Emacs" for about 30 years

A long history of Guile Emacs (1/3)

  • Early 90s: Initial discussion between RMS, Tom Lord, Aubrey Jaffer, begin work on replacing Emacs Lisp with Scheme.

  • 1999--2009: Ken Raeburn's Guile-Based Emacs. (My project is similar.)

    "This project that I (Ken Raeburn) have started is for converting GNU Emacs to use Guile as its programming language. Support for Emacs Lisp will continue to exist, of course, but it may be through translation and/or interpretation; the Lisp engine itself may no longer be the core of the program."

A long history of Guile Emacs (2/3)

  • 2010: Andy Wingo and Ludovic Courtes take maintainership of Guile project.

  • 2009--2011: Emacs Lisp interpreter implemented in Guile. Still ships with Guile.

  • 2011: Guile 2.0 is released

  • 2011--2015: Robin Templeton's GSoC project. (Is presenting later today!)

A long history of Guile Emacs (3/3)

  • 2020: Vasilij Schneidermann published an overview called "The State of Emacs Lisp on Guile".

  • 2020 to present: Guile Emacs is dead? Andrea Corallo, GCC Emacs, JIT-compiler for Emacs Lisp based on "libgccjit", brings into question any need for combining Guile with Emacs.

Demo

GUI is barely working

  • I have almost no experience with Gtk or GObject Introspection

  • Hard to debug, crashes at C-level produce no stack traces

  • Using GDB requires rebuilding all of Gtk, GIO, GLib, etc.

Emacs Lisp parser based on Guile Emacs Lisp

  • Foked the Guile Emacs Lisp implementation for easier development

  • Have already submitted a patch to the parser upstream

Emacs Lisp interpter is barely working

  • Implementing my own interpreter in portable Scheme

  • Monadic pattern matcher

Can parse but not interpret "subr.el"

  • "subr.el" is the first ELisp file run by Emacs

  • A good way to determine what to work on first

A call for help

Latest Emacs has 1,393 built-in functions

  • I could never implement that many functions alone

  • Probably not all are required to create a useful editor

My job is to make contributing easy

  • Document the build and test process

  • Document the system architecture

  • Prioritize which built-in functions are most essential

  • Find low-hanging fruit, use as means to teach others

The work for which I will take responsibility

  • Clone enough Elisp to be able to run ERT tests

  • Then use GNU Emacs's own regression tests to test patches

  • Make sure there is a usable GUI

  • (Someday?) be able to contribute a patch from within

Quick architectural overview

The editor is based in Scheme, not Emacs Lisp

  • Config, scripting, packages all done in Scheme

  • Use of Emacs Lisp for scripting not encouraged

  • Should still be able to run your init.el

  • Ideally should be able to run ELPA packages

Difference with Robin Templeton's project

  • Guile-Emacs links Guile runtime into Emacs

  • Not a Scheme application

  • An IDE for Schemers

Emacs Lisp is an "environment"

  • "Environments" are a feature of Scheme

  • Scheme procedures can be called from Emacs Lisp

  • Scheme state can be mutated by Emacs Lisp

  • (See "./gypsum/elisp-eval.scm", "new-env")

"Functional Lenses"

  • Because R7RS does not standardize MOP (not even in "large")

  • Inspired by Haskell

  • Composes getters and setters

  • Single source file, easy to port

  • Ported to 3 other Schemes

A lot of work went into keymaps data structure

  • Keybindings are an important part of Emacs

  • Had to do this well from very beginning

  • Keybindings work correctly in demo

A lot of work went into separating GUI from Editor logic

  • "Parameters" are a feature of Scheme

  • Platform-specific APIs are always parameterized

    • Windowing and widgets

    • Translate key events to bindings

    • Evaluating Scheme expressions

    • Text buffering and rendering

  • (See "./gypsum/editor-impl.scm")

Monadic pattern matching

  • Simpler, more portable

  • (Not as feature-rich)

  • Easier than porting SRFI-241 ("Match") to Guile

  • No relation to SRFI-247 ("Syntatic Monads")

  • You can still use pattern matching

Monad pattern matching

Example program

(define push-stack (put-with cons))
(define collatz
  (many
    push-stack
    (either
      (try (check (λ (n) (<= n 1)))
                  (success))
      (try (check odd?)
           (next (λ (n) (+ 1 (* 3 n)))))
      (try (check even?)
           (next (λ (n) (quotient n 2))))
      (fail "not an integer")
      )))

Conclusion

Original presentation proposal

I would like to demonstrate an Emacs clone I have been writing in Guile Scheme for the past year, which I am tentatively calling "Gypsum". Unlike other editors which only clone the Emacs keybindings (Edwin, Jed, jEdit, Jove, Lem, MG, Yi, Zile), I hope my Emacs clone will also fully clone the Emacs Lisp programming language well enough that many of the packages in ELPA, Non-GNU ELPA, and perhaps even MELPA, can be used in "Gypsum" without any modification. I would also like to talk a little bit about the how I am implementing it (the software architecture), and invite others to contribute.

I think my project is of interest to many Emacs users because, firstly, I have personally spoken with a relatively large number of people who have expressed interest in making Emacs programmable in Scheme. Secondly, there is a good amount of prior art for Scheme implementations of Emacs. There are even builds of Emacs that link to Guile which provides a "scheme-eval" built-in function that translates between Elisp data types and Scheme data types. The Guile compiler itself ships with an Emacs Lisp compiler as well, although it does not provide enough of Emacs's built-in functions to be of much use.

So by using Guile, we can make use of a lot of the prior art, in fact I am currently using the tokenizer and reader used in Guile's built-in Elisp interpreter to implement "Gypsum's" Elisp interpreter. That said, I have gone out of my way to make my code fully R7RS compliant, so I hope I can port it to other Scheme implementations like MIT Scheme, Gambit, Stklos, and perhaps Chez Scheme with Gwen Weinholt's R7-to-R6RS translator. I consider the Guile version of Gypsum to be the reference implementation of what I hope will become a fully cross-platform programming language and text editor written in portable R7RS Scheme.

The reference implementation of "Gypsum" is a GUI application based on Gtk using a library called "Guile-GI". Guile-GI uses the GObject Introspection framework to automatically generate Scheme language bindings to libraries like Gtk and Glib which are written in the C programming language. There is not yet any terminal-emulator version of "Gypsum."

The next step of the project will be to implement enough of Elisp that we can run tests written in the Emacs Regression Testing (ERT) framework. We can then incorporate the original GNU Emacs regression test suite into Gypsum. Any new API added to Gypsum Elisp will most likely already have regression tests we can use to make sure it is working in a way that is compatible with GNU Emacs Lisp. I would like to make it as easy as possible for people to contribute to this project, and having a list of APIs to be implemented each with a set of regression tests the APIs are expected to pass, is a very good way to do that.

About the speaker:

My name is Ramin Honary, I have been a professional software engineer of 16 years, lately mostly doing full-stack software development. I have always been fascinated with programming languages, and especially functional languages like Lisp and Haskell. I have been using Emacs since 2017. But lately it is with Scheme that I have been spending most of my free time. I am only a Scheme programming enthusiast, I am not involved with Scheme professionally.

You may also like another talk by this speaker: EmacsConf - 2022 - talks - Build a Zettelkasten with the Hyperbole Rolodex

Discussion

Questions and answers

  • Q: Would it be possible to support a GUI toolkit other than GTK? Like how GNU Emacs still supports Lucid
    • A: Yes this planed by having proper backend: emacs-lisp running into a module and the GUI being another module. So normalized communication. Currently GTK being standard implementation, also done here.
  • Q: Do you plan to provide improvements to Elisp as a language, or is the focus on a compatibility layer to facilitate doing all new extensions, etc. in Scheme?
    • A: Plan is to keep up-to-date with new releases. So new GNU feature should be included with each release. But also intend to have support for pure Scheme features.
  • Q: If Emacs Lisp support for Guile was documented better, could you be nudged/convinced to (re)start using, and contributing to that?
    • A: Compatibility is the most important things. Documentation not sufficient to convince users to switch.
    • IRC: Where do you think elisp documentation should be improved? I've always found the built in documentation to be excellent
      • janneke is referring to Guile's ELisp support, I believe; sorry, i meant the documentation of guile's elisp backend
  • Q: Why is being able to interpret all of `init.el` an useful goal? Sure, there is a lot of code written in elisp - can we consider a translator like utility to convert elisp to scheme, once guile-emacs becomes a reality? 
    • A: Probably, but first step is getting the interpretter working. Emacs-lisp basically compiled down to intermediate representation of the guile compiler [this was one of the hard things to get to work]. But unclear how this works for other schemes. Best solution probably translation elisp -> scheme, but this is not the approach that was done. Would be very cool to have. Feel free to give a PR.
  • Q: What is the plan to handle elisp packages that depend on 3rd party/external libraries? (libgit/magit or rg/ripgrep)? 
    • A: Will be tricky. If loading directly into the elisp process, very hard. Cairo could help, but you need emacs lisp binding on top of that. For magit, you can call regular process communication via stdin/stdout; so you can reuse existing scheme libraries. Dynamic libraries not a goal. Rg/ripgrep probably the same with process communication.

  • Q: Why is it not feasible for the Emacs layer that interprets Emacs Lisp (the core in C) ot have a Scheme interpreter, instead of using Guile?

    • A: Guile is a scheme. Not sure what you mean.
    • A: Check presentation later of Robin Templeton ("Beguiling Emacs: Guile-Emacs relaunched!"): the attempt exists by translating elisp to guile. 
      • thank you

  • Q: Not really a question, but how about Schemacs as a name?

    • A: Cool name, but did not check if it is already used. Feel free to discuss by email.
  • Q: I'm curious to know how the hell guile-emacs deals with all of the dynamically scoped modules out there. Is there any effort to automatically modularize and namespace stuff?

Notes

  • oo neat, i didn't know about that first bit of history
  • i've heard rms say that scheme (guile) is just a nicer lisp; but didn't know there were concrete talks/attempts to use guile for emacs that early
  • robin: yes, guile-elisp not being portable might be a showstopper for ramin
  • I've heard good things about guile from guix people. Never really tried it out
  • FWIW, I think there have been various attempts to make an Emacs clone in Common Lisp. I guess lem is currently the most active. https://github.com/lem-project/lem/
  • I like how he edited his slide mid-presentation.
  • https://www.emacswiki.org/emacs/GuileEmacsHistory has some info on very efforts; i was surprised that there were so many (especially with old-school guile, eww ;))
  • joining for all the guiles and all the emacsen
  • He's got a long way to go.
  • of course there were, RMS had decreed that guile was the GNU scripting language and it was a bit embarrassing that it was so little-used...
    • At least Guix uses it now.
    • oh yes, guile 3 was a great step forward, and I do wonder how much of that was due to the impetus of its having real users :)
  • if you're interested in guile emacs, be sure to check out robin's guile talk this afternoon
  • nice, my silly https://gitlab.com/janneke/guimax also used guile-gi but this looks much more mature
  • developed actively til 2014, but more recent work is on a branch so may not be as obvious...
  • so...i guess that some basic documentation on elisp may be very helpful
  • wbn if you could join efforts somehow
    • there should definitely be some overlap between the projects
  • I have 6500 interactive ones, according to Vertico...
    • I got 8021 interactive ones ;)
    • 7690 here
    • 34557 callables \o/
  • working towards a similar goal approached from different directions
    • however, working on guile's elisp backend may be a common ground
  • ramin's probably talking about subrs, i.e. primitives not themselves implemented in elisp
  • you mean providing modularization for elisp programs? or something else
  • Being more specific, removing the need to namespace every single internal variable/procedure in an elisp module. Maybe that isn't a goal, but I wish it were
    • yes, i have some ideas for adapting the CL package system for that. it'd have to be opt-in, maybe some tools for automated refactoring
  • And... the part where we need more bandwidth for any core runtime efforts to be viable
  • my-special-module--loop-variable-3
  • How can I get involved with this? if I want to contribute
    • hang out in #guile-emacs and/or subscribe to the mailing lists https://guile-emacs.org/
  • an embryonic re-implementation by ramin of emacs in guile, with their own new elisp interpreter that should be r7rs compatible
  • would love robin's guile-emacs and ramin's efforts to somehow share some of their efforts
  • Robin's talk mentions developing a better elisp in Scheme. Why can't your project leverage it?
  • guile-elisp is part of guile's compiler system (elisp -> tree-il -> cps -> bytecode), unless another scheme sets out to be guile-compatible in that respect it won't be portable at all
  • To be fair, I've been screwing around with chicken, guile, and racket. I haven't found any 2 scheme implementations to be compatible, even within SRFI implementations
    • Just the basic syntax and semantics, nothing else
    • yeah, scheme's lack of portable libraries in practice motivated me to suggest something that's sure to piss everyone off: Javascript
  • YouTube comment: Cool, great, amazing. Extremely ambitious. A clone like this project might be significant harder than for example Lem, which does not care about backward compatibility with Emacs. A clone also is somewhat harder to create a unique selling point for. Especially if Guile-Emacs is also going to be a thing now. Just my 2 cents up to discussions. Anyway I am excited and will follow it:)

Transcript

Hi, my name is Ramin Honary, and I'm here to talk to you today about my clone of Emacs and Emacs Lisp that I've written in Scheme so far. So I am an Emacs enthusiast since 2017, currently employed as a full stack developer, mostly working with Python and JavaScript, although my true love is functional programming, especially Haskell, and Scheme. I started learning Scheme about two years ago. And for the past year, I've been working on a project that I'm tentatively calling Gypsum. Naming things is hard. It's not a great name. I'm open to suggestions. But yes, this is the project in which I am trying to write an Emacs Lisp interpreter in Scheme. There are many clones already of Emacs. You've probably heard of Edwin, Jed, Jedit, Jove, Lem, MG, Yi, Zile. Edwin itself is also written in Scheme--MIT Scheme. These only clone the key bindings of Emacs and not Emacs Lisp itself. The only alternative to GNU Emacs that I'm aware of is XEmacs, which is a fork of GNU Emacs. Most people don't use Emacs for the key bindings. I mean, this is anecdotally speaking, but the people who I've talked to, I would say don't use Emacs for the key bindings. They use it really more because of the power of Emacs Lisp. Emacs is as powerful as any system shell, perhaps even more powerful than system shells like Bash. The reason why it's so powerful is because there's a good programming language which you can use to control everything on your system. You can control processes. You can load and save files. You can create files. You can configure things. You can capture the output of processes in buffers. You can filter text through buffers. And a good programming language is what you need in order to do all of this. So one big goal of this project is to try to stick as closely as possible to the R7RS standard Scheme definition. That is the latest Scheme standard: R7. And this is just because I want my project to work on many scheme implementations, not just Guile. Although Guile certainly is the reference implementation. So another goal is to be able to run any "init.el". So you can take your existing "init.el" and run it in my program without significant changes. That's one of my goals in the end. I should be able to do that. A lot of people invest significant time in their configs, and it's kind of disruptive if you want to change editors, not be able to use your Emacs Lisp config. And so I think a useful Emacs clone would be able to clone Emacs Lisp well enough that you can run your "init.el". And so overall, why am I doing this? It's just because I like the Scheme programming language. I love its simplicity and its power. It's an extremely well thought-out language. It's one of those languages where you can understand the entire language from top to bottom. You can read the entire specification and understand it yourself. It's like the computers I grew up with when I was a kid. They were all very simple computers in the late 80s, early 90s. And back then, theoretically, an engineer could understand the entire system at the software level all the way down to the circuit level. You can't do that nowadays. And so nowadays, my computer is not really a physical computer anymore. It's the Scheme language standard itself. That is the core of computation, of all of computation for me. And I would like to use it as more than just an academic curiosity. It was originally designed for teaching at MIT, but it's found use in industry. And the R7RS standard is still relatively new. It's over 10 years old at this point, but hasn't, I mean, the Scheme ecosystem itself is already fairly small. There still, I don't think, has been a whole lot of adoption of R7RS quite yet. Kind of a shame. So I'd like a project like this, a very large scale, kind of a killer-app-like project where you're developing a text editor and perhaps even an integrated development environment in Scheme, I think would be very useful just even as a study of, you know, what can this language do? And just overall, there seems to be a lot of interest in Guile-based Emacs and well, maybe a Scheme-based Emacs, but Guile in particular. There has been talk of changing Emacs Lisp or the core of the Emacs Lisp over to Guile for about 30 years or so, talks originally in the early mid 90s. There were discussions between Richard Stallman, Tom Lord, and Aubrey Jaffer. They considered actually replacing Emacs Lisp with Scheme. In 1999, and going for about 10 years, someone named Ken Raeburn actually started a project where he started writing Emacs in Guile. My project is very similar to this. Here's a quote from his webpage, which is still up, even though it hasn't been updated in 15 years. This project that I have started is for converting GNU Emacs to Guile as its programming language. Support for Emacs Lisp will continue to exist, of course, but it may be through translation and/or interpretation. The Lisp engine itself may no longer be the core of the program. And this is my goal as well. In 2010, Andy Wingo and Ludovic Courtes took maintainership of the Guile project. From 2009, so while Andy... 2009 to 2011, the first Emacs Lisp interpreter was already being implemented in Guile. And even to this day, this Emacs Lisp interpreter ships with Guile. And so this was happening while Andy Wingo took control of the project. In 2011, so shortly after Andy Wingo took control of the project, Guile 2.0 was released. And also in 2011, in the summertime, someone named Robin Templeton, I believe it was a Google Summer of Code project, started actually trying to incorporate libguile, that's the guile interpreter, as a linkable or loadable library, linking it to the Emacs executable, and then providing some built-in functions in Emacs that allows you to call the scheme interpreter, the Guile Scheme interpreter, from Emacs. And so it's not like a wrapper around the REPL like Geiser or SLIME. It's actually the whole Scheme interpreter loaded into your Emacs process. And that means your Emacs will have the ability to actually load compiled Scheme programs and actually run them and share memory with Emacs Lisp processes. And, well, Robin Templeton will explain all of this. They're presenting today, and I'm very excited to actually see their presentation. They'll explain everything. So, let's see. Moving on. 2020, someone named Vasilij Schneidermann, I'm not sure how you pronounce that, published an overview called The State of Emacs Lisp on Guile. Let's see if I have that here. Yep, it's this page right here. He goes into detail about who has done what so far, and what can you do in Guile with Emacs Lisp so far, and so on. Like, what is the state of the project overall? And so (speak of the devil) (Andy Wingo on social media). So, 2020 to present. Guile Emacs is dead? So there's GCC Emacs now. Emacs Lisp now has its own JIT compiler. And it seems like over the past few years, Emacs Lisp has kind of moved off into the direction of becoming its own programming language in its own right, and it is decidedly Common Lisp-flavored. It is very similar to Common Lisp, and that seems to be the direction that it's headed now, and I don't know if there's really any interest anymore amongst the Emacs maintainers of continuing with a Guile-based Emacs. But as far as I know, there's still a lot of interest in the community amongst Scheme and Lisp and Emacs users who are interested in maybe continuing to try to get Guile to become the core of Emacs, or if not, you know, what Robin Templeton has been doing, at least trying to get Guile a language, a first class supported language in Emacs. So that's enough talking. Let me just show you what I have so far. The GUI is barely working, because I have very little experience with GTK or GObject Introspection. It's very difficult to debug, so it's very slow to develop. Any crash at C level produces no stack traces. So far, most of the crashes that I've experienced are due to simple mistakes like passing the wrong data type. So, so far, no, not a whole lot of need for GDB or rebuilding all GTK, glib, and so on with the debugging symbols. But yes, still development's been very slow. I'm learning as I go. I've chosen to use Guile GI as the foundation for the GUI. Let me just load it up quick here. "load main-guile.scm". And this will launch the GUI. I also happen to have a REPL that runs in a separate thread and submits any form that you type to be evaluated inside of the running GUI environment. But you can just type stuff. So "hello world." And of course there is... as you can see, it's not quite rendering correctly. This "Messages" thing here, that should be over here, obviously. I haven't been able to figure out how to get those little details down. But yeah, you can do M-:, and you get your eval, and you can just evaluate, like (what's an emacs,) (or what's a Scheme-specific thing?) Like "(import (srfi 1))", and let's see, do "(iota 20)", for example. And so that is the procedure that iterates and produces some 20 elements of a list. Or you can do something like, let's see, string-append "hello" with space "world". And you get the result and so on. And, you know, scheme allows you to return multiple values. So what I have done here is just every value is captured in a list and it prints all of the return values in the list. So if a procedure returns no values, you get an empty list. And that's that. It's still quite buggy. So like, here's a bug that I can reproduce fairly consistently. I can, yeah, if you do... there seems to be a problem with a widget being freed too soon, so it will crash. I'm going to try and solve that, hopefully, before this presentation goes live. Let's see here. The Emacs Lisp parser is based on Guile Emacs Lisp. So the Guile Emacs Lisp interpreter that ships with Guile, that is what I am using. I've actually copied and pasted the source code from the Guile source base into my own project so that I can iterate on it more quickly. And I've already had to make some modifications to the Emacs Lisp interpreter in Guile. So here's the evaluator. I've actually already modified the parser and the lexer a little bit. And it's at least able to parse all of the "subr.el" program, the Emacs Lisp program. It can actually load that, but not evaluate it, or parse it, but not evaluate it... Read, not eval. By the time this goes live, I will have submitted a patch upstream. And that's another goal of this project, incidentally, is that anything that we can contribute to Guile and any built-in functions that we can implement I would like to, for this project, I would like to try and contribute upstream to Guile. The Emacs Lisp interpreter is not working well, unfortunately. So this copy, this is the copy of the code base (from this commit in particular) and well, I can't get it working. I can't actually get the non-copy, the actual built-in version of the Emacs Lisp interpreter to work properly quite yet. So let me quick go to, (what is this here?) Guile Elisp. So suppose you have this "eval-elisp" procedure here and it takes an Elisp environment and then it evaluates an expression in that environment. And evaluates to a value. So this is the standard way of doing it in Guile. If you can see here, you've got this expression, "compile" expression. This is like "eval". And so actually trying to load this. So let's do "load gypsum". (Let's see here. This is, no), I wanted to "import gypsum backend guile Elisp". And if I actually want to do this... So elisp eval, first of all, it says it failed because there's an unbound variable "elisp-eval". Don't know what it's talking about. There's no such variable in any of my programs. I have no idea what's going on here. You can try to run eval elisp on some simple form like (+ 1 2). And it gives you this exception. This works. This is the same issue that I have with all of the, every version of the Emacs Lisp Interpreter in Guile. I can get it to work with this big ",L" mode. So I can actually do (+ 1 2) here. I can do "princ" like here. That all works fine. It gives me, for some reason, a stack trace here. And yeah, so it's a bit, it's not well-documented. The code base is fairly old. As I said, it was developed around 2011, and it's fairly opaque, and I have not been able to figure out how to get Emacs Lisp in Guile working smoothly. So I have started writing my own Emacs Lisp interpreter. And, uh, "gypsum/elisp/eval-tests.scm". It's, uh, not entirely ready. I can show you some of the tests at least. Here is a simple Emacs Lisp program that you can evaluate. You got "progn", "setq" a to 3, "setq" b to 5, "setq" c to the sum of a and b, return c. And this at least works correctly. As you can see here, the result is eight. Um, but the "let*" semantics are not completed yet. Lots of work left to do there. So in the time I have left, I guess I can just, talk a little bit about what my plans are for the future. I would like to begin by evaluating or actually loading the "subr.el" into my Emacs Lisp interpreter. I actually have tests set up for that as well, so I can actually select any form I want from "subr.el". I can just run this through my interpreter and test to see if everything is working once I get that far. And yeah, let me just say that this is my formal appeal to the community for help on this project. Emacs Lisp has 1,393 built-in functions. I could never implement that many functions on my own, so if this project is going to be useful to anybody in any reasonable amount of time, I'm going to need help. And I know that there are people out there who are very interested in a Guile-based Emacs, and so if you're watching this, please feel free to contact me on social media or over e-mail. My job, the way I see it, is if there's enough interest, and I do get a lot of people interested in starting to contribute, my job will be to document the building and testing process and make sure that it is as easy as possible to contribute code to this project. I want to document the system architecture. I'll write blog posts. I'll do videos on PeerTube explaining how everything works. And I will prioritize which built-in functions I think are probably going to be the most necessary, the most essential to get the interpreter running, and then find low-hanging fruit, functions that are easy for people to implement as a good introduction to getting them started on contributing to the project. And then, of course, I will take responsibility myself of making sure that we can get the Elisp interpreter to the point where it can run the Emacs regression tests. These are the test suites that are used to test Emacs Lisp itself in the GNU Emacs code base. And so ERT is itself written in Emacs Lisp. And so I think if we implement enough of the built-in functions to be able to run ERT, then we can actually start using the GNU Emacs regression tests to test our own interpreter, our own Emacs clone. And of course, I'll make sure that there's at least one usable GUI. I'm currently working on Guile GI and GTK. It would be great to have an... ANSI terminal based... something that works in your terminal emulator. And yeah, it would be great if someday soon, hopefully, we get enough done that you can actually contribute a patch to this project from within the Gypsum editor itself. I was going to do an overview, but that would be for more of an hour-long presentation. So I'm out of time. I guess the last thing I should quickly say is there's no meta object protocol in this project. I think that's a little bit too difficult to port to various scheme implementations. So I've created a substitute, which I'm calling "functional lenses", which is inspired by the Haskell project of the same name. Everything in this project is based on functional lenses. Yeah, also a lot a work went into the keymaps data structure. The point being that I think I have a pretty good foundation here upon which we can build, even though there isn't an actual, there isn't a lot done in the actual prototype itself, not yet anyway, but I made sure to get the fundamentals down from the beginning. And so I think we have something like a solid foundation on which to build. So, I'm going to conclude it there. And here's my contact details. Like I said, this is a project, I'm appealing to the community of all people who are interested in Guile and Emacs to help contribute to this project. I see myself as just getting the ball rolling. Again, taking-off from the work that Ken Raeburn left behind, with my own from-the-ground-up implementation. So yeah, contact me: e-mail, you can take a look at my blog where I talk about what I have done. My source code, the code for this project, is up on Codeberg... The presentation... this presentation, the home page for this presentation, you can find more details there. Oh, I'm on ActivityPub as well, so my handle is @ramin_hal9001@fe.disroot.org, and I'm on everyday. So yeah, please feel free to contact me if you're interested, and thank you for your attention.

Captioner: ramin

Q&A transcript (unedited)

Troy Hinckley's project that I'm talking about. I was going to mention this in my presentation, but it's possible, theoretically, that Troy Hinckley, his project could be used as a scheme of limitation that actually runs my own version of Emacs. And although, you know, This is completely theoretical, and I don't know how difficult that would be. But if Troy Hinckley implemented enough of the R7-RS standard in Rust, it would theoretically be possible to run the Gypsum editor in Troy Hinckley's own editor. I thought that was kind of interesting, and I thought it was worth mentioning, at least in the questions and answers. I also mentioned this in the presentation. I wanted to see Robin Templeton's project presentation, but unfortunately it's going to be at like four in the morning for me. So I'm going to try and watch that tomorrow, but that's also going to be a very interesting project to keep an eye on if you're interested in Scheme. That's the project where you've got the Guylain interpreter running inside of the Emacs process. It's dynamically linked as a library. I'm ready for questions from anybody. You can ask or you can type. It's up to you. Okay, let me check the etherpad. Let's see here. I'm not sure if I'm doing that right. Let me check one more time. Oh, there it goes. Let's see, so this is... I didn't know about that first bit of history. Oh, I've heard RMS say that Scheme Guile is just a nicer Lisp, but I didn't know there were concrete talks attempts to use Guile for Emacs that early. Let's see, that was from janneke.
[00:03:09.370] Q: I'm curious to know how the hell guile-emacs deals with all of the dynamically scoped modules out there. Is there any effort to automatically modularize and namespace stuff?
I'm curious to know how the hell Guile Emacs deals with all the dynamically scoped modules out there. Is there any effort to automatically modularize and name? Let's see. That might be a better question for Robin Templeton. In my own project, there's no module system for Emacs Lisp. There is a module system for Scheme. And the Emacs Lisp interpreter runs in its own environment. the require system or whatever module system that Emacs has, once it's implemented, all of that would just happen inside of the Emacs Lisp environment, which is inside of the Scheme environment. And environments are objects in Scheme. I think a more difficult question is how to handle threading, and Scheme has very good threading built in, in Serphe-18[??]. But I don't think it will be easy to write Emacs Lisp form bindings to the Scheme multi-threading implementation. Emacs Lisp was just not cut out for that kind of thing. So I think each Emacs Lisp, you could, I suppose, have multiple threads each running their own Emacs Lisp environment. Scheme would make that very simple to do. And then there'd just be a question of how you would get those different interpreters to communicate with each other, perhaps using the same protocol that's used by the Emacs server. But I haven't thought that far ahead yet.
[00:05:23.646] Q: Would it be possible to support a GUI toolkit other than GTK?
Would it be possible to support a GUI toolkit other than the GTK? Like, how is it still supports Lucid? Yes, this is absolutely a goal of the project. I'm trying to keep the back end separate as possible. The scheme has what you call parameters. And these are like global variables that are still somewhat thread safe. And every call to the GUI goes through a parameter. So the Emacs, the interpreter and the editor logic is all in one module. And then that module calls out into a separate GUI module. And then you can implement different GUI modules. So you could have one for GTK3, one for GTK4, if you want to write the extern C bindings around Qt or full tick, that would certainly be possible as well. It would be nice maybe to have an SDL implementation based maybe on Chikiti or some kind of immediate mode GUI, something like that. But definitely GTK3 through Guile GI is the reference implementation. Things start there. But I'm very interested in supporting other GUIs, yes. Let's see.
[00:06:45.257] Q: Do you plan to provide improvements to Elisp as a language, or is the focus on a compatibility layer to facilitate doing all new extensions, etc. in Scheme?
Question, do you plan to provide improvements to ELisp as a language or focus on a compatibility layer to facilitate all new extensions in Scheme? Yeah, the second one. I want to move off to Scheme. I would like for this project to try and keep up to date with each new release of Emacs and Emacs Lisp. That's a difficult moving target to follow, I realize. But to the greatest extent possible, any new features to Emacs Lisp will be pulled in from GNU Emacs. If we happen to be able to implement something cool in Scheme, and be able to port it over to Emacs Lisp, then sure, it'd be nice to be able to upload or to submit that upstream to the GNU Emacs. But I think I would prefer to have new features written in Scheme. I would like this gypsum to be more of a Scheme app platform that just happens to be able to also run Emacs Lisp. That's how I see it. Of course, this will be a community project. I'm open to debate about that if anybody wants to convince me otherwise. Why is being able to interpret all of that EL a useful goal? Sure, there is a lot of code written in Elisp. Can we consider... Oh, it's still being written. Please go ahead and finish writing.
[00:08:29.673] Q: Can we consider a translator like utility to convert elisp to scheme, once guile-emacs becomes a reality?
Can we consider a translator like utility to convert eLisp to Scheme once Guile-Emacs has become a reality? Certainly. For the time being, I just wanted to get the interpreter running. So the actual, the Guile-Emacs Lisp, the one that was written in 2011 that I didn't write, that actually does compile to, I think it's the tree intermediate representation It's one of the intermediate languages that Guile uses to compile Guile scheme itself. So the Emacs lisp that was written before actually does that. It actually compiles and makes use of the entire Guile compiler tool chain and actually produces like JIT compilable binaries, which is really cool. Like I said, that's the one that I had trouble getting to work properly. Maybe we can follow that architecture. I'm not sure how to do that, but I would like to be able to do some kind of translating, keeping in mind that we want to have this be portable, do various schemes. And so Guile makes this very easy, but other schemes don't. Gambit might do this pretty well as well. It compiles to C and then compiles C down to a dynamically linkable library. So yeah, I think probably the most portable, I'm just thinking out loud right now, most portable implementation will just be able to translate Emacs Lisp directly to Scheme, which is not what the old Guile Emacs Lisp implementation does. That goes to TreeIL, so it's very, very Guile-specific, can't be ported. But yeah, if we could somehow get Emacs Lisp translated to Scheme and then compiled, say, in Shea Scheme or Gambit or MIT Scheme or one of those other compilers, that would be very cool. And I would absolutely love to do that. And I would very quickly accept any code into the code base that would do that.
[00:10:54.390] Q: Why is being able to interpret all of `init.el` an useful goal?
Oh, and to answer the question about init.el, It's just because people spend a lot of time on their configs and it would be nice if, you know, you're starting to use this new editor and want it to be similar to Emacs users, just the Emacs community in general and people who are familiar with using Emacs. It would be more useful to everybody in the Emacs community if this were more compatible with GNU Emacs. And so that's why that's, I think that's an important goal. Question is not yet. Great. Oh, here comes another question.
[00:12:08.539] Q: What is the plan to handle elisp packages that depend on 3rd party/external libraries? (libgit/magit or rg/ripgrep)?
Okay, what is the plan to handle elisp packages that depend on third-party or external libraries like git or magit or ripgrep? So that's going to be tricky. It depends on how these external packages are linked into emacs. If it's going to be a dynamic library like Robin Templeton's project which you load the libgit library into the Emacs process, that is going to be extremely difficult. So if you have an external library like, I don't know, libgit or what's the GUI thing? Cabal. No, not Cabal. Cairo, libcairo to do SVG graphics and so on. You can do that very easily with Guile, but then on top of that, implementing Emacs list bindings to it, I mean, you've got two layers there, and that makes things pretty difficult. So it's possible. And to some degree, maybe necessary for example, Cairo, if we want to do SVG graphics the way that Emacs Lisp does, we're going to have to have that. So that would be necessary. We would have to have those two layers. Yes, let's do that. But if it's like for Magit, you can just call out to your git process, and then you're just using the regular process APIs that Emacs Lisp has. And that can be, already we, like Guile has some very good implementations for process management. And so it would just be a matter of wrapping up those in the Emacs lisp form bindings. So yeah, dynamic libraries, I wanna try to avoid. And I would prefer to do things more through, you know, launching a child process in the Emacs process. and then communicating over the standard in, standard out channels. That's the easier way to do things, I think, because then you can just use the process library that Emacs already has, and you can just reuse all of that code. I'm not sure how ripgrep works, unfortunately, but I believe that's also a process, a child process. So, we can just reuse all of the Emacs Lisp code that does that already. We just need to make sure that the process management implementation and scheme is properly bound to Emacs Lisp, and it works the same as GNU Emacs does. Once that's all set, then these porcelains, like around git, should fall into place. without too much difficulty, hopefully.
[00:15:21.112] Q: Not really a question, but how about Schemacs as a name?
How about Schemax as a name? I like the name. I like that name. I haven't really looked into like, is that already used or is that going to be confusing? But certainly something we can discuss. Another thing I should mention, I should probably set up a server or something like Discord or something like that. Discourse, not Discord. Discourse, the open source one, where we could actually chat about this stuff. For the time being, ActivityPub, mostly Mastodon, is how I communicate with people in real time, that or email. So if you want to get a hold of me, check the notes for this presentation and just send me an email. Any question at all is fine. If you want to contribute code, if you want to just learn how to contribute code, send me any questions. It's fine. I'm happy to answer them. And we can talk about the name as well.
[00:16:45.931] Q: Why is it not feasible for the Emacs layer that interprets Emacs Lisp (the core in C) ot have a Scheme interpreter, instead of using Guile?
Okay, why is it not feasible for the Emacs layer that interprets Emacs Lisp, the core in C, have a Scheme interpreter instead of using Guile? Let's see, I have to, okay. Emacs layer interprets Emacs Lisp, the core in C, have a Scheme interpreter instead of using Guile. Okay, so that, the question xlarsx is asking, xlars, x, So Lars is asking, is it not feasible for there to be an Emacs layer that interprets Emacs Lisp have a scheme interpreter? This is Robin Templeton's project. And they're presenting later today. So check the roster and be sure to see that presentation because that's exactly what Robin Templeton is doing. That's not what I'm doing though. I'm trying to create something in Scheme. But yes, there is an attempt to get an Scheme interpreter to run inside of Emacs itself. And it has its own method of binding to Emacs Lisp functions and translating data like Lisp structures between Guile Scheme and Emacs Lisp. Robin will explain all of that in their presentation. OK, I think I've got through all the questions on Etherpad. But I'm going to hang out here for a bit longer. And yeah, feel free to do a video chat with me or send me more questions on Etherpad or here in the big blue button. And so I'm just going to hang out. And thanks for asking all your questions. And yeah, I look forward to working with all of you if you're interested. take it easy. Thanks so much for the talk and looking forward to seeing some of your progress as this moves forward, exciting space. We'll go ahead and leave the room open for you and thanks for offering to hang out and chat with other people that come by. Feel free to throw something in the chat if you want to remind people you're still here. Meanwhile, on the stream, we have moved along to our next talk on Rust, and that is just getting started. But again, we're continuing to record this, and I'll just keep an eye on it to stop the recording. Thank you. Thank you. It was awesome. So it seems like it's slowed down here for the Q&A. I don't see anybody else on BBB, so I'm going to go ahead and stop the recording. We can start it back up. I would say, yes, there's a lot of things you can do with this. You can handle processing. Yeah, I'm going to try and join over the chat for the next talk. I'm not sure if I can do both big blue buttons at the same time. You should be able to just watch your mute settings and mute tab settings and whatever all you have to avoid bleed through. Okay.

Questions or comments? Please e-mail ramin.honary@gmail.com

Back to the talks Previous by track: Transducers: finally, ergonomic data processing for Emacs! Next by track: An experimental Emacs core in Rust Track: Development