Back to the schedule
Previous: Emacs Lisp native compiler, current status and future developments
Next: Turbo Bindat

Old McCarthy Had a Form

Ian Eure

Name Pronunciation: (EE-un YOU-r)

Pronouns: he/him/his

Preferred contact info: ian@retrospec.tv / ieure on Libera

Q&A: IRC
Duration: 12:44

If you have questions and the speaker has not indicated public contact information on this page, please feel free to e-mail us at emacsconf-submit@gnu.org and we'll forward your question to the speaker.

Talk

Download video, 12:44, 24.9M

Download --compressed56.webm (13.8MB)
Download --main.vtt
View on Toobnix

Q&A

Download video, 2:41, 5.5M

Description

http://atomized.org/blog/2021/11/28/old-mccarthy-had-a-form/

Most practical languages are multi-paradigm, offering several abstractions for the programmer. But did you know that Emacs Lisp comes with a powerful system for object-oriented programming? Join me for a discussion of EIEIO, and learn how it can help you write more modular, flexible Emacs Lisp.

Discussion

IRC nick: ieure

Q2: AFAIK, EIEIO is generally slower than, e.g. cl-defstructs. When do you think EIEIO is not suitable for performance reasons?
- A: I agree with Dmitry: first make it work, then make it fast. I don't think there's a blanket reason not to use EIEIO, but definitely profile if you're using it in a performance-critical context. EXWM is one project that uses EIEIO extensively and seems to perform well, so I don't think it's off-limits for performance-critical code.
Q3: Do you have any tips about introspection? e.g. IIRC there's an EIEIO introspection facility, though it may be somewhat primitive.
- A: It is somewhat primitive, but seems to work okay (https://www.gnu.org/software/emacs/manual/html_node/eieio/Introspection.html). I haven't found a need for anything fancier (yet).
Q4: Have you used any of the EIEIO-related serialization tools? IIRC there are some limitations with regard to printable/readable values.
- A: I haven't had call for this, but https://www.gnu.org/software/emacs/manual/html_mono/eieio.html#eieio_002dpersistent is the mechanism (for anyone wondering)
Q5: I did not get how generic functions can work with non class objects
- A: Dynamic dispatch is very powerful!
Q6:So with that Emacs is on pair with Smalltalk development environments now
- A: Not very familiar
Q7: Most of what you presented can be done without defclass. AFAICT, the only exception is multiple inheritance (since cl-defstruct also supports single inheritance via :include).
- A: Yes, you can mix and match structs/objects or any other type. You need classes if you want the EIEIO customization editing facility or MI. I think also initialize-instance is class-only, so you need classes if you have to do some kinds of complex (cross-slot) initializtaion.
I didn't know that custom.el works with EIEIO that way, very nice
Dang Ian. What a talk, great demos.
Wow, that's a great talk.
Great talk. So with that Emacs is on pair with Smalltalk development environments now
For reference, transient.el, which we all know and love as the engine that drives the magit interface, is written via EIEIO afaik.
I reckon I should look more into it, I've always avoided it because I was afraid it wouldn't be /quite as nice/ as CLOS or GOOPS.
- ieure: It's missing a few things (most documented in the manual: https://www.gnu.org/software/emacs/manual/html_mono/eieio.html#CLOS-compatibility), but it's solid and worth using.
  - Yeah when transient.el first came out I was impressed by how naturally it worked as part of that abstraction.
ieure: EIEIO all the things! I had to cut it, but you can use dynamic dispatch based on major-mode, like: (cl-defmethod whatever ((mode (derived-mode python-mode)))) and then (whatever major-mode).
Also really nice for things like 'window-system. I really like when callsites are clean and not cluttered with conditionals.
Can eieio do regexp dispatch?
- ieure: Not currently, but it's possible to add.
- okay, so I don't need to feel too bad about coding up my own vtable for those then
- ieure: This is the thing that implements (thing (eql :whatever)) specialization, should be a good starting point if you want (thing (string-match-p "^foo")): https://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/emacs-lisp/cl-generic.el#n1164
- thanks for the pointer, but I think I have some more pressing cl-defgeneric reimplementations to make before I touch that
- ieure: Extremely fair. One thing I didn't get to touch on is that you can extend generic functions from anywhere. So you don't have to patch up cl-generic.el, you can define a new method for a generic function defined anywhere, in any file. Which rules.
This is not a question: Brilliant title for the presentation.

Links:

https://www.gnu.org/software/emacs/manual/html_mono/eieio.html

Outline
What is EIEIO?
Why OOP?
The CLOS Model
- Classes
- Generic Functions
- Methods
- Specialization
- Method Qualifiers
- Multiple Inheritance
- Nice Properties
Practical Examples
- Encapsulation
  - Example: transmission.el
- Abstraction
  - Example: sql.el
- Extensibility
  - Example: comint
Conclusion

Transcript

My name is Ian Eure, and welcome to my talk "Old McCarthy Had a Form". In this talk, I'm going to be discussing EIEIO, which is an Emacs Lisp implementation of the Common Lisp object system. CLOS is a way of writing object-oriented Common Lisp code, and with EIEIO you have much of that same power but inside Emacs. I'm going to be using those two names interchangeably throughout this talk, since they're nearly equivalent. You might wonder, "Why would I want to write object Emacs Lisp code?". I like it because I like writing in a functional programming style, or I like to do an imperative style that captures interactive key sequences that I might enter manually. Well, I think, different kinds of programs need different kind of programming paradigms, and sometimes OOP is the one that fits. Also, if you've done much OOP before, you might be surprised by how EIEIO works and the sorts of power that it brings to your programs. So, let's talk about that model now. Classes are pretty much what you would expect if you've done OOP programming before. In the CLOS model, they only have fields, they only encapsulate values, they don't have anything to do with methods. So, in this case, we have a base class for an EMMS player backend, and it has one field (a slot is what it's called in CLOS terminology), which indicates whether it's playing or not, and it's declared abstract, so you can't create an instance of an object based on this class, you can only extend it with another class, that's an EIEIO extension, but I think it's a pretty good one. You can also see there's a class that implements an mpv player back-end, and it extends the base class, it doesn't add any slots, and those get inherited from the base class. If you want these to do much more than encapsulate data, you need to start writing methods. The CLOS model is to have a generic function. A generic function is kind of like an interface, it's just a name and an argument list, and there's no implementation, you have to write a method in order to have an actual implementation. When you call the generic function, the system will do a dynamic dispatch to a method that matches based on its argument types and how the method has declared that it should be invoked. Here's an example of some methods for our mpv player backend, you can see that it'll play anything other than something with a keyword of unplayable, and it just has dummy start and stop methods that return "Started" and "Stopped" text. A method is just one implementation of a generic function, and you can have as many as you need. In order to determine which method gets dispatched when you call a generic function, they're specialized based on their argument type. In this case, you can see that first argument says player talk/emms-player-mpv, that means that if that first argument is an instance of the mpv player or a subclass of it, then those methods will be invoked, unless there's a more specific one based on that argument type. You don't have to define the generic functions. If you define a method, then it'll implicitly define the generic function for you. Specialization is really powerful. It lets the methods define how they get invoked If you've done much programming in Clojure, this sounds a little bit like multi-methods, but with multi-methods, only the main method, the equivalent of the generic function, can determine how they're dispatched. So, as the writer of an interface, you might not be able to foresee every way that someone who's implementing a version of that interface would like to dispatch. CLOS doesn't have that problem, you get to define your own invocation semantics. You're also not limited to dispatching based on the value being a subclass of an EIEIO type. You can dispatch based on a primitive type like integer, you can dispatch based on the value, you can dispatch on multiple arguments, and there's also a default dispatch that will get applied if there's no others that are defined. If you don't have a default, then you'll just get an error from the system, and if that doesn't cover it, you can even define your own. Definition is with the cl-generic-generalizers, which is itself a generic function. Much of CLOS is built in CLOS, which I think is really cool. In addition to all that, you have four different types of methods, and those are distinguished by what's called a qualifier. Every function can have methods that have all four different types of qualifiers, and based on your class inheritance, you might have multiple of each type. There's the primary method, which is equivalent to the method in any other OOP system, so we're not going to cover that too much. Then there's a before method. This is evaluated before the primary method for side effects, and its return value is discarded. There's an after method, which is the same but happens after the method has finished evaluating. And then there's an around method that happens around all the other three. And by using these types of methods and using class inheritance to compose them into your classes, you can add some really powerful mixin type functionality. You can use before and after to build things like logging, and you can use around to implement things like memoization. If you've done much Emacs Lisp programming, those before after and around might jog your memory because they're the same features you get with Emacs's built-in function advice. The thing with function advice is that it only works on functions in the global namespace, and there's no kind of conditionality, they always get dispatched when that function is invoked. The nice thing about the CLOS system is that whether they get invoked or not depends on whether they exist in the class hierarchy, so you can add them to your class hierarchy if you want that extra functionality like logging or memoization, and you can exclude it if you don't. I think that's really powerful and a very interesting way of doing it. It also supports multiple inheritance, which is the mechanism that you can use to compose all these different kinds of behaviors into a single object that does all the things that you want. Here's a quick example of a logger. So, you can see the class just has a single slot called messages, it has a log method that pushes a new message, which is a format string, into that, and then it will return them back out, or it'll just return the latest. And there's a simple example that shows it logging, and then shows it coming back out, pretty much what you would expect. Here's another class that adapts the emms-player to the logger class. It only extends the logger because it doesn't need any features of the emms-player class itself. It just implements methods that dispatch based on it being that logging player class, and you can see it logs whenever a track is started or stopped, and it also adds some track to tell you whether or not the track was playable, that is using the around method. So, you can see we have all three methods before, after, and around in this class, so you can see how those work. Then you need one more, which is the class that mixes it all together, So, that's the logging-player-mpv, and it extends both the logging-player class and the emms-player-mpv class. What's really interesting about this is that even though the logging player is part of the emms-player hierarchy, it doesn't depend on a specific implementation, so you can combine the two different classes that lets the logging class only care about logging, and the emms-player class only care about playing, and that is a really nice way of separating your concerns that I think is very powerful. Here's a quick example of just how that works, and you can see the unplayable track is not playable, and it gets logged as such, foo is playable, and you can see the logs in started and stopped. So, you can see it's having the side effects from those methods, and it's also returning the value off the player as well. I think this system has a bunch of really nice properties. First and foremost, it feels like a normal Lisp, all you're doing is calling functions, there's no magic involved. Also, you can use either or both of the classes or generic functions. If you only need to encapsulate data into a structure, then you can only use classes, you don't have to use generic functions. And if you only need dynamic dispatch, you can only implement a generic function and methods. You don't get forced into a model where you have to use both. You can mix and match for whatever needs your program has, which I think is really amazing. Any value can conform to an interface, meaning a generic function. So, you don't have to use those classes. You can even implement a generic function over nil, which gives you those really nice properties of Lisp, where you have nil-punning, you know, if you take the head of a nil list, then the output is nil, but you can do that with your object system too. And a really nice feature of that is that you have no possibility of null pointer exceptions like you do in other languages. They have a calling convention where you call object dot method, but in CLOS, you call a generic function, and you just give it some arguments. Typically, the first one is going to be an EIEIO class object, but it doesn't have to be, but because you're not calling that instance of an object, there's nothing to be nil in the first place, so there's no possibility of a nil pointer exception. And then the ability to have multiple inheritance and mix in all of these different functionalities into your final object is very powerful. Because you have multiple inheritance, your final object that composes all of those things is both a player and a logger, and it can be substituted into code that expects either of them. It's not an adapter class that only allows you to do one thing, it does both of them. I think that's amazing. So, here are some practical examples where I think maybe this could be a good idea. And I like to think about, "What is OOP actually good for?". I think it has three really amazing powers, encapsulation, abstraction, and extensibility, so let's look at those. Encapsulation is just keeping related data together. Here's an example from the transmission Torrent client. In order for it to work, it needs to have all four of these variables set consistently, but if you use the customization interface, they're kind of strewn all over the buffer because that shows them alphabetically by variable name instead of grouping them logically by function. You also have all these in the global namespace, so you need this disambiguation in front, you have to have a transmission prefix, so it's kind of ugly. An alternative example would be to encapsulate all of that in a single class. This has one slot for each of those values, except the username and password is broken out, there was only one in the previous example, but it works pretty much the same. The really neat thing about this is that the customization interface understands how to customize EIEIO objects, so you can set your custom variable to the value of an object, and in the customization interface, it shows you all of the fields, and it lets you edit the values of those slots directly. So, that keeps that logical grouping that I think makes things really easy to use. Another thing it's really good at is abstraction, and this is really core to a lot of what Emacs does because it runs on so many different systems, and it works with so many different kinds of similar tools. Here's an example from the built-in SQL implementation. This is the definition of the postgres backend, there's one of these for every supported database backend. And you can see, this is a pretty classic interface abstraction pattern. On the left-hand side, you have a symbol that's common among all the database backends, and the code that doesn't know about the implementation can use, no matter what implementation is being specified. On the right-hand side, you have the implementation specific values, in some cases these are just strings, or regexes, and in others those are actual functions. So, really this already is object orientation, it's just an ad-hoc system for it, but you don't have to use an ad-hoc system because there's this nice formal one instead. Another thing that it's really good at is extensibility, we saw some of that with the emms-player example earlier. But here's another thing I think it would be interesting to explore. Emacs has this idea of derived modes where you have a mode that's based on another, and that's a pretty clear inheritance pattern straight out of OOP as far as I'm concerned. What would it look like if major modes were EIEIO classes, and you could extend your mode by extending the class. I think that's a really interesting idea, and I'd like to explore that more. In conclusion, I think EIEIO is amazing, and I had no idea that such a powerful object orientation system was available in the Emacs. My goal with this talk is for anyone who writes Emacs Lisp, to look at these classes of problems, encapsulation, abstraction, and extensibility, and if you run into those problems in your code, instead of immediately reaching and building your own system, I want you to think: "Oh there's a thing for that, and I can just use it." That's my talk, thanks. Hack on! captions by bhavin192 (Bhavin Gandhi)