Back to the schedule
Previous: Yak-shaving to a UI framework
Next: Extending the "model" of Emacs to other applications

Moldable Emacs, a step towards sustainable software

Andrea mailto:andrea-dev@hotmail.com - pronouns: he/him -- https://ag91.github.io

Q&A: IRC or Etherpad
Duration: 9:34

This talk was also streamed at an alternate time for APAC hours: https://libreau.org/past.html#emacsconf21

If you have questions and the speaker has not indicated public contact information on this page, please feel free to e-mail us at emacsconf-submit@gnu.org and we'll forward your question to the speaker.

Description

We could learn about things better. Mountains of knowledge hide in places we cannot access or use. The more we write down, the more it takes to find and understand things we find useful.

Knowledge (web, software, books) keeps growing faster and faster! This is not sustainable: we cannot keep up with it! What if we repeat the error of somebody else, only because it would take too much reading to know? What if that knowledge is in some code we work with everyday?

Moldable development is a paradigm shift that attempts to solve this problem. In a gist, the tool you use should let you create special tools to learn smartly from what you have already.

Since we use Emacs, let's make our great editor moldable!

This talk shows my progress in making Emacs closer to such a tool. We are going to see how we can mold structured (and maybe even natural) text to learn better, how we can inject notes in our projects and how self documenting this tool is!

I aim to inspire you to find a quicker way to learn from our digital world!

You can learn more about this at: https://github.com/ag91/moldable-emacs

Discussion

IRC nick: `andrea

Pad:

• Q1: How to find a balance between «generic» molds that do not provide specific enough info vs writing a new mold for every new query/question?
  • A: You can write molds that are private for your special problem. I created molds for my work that I don't share: like find the stories I am working on and how long time I spent on tasks lately. Also, moldable-emacs is to make these tools easy to write, so you should free to throw away tools when you need them once only. If I believe a tool is a good start for many other tools, I put them among the core molds, else if I use them often I store them as contrib. If it is a one off, I throw it away.
• Q2: How would you evaluate this workflow for package managemnt in large independent codebases. Can one integrate it with code sematic analyzers to make for a better work flow?
  • A: moldable-emacs is about creating custom tools you can apply to your situation. I started experimenting with molding NPM json packages + security data from OWASP to view/display security issues in my packages to my colleagues in the past. moldable-emacs gives me the infrastructure to answer my question about security, and I now started asking myself about architecture coherence, so I have scaled up tree-sitter over projects to check that modules don't use packages from other modules. By that I mean that as long as your code semantic analyzers output data, you can mold that (context) data to tell your story (answer the question you have). Does this answer your question?
  • You answered it very well. I am also a security auditor for multiple development teams. And I am incharge of code analysis in an understaffed security team. So your usecase example got my usecase spot on. 
  • Cool! For now you can define insecure patterns using tree-sitter expressions (for example, I find a variable called "password" in the code set to a string. For the package.json I linked to OWASP API and looped through the packages using tree-sitter tokens. I didn't get there, but I wanted to see an Org Mode buffer with the list of the most vulnerable deps highlighted by color + how to solve them: so I could pass them to developers to resolve them (I am a dev, but sometimes others don't know about security risks).
  • Often molds are to tell stories to others.
  • This is probably the most important thing for my personal usecase. Thank you very much. Now it's my turn to learn it and use it well. 
  • Please open issues or email me, and I will try to help if you like how it works
  • I'll do so.

IRC:

• cool...so essentially you are developing a text based version of Glamorous Toolkit.
  • `andrea: yup, but only because I don't have good imaging in Emacs yet (but with tui.el...)
• your talk helped a lot with that though. I'd been seeing posts from you for a little while, but now I "get it"
  • `andrea: yeah sorry, I am still building my vision: it may look I have been all over the place (image recognition, editing css, parse English lately), but the common thread is the easing of creation of micro tools that help me tell the stories I need
• I love your approach of mining other 'nuggets' from other contexts and bringing them to Emacs. I really look forward to looking in to your work and see if I can implement some of it. Thank you so much for your talk.

Links:

• Telling a story about code using buffer views https://moldabledevelopment.com/

Outline

• 5-10 minutes: quick demo of moldable-emacs

Transcript

Welcome to my talk, Moldable Emacs: A Step Towards Sustainable Software. Who am I? I am Andrea. I work as a Clojure software engineer somewhere in the middle of the UK. I inherited my passion for Emacs from my Ph.D. supervisor, and from that moment on, I got in synergy with it. You can learn more about my interests and my Emacs adventure at ag91.github.io. So let's get in the talk. Why moldable development? There is too much information to read it all. Reading is very difficult. It's a very slow activity. You need to go word by word or paragraph by paragraph, if you speedread. But anyway, you take a lot of time to absorb that information. And we urgently need to stand on the shoulders of giants, so the idea is we should stop doing always the same errors and we should be able to absorb as much of the good ideas that the bright people around us generate. For example, if I create a magnificent program in COBOL, and nobody knows any more how to learn or read COBOL, (and in order to read, you take a lot of time), well, that fantastic idea should be easily translatable to C, or to Clojure, or to Common Lisp, or to a language that will come after. The idea shouldn't be lost in a codebase somewhere in an old mainframe. It should be still accessible. Let's get in practice. What does it mean? It means that, for example, the proponents of moldable development prepare this slide to give a sense. So the idea is... Look at this. What is here? You will see that all these little things look like the same. The first time I looked at it, this was looking like a class diagram. This is actually code describing a little system. If you look and if you read, you can see that there is a numerator, a denominator... So this, you see, is interactive, because it's code. It's something that is running, and it's an object because this is Smalltalk -- Pharo, a dialect of Smalltalk -- but in the next slide, since this is a moldable tool, you can see that you can... there is a representation of the same software in a human way. So, for example, here you can see there is a mathematical formula. The other object, the second one, was a file system kind of thing. The third one was an image. And the last one was sort of a graph. So you can see that there is a better way to learn, to distinguish, to intuitively get a sense. And there is not only a single way. It's custom to what you need. For example, this is a very general way to understand what is this object about and maybe you want to see some other little things. For example, the documentation of the code, because you are interested in developing with it. For example, an image, you can see there's a path on the filesystem, or as a hexadecimal representation. In a sense, there is not only one view. You need to have the view that you need at the moment, and your tool needs to make this easy for you. So, why moldable Emacs? I wanted to bring that idea of having multiple view representations of what you need to understand better in Emacs. And so I want to create immediate story telling. Immediate, because it needs to be very quick, and story telling is because you want to allow connection from something that you needed to develop it into something new. So you are really telling a story: what is this mathematical formula I created because I need this, or this numerator and denominator produce this number. So this is a story that you are telling in my mind. And I want multiple views for buffers. Buffers is the main concept in Emacs, and so buffers are what I want to integrate in a story. I create a buffer and I start manipulating it, creating a view and then another view in order to tell something to myself, in order to learn, but also to tell something to others. So, for example, let's start from a use case: learning better. I had, at work, a list of changes for a pull request, so a code change, and I was very tired. I couldn't understand what this much text was about. So what I generate, I create a value for myself to understand it easily. And for me, understanding it easily, for example, was a little flow diagram. It showed me, okay, there is first this, this, and this, and so I could follow. having it next to the change. Having this image next to the change. And this is describing an Italian recipe for pasta with butter, so if you want to try, you're welcome. It's very tasty. Anyway, the other thing that we can do is query text -- structured text. So for example, this presentation is an Org Mode buffer. So when I call the Playground (that is one of the molds that lets me write some Elisp to query the original buffer,) if I evaluate this, you will see that I have just asked my Org Mode buffer to tell me the content length of the headings with some interesting content. So all the headings at third-level. Do you understand? I've just asked a file to tell me its contents without reading it. Or we can do something similar for code. We can do... I don't know... No idea what is written there, but I want to know which function is the most complex or is overcomplicated. I have defined in red, (so again, I don't need to read the number to know either what it is about!) So, I've written in red, I've shown in red the function with more complexity, and I can jump to it. So everything is very accessible to facilitate my operation and my understanding. Or I can take notes. For example, I can annotate something, and you see the note is again structured text, because you will know that I'm going to query my notes at some point. For example, I can show all my notes, for example, by mode, or I can show all the notes by mode in Org Mode. Because it's structured text, I can manipulate it very easily. So these are all my notes. Finally, the superpower of this moldable Emacs is the fact that you can compose molds. So, for example, let's go in showing all my notes. Let me show you all my notes. And then let's say that I want to know how they are... how many lines are these notes? Look, this is the answer. So of all the notes I take, I can actually query it and say "What are the lengths?" But let me show something more. Which one is the longest note? Now there are lots of notes in there so it's difficult to know but what if I can, in a click, generate a view that is very immediate? Look, there is a note that is very long. It's about 35 lines. Do you understand? I didn't read any note. This is all coming from being able to query your text and having multiple representations. My presentation is very short. What is next? Next is to integrate molds with other software like code-compass. I did a presentation last year and I want to make those nice diagrams available for small molds so that you can use them, for example, for notes or text that you have. To integrate better with Nyxt, the Common Lisp browser, because there's a lot of opportunity there to make funny things, a browser accessible for molding, and then having some interaction with Smalltalk through Glamorous Toolkit, so that we can have the best tools, Emacs and Glamorous Toolkit and Nyxt and others, to work together to make our learning easy. Then... You've seen the tool; my molds that I have shown were basically by buffer. I want project statistics. What about... Give me the complexity of all the functions in a project, of all the paragraphs, whatever. And then there is a nice issue on my issue-tracker for moldable Emacs is about: "Emacs: tell me how can I compose the molds that I have to make new things?" It is a sort of a research-y thing that is pretty cool. So if you want to learn more, just check out at ag91.github.io, check out moldable Emacs on GitHub, and enjoy the rest of the conference. Bye.

Back to the schedule
Previous: Yak-shaving to a UI framework
Next: Extending the "model" of Emacs to other applications