Back to the schedule
Previous: Emacs development update
Next: Incremental Parsing with emacs-tree-sitter
Powering-up Special Blocks
Musa Al-hassy
Download compressed .webm video (58.5M)
Download compressed .webm video (29.2M, highly compressed)
View transcript
Users will generally only make use of a few predefined `special blocks', such as `example, centre, quote', and will not bother with the effort required to make new ones. When new encapsulating notions are required, users will either fallback on HTML or LaTeX specific solutions, usually littered with `#+ATTR' clauses to pass around configurations or parameters.
Efforts have been exerted to mitigate the trouble of producing new special blocks. However, the issue of passing parameters is still handled in a clumsy fashion; e.g., by having parameters be expressed in a special block's content using specific keywords.
We present a novel approach to making special blocks in a familiar fashion and their use also in a familiar fashion. We achieve the former by presenting ``defblock'', an anaphoric macro exceedingly similar to ``defun'', and for the latter we mimic the usual ``src''-block syntax for argument passing to support special blocks.
For instance, here is a sample declaration.
(defblock stutter () (reps 2)
"Output the CONTENTS of the block REPS many times"
(org-parse (s-repeat reps contents)))
Here is an invocation that passes an optional argument; which defaults to 2 when not given.
Emacs for the win ⌣̈
Upon export, to HTML or LaTeX for instance, the contents of this block are repeated (`stuttered') 5 times. The use of ``src''-like invocation may lead to a decrease in `#+ATTR' clauses.
In the presentation, we aim to show a few `practical' special blocks that users may want: A block that …
- translates some selected text —useful for multilingual blogs
- hides some selected text —useful for learning, quizzes
- folds/boxes text —useful in blogs for folding away details
In particular, all of these examples will be around ~5 lines long!
We also have a larger collection of more useful block types, already implemented.
The notable features of the system are as follows.
- Familiar ``defun'' syntax for making block —``defblock''
- Familiar ``src'' syntax for passing arguments —e.g., ``:key value''
- Fine-grained control over export translation phases —c.f., ``org-parse'' above
- Modular: New blocks can be made out of existing blocks really quickly using ``blockcall'' —similar to Lisp's ``funcall''. We will show how to fuse two blocks to make a new one, also within ~5 lines.
It is hoped that the ease of creating custom special blocks will be a gateway for many Emacs users to start using Lisp.
Resources
https://alhassy.github.io/org-special-block-extras/emacs-conf-2020
- Actual start and end time (EST): Start: 2020-11-29T09.19.39; Q&A: 2020-11-29T09.36.14; End: 2020-11-29T09.48.34
Questions
Should packages implement the interface to one specific format, or attempt to be conclusive to all the potential output targets?
How to share "recipes"? Will this become a "large" project, or minimal that requires you to write most customizations yourself?
Could you make slides that show the source form on the left and the output on the right? That would make understanding each capability much simpler.
Does typing in a block mess up the syntax highlighting? Usually themes use a single color inside an example block, for example.
"You found my crutch!". Colors in source code blocks within blocks are
hard. Didn't have time yet to implement it. Any help is appreciated!
- That's where you can get help from org-mode core developers
If you export to LaTeX->PDF does that work well with beamer as well? To create slides with columns for example?
You have to format the LaTeX appropriately for the backend "beamer".
How does this relate to pandoc, which is used for converting between markup formats?
Side question about org-reveal: How do you get bespoke/multiple-column layouts without using #+HTML (and ) everywhere in the Org file?
It's a custom #begin_parallel block! See the main article linked
below.
Parallel section:
https://alhassy.github.io/org-special-block-extras/#Parallel
What is used to produce colorful boxes around the cursor in your browser?
Commercial software called ScreenBrush.
Why did you put optional arguments in a separate list rather than using cl-style argument lists? e.g. (defblock feedback (who &optional (color "red")) …)
The first argument may take some meta-information when you define it,
which is easier to handle with two arguments.
Do you intend to try to upstream this amazing work into Org?
No prior experience on how to upstream; suggestions and help
appreciated.
- https://orgmode.org/contribute.html.
- Yes, I would suggest simply posting a short proposal for an
org-defblock macro on the orgmode mailing list, and hopefully
Bastien and other maintainers like Nicolas will discuss it with you.
I think they would be excited to have this feature standardized in
Org. +1+1+1+1 I am excited+1+1
Add a little beginner-focused documentation and this becomes another
great reason to use Org over Markdown, I imagine the maintainers would
love to have it.
Notes
- Main article: https://alhassy.github.io/org-special-block-extras/
(HTML or 48 page PDF).
- Slides for this talk:
https://alhassy.github.io/org-special-block-extras/emacs-conf-2020.
- Reddit
discussion.
- Elisp Reference Sheet:
https://alhassy.github.io/ElispCheatSheet/CheatSheet.pdf.
- My Emacs init:
https://github.com/alhassy/emacs.d#a-life-configuring-emacs.
Transcript
All right, then. Well, hello everyone.
I hope you're all enjoying the EmacsConf.
My name is Musa Al-hassy, and I hope you're
excited to learn about powering up special blocks.
[00:00:19.840] Let's first off find out what these special blocks are,
and see what we can go from.
Yesterday, I saw a lot of cool talks
and people were chatting about
how should you present? Should you do it this way or that way?
I thought maybe I should try a different way.
But I'm talking about special blocks
and if I show you an Emacs, then I have to export
the HTML so you can see what it looks like
or export to a PDF so you can see what it looks like.
So I ended up writing in org-reveal,
and joyously, this just works.
You can just see things here.
I was worried that I'd have to take pictures
and then insert pings, so that was a delight.
[00:01:09.760] Okay. Special blocks are these things like a center small quote.
That's what a special block is,
and with a bit of Lisp, we can make
special blocks and link types.
Right. Using a single interface.
The interface is going to be similar
to one many people are familiar with.
In particular, Org Babel's src interface
as well as using global header arguments for link types.
The idea is to write it once
and generate many different kinds.
You write in Org markup and you can have HTML,
you can have PDF, and joyously, org-reveal.
That was an unexpected delight.
[00:01:53.600] Here are a few that you'll just see
in this presentation.
I won't show some of these link-only ones,
but we'll see a few of these other ones
just to make the presentation look nice
So the presentation is really going to
present these blocks and the mechanism
at the same time.
No HTML was written.
Look, Ma! No HTML, just pure Org Mode,
and you get all these beautiful boxes and things.
[00:02:25.840] The motivation for this is...
you're online, you run into a blog,
and you see something you like,
and you're like, man,
you know, I wish I could produce that.
But you check, and the author wrote raw HTML.
You know, #+HTML: everywhere.
That's going to obscure your real content.
It's going to be surrounded by all this
styling information. That's unfortunate.
The author decides to use an Org macro.
All right, a bit better,
but then what if you decide,
hey I want to make a PDF? Not great.
And then the worst of all,
the author doesn't give you the source,
and then you have to view page source,
and learn cascading style sheets,
and sit in a corner and cry,
and decide to do other things with your life.
[00:03:18.080] We want to give you Org users numerous styles
and an extensible mechanism to add more of these
aesthetically pleasing styles,
to have really nice things
look one way in the HTML
and look almost the same way in the PDF
and other back ends.
And if by having these newer ones,
people might be encouraged to try making new ones,
especially when the interface is not so difficult,
that's the aim.
[00:03:54.159] So let's have a real story to motivate this even more.
Here's three friends. I hope I don't butcher their names,
but these friends are called
Amin, Sacha, and Corwin. They're organizing a conference,
EmacsConf 2020.
So Sacha decides to write an Org file
and she would like some feedback.
Okay. Just to make it clear, there's no...
just how easy this looks,
let's look at the source for this block.
Notice it's just the word "green," then a colon,
then Amin. No div style coloring,
just green:Amin.
A very pleasant Org markup.
So that's quite nice. Put some bold around it.
Not too difficult.
Hopefully, this will be useful to other people as well.
So what kind of feedback
would Sacha expect to get?
Maybe she would expect top-level remarks
visible in the export.
When she makes an HTML, she can see right there a big block.
Right. Maybe Amin will suggest to Sacha,
please replace this part
with this other part
or replace this word
with this other word.
This is not really possible
with raw HTML or with even LaTeX.
You'd have to have multiple arguments:
the first argument, and then
the replacement argument. It's a bit clunky.
[00:05:28.800] But with our setup, you just write some text,
write #+replace_with
and then write more text, and you're good to go.
Normal Org markup.
Everyone speaks different languages.
Maybe they want to use one word,
or they're arguing about
whether we talk about frames or windows,
so maybe they want to have some translations.
[00:05:50.560] So there are different kinds of feedback.
Let's take an example.
Look at what they are.
For example, Sacha might write
this Org Mode right here,
and then in her HTML exports, you might see this,
and her feedback might look really nicely
from anyone who says
let's do some Lisp instead of mathematics.
Let's just do some Lisp.
Corwin says, let's not be so silly.
Let's just say 9 a.m. and move on.
Amin likes to export to PDF,
and so he writes his top-level remarks using LaTeX.
That's how. To get this square
Amin: please change whatever,
he might write like this: #+latex:.
But then Sacha only exports to HTML, for example,
so she doesn't look at the PDF,
and she may not see his top-level feedback
with those nice brackets and and bold.
She might think everything's good.
That can be a bit disastrous.
So maybe Sacha will then
make some of her own feedback.
To produce it, she might write
HTML commands, #+html: to get that.
But then Amin will make a PDF,
and this won't stick out.
So he might think everything's okay,
even though it's not.
Then Corwin actually decides,
"Hey, let me read the exported result
and there's all those feedback from two people
who haven't read anything, because maybe they were in a rush,
and didn't see the top-level feedback.
So they agree. "Hey, let's have a uniform Org interface that exports
to both HTML and PDF.
Make both of us happy."
[00:07:54.080] Okay. So they decide to use Org special blocks.
Right. To set this up,
they need to read a little bit of Lisp,
hooks, advice, macros to get all of this set up,
and then they'll use Org as the main interface.
It's a lot of work, but it's worth it, right? maybe?
But then Corwin's a bit terse.
Corwin maybe doesn't want to write using blocks.
He thinks they're overkill.
Sacha wants HTML, and Amin wants PDF, and Corwin wants org-reveal.
So now they have to reformat all their code.
And then they need to use org link types
to reduce the overkill,
so they can try to avoid duplication
by factoring things out into self-contained functions.
[00:08:46.800] But now, to set up our links,
we'll have to learn a new interface, org setup link.
Learn a little bit about fonts, follow links, export handlers...
It's so much. That's so much. But then,
the friends, they learn a lot.
They learn about defun.
So these words are red.
You get a little explanation.
I think it's a bit too small for anyone to read.
This is Lisp documentation for defun.
advice-add. There's some Lisp documentation.
They learn about destructuring -let.
This is from the dash library.
Here's all that glorious,
glorious documentation with examples.
Sorry. I like that.
[00:09:30.300] They might make an ad-hoc mechanism
to simulate arguments for special blocks,
so something maybe called extract-arguments,
and then, of course, to make new link types,
they have to learn about
org-link-set-parameters and its numerous bits and pieces.
Let's close all these ones down.
Of course they also need to be comfortable
with loops and maps and matching and string functions.
So it's a bit of a pain.
It's probably not worth it.
Maybe I'll just rush things quickly,
or do it ad-hoc...
We have things to do.
[00:10:13.680] But maybe the squad wants to have a modular and unified interface
so everyone's comfortable with defun to define a function
and they say, "It would be nice if we could just define simultaneously
both a block and the link type."
That way, we have a single interface
Org mode, for these things.
It would be nice if it was modular.
If I defined a one kind of block and you defined another,
we could compose them,
then get a nice bigger block, like LEGO.
That would be nice. Building blocks.
This is what we have come up with, called defblock.
It also has a long documentation string containing examples and things.
So that way, it can try to be useful.
Let's look at a solution to these friends' trilemma.
So here's a way to define a block.
It doesn't look that difficult, but this is how they can define a block
for their top-level feedback.
Let's look at the three main parts together.
It's not that difficult, I hope.
Just six lines, and that's including a documentation string,
newlines and things.
[00:11:37.633] So in line 1, we define the block just like you define a function.
We define a block.
The block name is going to be called "feedback."
It has an author, "who."
The author has no default value.
It has a color, and the color has a default value of red.
So just as when you define functions,
you start by define or defblock,
then the name, some mandatory argument, and some optional arguments.
Then the next stage is definition. Documentation.
The people who use this,
which are future you or future me,
might want to know what this is.
So let's get to document this.
For Corwin, who might want to use tooltips...
When Corwin writes feedback in Emacs, they'll see a nice little tooltip,
and the tooltip will have
this documentation string.
That'll be nice.
And then here's the third part.
The last three lines are not so difficult.
If the backend is HTML,
please use this template string.
Otherwise, use the other string.
For each of these string markers,
please put in the color, who wrote it, and then the contents
of the special block or the link type.
So that's pretty neat. Not so difficult.
I thought that was kind of cool,
then noticed it's anaphoric.
This defblock gives you two new names.
It gives you a name called contents,
and it gives you a name called backend.
So even if you're writing a defblock
and you intend it to be used only for links...
Like these colors, for example.
These colors were defined using defblock.
I used them as links right here.
You don't need to worry
where does the text come from in the link.
If I say "red:Bob," is it Bob?
Or if I put a description, is it the description?
So it's whatever is available will
become the value of contents.
If you're really interested
and you want to do some intricate stuff,
defblock also gives you something called raw-contents,
if you really want to touch
the raw contents with all of the Org markups still there.
[00:14:12.639] Let's see how everyone can communicate amongst themselves
using this new interface.
So, Sacha speculates and she... How does
she speculate for her Org HTML?
She might just write. Hey look at that, no HTML, nice.
Amin wants to have some green,
and so he just says, hey here's some color green.
There you go. It looks almost the same.
Notice that the main argument is right here.
defblock took an author,
and here's the author again.
And now the optional argument
uses the org babel source interface
You just say :, then a key, and then the argument. Quite nice.
Corwin doesn't want to use blocks. It's a bit of an overkill.
He can just write a link.
So the main argument is now the label of the link,
and the description of the link is the contents of the feedback.
So that was quite nice.
So it looks like everyone uses the same interface on the left
and can have varying outputs.
I think it looks quite nice,
and I hope you do too.
[00:15:36.639] There's a few more.
Maybe, as you saw in some previous ones,
we had text side beside side,
or we folded some regions away.
We put some things in pretty boxes.
We had some spoilers at the very beginning that we hid some text.
We demoed some texts. Here's some Org and here's what it looks like,
and most importantly, they compose.
There's a a macro called thread-block.
thread-block call, and it lets you thread the contents
through a number of blocks,
treating them as if they were functions.
So, really, you can think of a block
as a string-valued function.
That's pretty neat, I think.
Thank you for listening.
I hope you've enjoyed this little
happy fun time with the Emacs and friends.
I'll happily answer questions right now.
[00:16:45.360] Someone says: "Why did you put optional arguments
in a separate list
rather than using cl-style argument lists?"
So that's a very good question,
and I will answer that
by showing you a more involved definition of feedback.
Let's look at a more involved one right here.
So, for example, this one is called rremark.
Please let me know if my text is not sufficiently big.
Here is why we have two arguments.
That takes two arguments instead of one
for its argument list.
You have def block,
then you have the name,
then you have the first argument list
and the second argument list.
The first argument list
takes the text right after the begin.
The text right after the begin is the main argument.
And then the remaining key-value pairs are in the second argument list.
Now the reason we have two is because
in order to streamline the interface to account for both special blocks
and Org link types, what we do is we say,
in the first argument list, you can give a name to the first argument,
give it a default value,
and anything else you provide will become part of the link information.
For example, this link,
we decided to make its face angry red.
You might want to give other features to links.
So we're trying to streamline the interface
for both special blocks and Org link types,
and we thought this way was quite nice.
That was the main reason.
[00:18:47.500] Someone asks--
if you have follow-ups, please ask--
Someone asks, "Do you intend to try to
upstream this amazing work into Org?"
Well, I'm glad you like it.
I don't know how to upstream,
but I will look into it,
and any advice or guidance would be much appreciated.
Lisp is awesome. Just as defun is a macro,
defblock is a macro, and then
source blocks are awesome.
Now maybe we can have arguments in special blocks,
and motivate and encourage more people to learn Lisp.
[00:19:28.799] So another person asks,
"What is used to produce colorful
boxes around the cursor?"
I'm not quite sure if you're asking...
Are you talking about my cursor right here,
or are you talking about in the slide?
So this cursor is some application called Streambrush,
that I had to purchase.
Unfortunately, I could not find a a suitable free one.
The blocks... I can demonstrate some Emacs Lisp.
I can open up my Emacs, if people like,
and we can try some things out.
Happy to do that.
You're welcome.
[00:20:10.133] Someone asks a side question about org-reveal: "How do you get
bespoke or multiple-column layouts
without using HTML?"
Excellent question. That's what we do.
That's what this project is about.
So it's not org-reveal,
it's our fancy parallel block.
So we have this thing. You say, #+begin_parallel.
You say how many columns you would like.
Do you want a bar or not?
And then you write some text,
and then you get some text, and according with the bar or not.
That's how we achieve that in our slides.
I'm not quite sure where this was.
Somewhere here, I think.
Let me try to find this for you.
I can't seem to find where the parallel blocks were. Apologies.
Let's move on to the next question, I suppose.
I'm pretty sure they're here. Ah, there they are.
So these were just instances of using
the parallel block, and it makes things parallel.
So that's quite nice.
[00:21:27.633] Another person asks,
"How does this relate to pandoc,
which is used for converting between markup formats?"
So all we're doing is we're saying,
hey, please write Org because Org is just fantastic,
and we love it, and it's the dream,
and if you would like to view things
in HTML, or in org-reveal, or in PDF,
that's up to the user.
Made it too small now. So here is an example.
Here's how parallel is implemented,
just as a quick example, not too long.
About half of the implementation is documentation,
so, hopefully, that speaks for for how useful this feature is.
We decide if there's a rule or not.
We look for the column break.
Here we're looking at the backend.
If the backend is LaTeX,
please use this incantation
with multicolumns, minipages, what have you.
If the backend is something else, please do this:
div, style and other gibberish that we don't really want to look at.
Pandoc works from Org,
so it might not work directly,
since our interface... The way we set it up is: when you try to export,
we hook in and we do a bunch of pre-processing,
so this defblock is a string-valued function.
Whenever we see these #+begin_parallel when you do an export,
I tell Emacs, hold up, look for those #+begin_parallels, please.
Oh, you found them? Grab that text.
You grabbed it. Great.
Now please apply this person's function
onto that text, and splice in the result.
So when you export, we're performing
arbitrary computations on your text.
Some people might not find that comforting,
to have arbitrary computations happening.
In this article, there's a few where
we change your text upon export.
We translate it, we do other things to it.
[00:23:51.760] So someone says, "If you export to LaTeX, to PDF,
does that work well with Beamer as well
to create slides with columns?"
I made a bunch of these changes
earlier this morning,
and it just says LaTeX right here.
So if you want to go to beamer,
I think the back end for me,
beamer is called, well, beamer,
so instead of a pcase, what we would do is, we would say,
if it's a 'latex or it's a 'beamer, then use this.
Otherwise, it's not a LaTeX,
it will simply default to this one,
which could be dangerous for your needs.
I think it's a bad practice to put a underscore,
but I did it really quickly
because I just wanted to show you that it works fine in org-reveal
Contributions are more than welcome.
I happily would love any assistance.
We have a Lisp reference cheat sheet here
to learn a little bit about Lisp, if you're not comfortable,
or to ask some questions.
Lots of helpful people.
[00:25:06.400] So there's another question that says,
"Does typing in a block mess up with syntax highlighting?
Usually, you use a single color inside an
example block, for example.
Ah, you found my crutch.
Emacs is all encompassing,
and I'm not quite sure how fonts work.
I learned enough to get by.
Here's how links work.
They're a bit complicated.
This is a bit scary.
I don't recommend anyone read it.
Actually, let me open up an email
and you can see what I see.
So here's an Emacs.
Let's make that a bit bigger.
Let's change this slightly.
Nope, that's worse. There you go.
Here's some words. Here's red hello.
But you're worried about preserving fontification.
Let's make an emacs-lisp block.
Let's say, (+ 1 2).
Ah, where's the fun? Hello. Bye.
Okay. Where's the coloring?
If we zoom in on this #+begin_src block,
you can see down here
we have our our coloring
when we zoom in.
If we zoom out, no coloring.
Zoom in, coloring. Zoom out, aah, no coloring.
Let's take off these bad boys, and oh, look, my coloring's back.
In a previous iteration of the system, I was able to maintain coloring.
In this new iteration, I am not.
I don't know how to do it.
I haven't had the time to implement it.
I spent a lot of time writing this 48-page documentation
with some fun examples to try to help people learn.
But I would appreciate any help or guidance
on how to maintain the fontification.
I really would like to keep those colors in.
[00:27:29.200] Amin: Musa, we have time for maybe one more question,
one or two more questions,
and then we have to move on to the next talk.
You're more than welcome to
continue taking the questions via IRC or the pad.
[00:27:42.559] Musa: Okay. Thank you.
The final question we'll take is,
"Should packages implement
interface to one specific format,
or attempt to be inclusive
to all the potential output targets?"
I think you should just make them as you go,
and add them as you need them.
We'll make Github requests for things.
We can share recipes in this document,
and then try to add other techniques,
and then we can use these blocks as a common interface
for exporting to PDF and other things.
Since someone asked,
here what a PDF looks like.
This is the same PDF rendered.
I made no effort to make it look good,
but it surprisingly does look good.
That was nice.
That was a terrible magenta, but that is life.
[00:28:44.320] Anyhow, I hope you all enjoyed this talk.
I hope you will find defblock useful to you.
It is available on MELPA.
In a rush to make it available for EmacsConf 2020,
some MELPA guidelines may not have been adhered to.
Please do not hit me.
I hope everyone enjoys the rest of the EmacsConf 2020. Thank you!
Sunday, Nov 29 2020, ~ 9:33 AM - 9:53 AM EST
Sunday, Nov 29 2020, ~ 6:33 AM - 6:53 AM PST
Sunday, Nov 29 2020, ~ 2:33 PM - 2:53 PM UTC
Sunday, Nov 29 2020, ~ 3:33 PM - 3:53 PM CET
Sunday, Nov 29 2020, ~10:33 PM - 10:53 PM +08
Back to the schedule
Previous: Emacs development update
Next: Incremental Parsing with emacs-tree-sitter
It's a custom #begin_parallel block! See the main article linked
below.
Parallel section: https://alhassy.github.io/org-special-block-extras/#Parallel
What is used to produce colorful boxes around the cursor in your browser?
Commercial software called ScreenBrush.
Why did you put optional arguments in a separate list rather than using cl-style argument lists? e.g. (defblock feedback (who &optional (color "red")) …)
The first argument may take some meta-information when you define it, which is easier to handle with two arguments.
Do you intend to try to upstream this amazing work into Org?
No prior experience on how to upstream; suggestions and help appreciated.
- https://orgmode.org/contribute.html.
- Yes, I would suggest simply posting a short proposal for an org-defblock macro on the orgmode mailing list, and hopefully Bastien and other maintainers like Nicolas will discuss it with you. I think they would be excited to have this feature standardized in Org. +1+1+1+1 I am excited+1+1
Add a little beginner-focused documentation and this becomes another great reason to use Org over Markdown, I imagine the maintainers would love to have it.
Notes
- Main article: https://alhassy.github.io/org-special-block-extras/ (HTML or 48 page PDF).
- Slides for this talk: https://alhassy.github.io/org-special-block-extras/emacs-conf-2020.
- Reddit discussion.
- Elisp Reference Sheet: https://alhassy.github.io/ElispCheatSheet/CheatSheet.pdf.
- My Emacs init: https://github.com/alhassy/emacs.d#a-life-configuring-emacs.
Transcript
All right, then. Well, hello everyone. I hope you're all enjoying the EmacsConf. My name is Musa Al-hassy, and I hope you're excited to learn about powering up special blocks.
[00:00:19.840] Let's first off find out what these special blocks are, and see what we can go from. Yesterday, I saw a lot of cool talks and people were chatting about how should you present? Should you do it this way or that way? I thought maybe I should try a different way. But I'm talking about special blocks and if I show you an Emacs, then I have to export the HTML so you can see what it looks like or export to a PDF so you can see what it looks like. So I ended up writing in org-reveal, and joyously, this just works. You can just see things here. I was worried that I'd have to take pictures and then insert pings, so that was a delight.
[00:01:09.760] Okay. Special blocks are these things like a center small quote. That's what a special block is, and with a bit of Lisp, we can make special blocks and link types. Right. Using a single interface. The interface is going to be similar to one many people are familiar with. In particular, Org Babel's src interface as well as using global header arguments for link types. The idea is to write it once and generate many different kinds. You write in Org markup and you can have HTML, you can have PDF, and joyously, org-reveal. That was an unexpected delight.
[00:01:53.600] Here are a few that you'll just see in this presentation. I won't show some of these link-only ones, but we'll see a few of these other ones just to make the presentation look nice So the presentation is really going to present these blocks and the mechanism at the same time. No HTML was written. Look, Ma! No HTML, just pure Org Mode, and you get all these beautiful boxes and things.
[00:02:25.840] The motivation for this is... you're online, you run into a blog, and you see something you like, and you're like, man, you know, I wish I could produce that. But you check, and the author wrote raw HTML. You know, #+HTML: everywhere. That's going to obscure your real content. It's going to be surrounded by all this styling information. That's unfortunate. The author decides to use an Org macro. All right, a bit better, but then what if you decide, hey I want to make a PDF? Not great. And then the worst of all, the author doesn't give you the source, and then you have to view page source, and learn cascading style sheets, and sit in a corner and cry, and decide to do other things with your life.
[00:03:18.080] We want to give you Org users numerous styles and an extensible mechanism to add more of these aesthetically pleasing styles, to have really nice things look one way in the HTML and look almost the same way in the PDF and other back ends. And if by having these newer ones, people might be encouraged to try making new ones, especially when the interface is not so difficult, that's the aim.
[00:03:54.159] So let's have a real story to motivate this even more. Here's three friends. I hope I don't butcher their names, but these friends are called Amin, Sacha, and Corwin. They're organizing a conference, EmacsConf 2020. So Sacha decides to write an Org file and she would like some feedback. Okay. Just to make it clear, there's no... just how easy this looks, let's look at the source for this block. Notice it's just the word "green," then a colon, then Amin. No div style coloring, just green:Amin. A very pleasant Org markup. So that's quite nice. Put some bold around it. Not too difficult. Hopefully, this will be useful to other people as well. So what kind of feedback would Sacha expect to get? Maybe she would expect top-level remarks visible in the export. When she makes an HTML, she can see right there a big block. Right. Maybe Amin will suggest to Sacha, please replace this part with this other part or replace this word with this other word. This is not really possible with raw HTML or with even LaTeX. You'd have to have multiple arguments: the first argument, and then the replacement argument. It's a bit clunky.
[00:05:28.800] But with our setup, you just write some text, write #+replace_with and then write more text, and you're good to go. Normal Org markup. Everyone speaks different languages. Maybe they want to use one word, or they're arguing about whether we talk about frames or windows, so maybe they want to have some translations.
[00:05:50.560] So there are different kinds of feedback. Let's take an example. Look at what they are. For example, Sacha might write this Org Mode right here, and then in her HTML exports, you might see this, and her feedback might look really nicely from anyone who says let's do some Lisp instead of mathematics. Let's just do some Lisp. Corwin says, let's not be so silly. Let's just say 9 a.m. and move on. Amin likes to export to PDF, and so he writes his top-level remarks using LaTeX. That's how. To get this square Amin: please change whatever, he might write like this: #+latex:. But then Sacha only exports to HTML, for example, so she doesn't look at the PDF, and she may not see his top-level feedback with those nice brackets and and bold. She might think everything's good. That can be a bit disastrous. So maybe Sacha will then make some of her own feedback. To produce it, she might write HTML commands, #+html: to get that. But then Amin will make a PDF, and this won't stick out. So he might think everything's okay, even though it's not. Then Corwin actually decides, "Hey, let me read the exported result and there's all those feedback from two people who haven't read anything, because maybe they were in a rush, and didn't see the top-level feedback. So they agree. "Hey, let's have a uniform Org interface that exports to both HTML and PDF. Make both of us happy."
[00:07:54.080] Okay. So they decide to use Org special blocks. Right. To set this up, they need to read a little bit of Lisp, hooks, advice, macros to get all of this set up, and then they'll use Org as the main interface. It's a lot of work, but it's worth it, right? maybe? But then Corwin's a bit terse. Corwin maybe doesn't want to write using blocks. He thinks they're overkill. Sacha wants HTML, and Amin wants PDF, and Corwin wants org-reveal. So now they have to reformat all their code. And then they need to use org link types to reduce the overkill, so they can try to avoid duplication by factoring things out into self-contained functions.
[00:08:46.800] But now, to set up our links, we'll have to learn a new interface, org setup link. Learn a little bit about fonts, follow links, export handlers... It's so much. That's so much. But then, the friends, they learn a lot. They learn about defun. So these words are red. You get a little explanation. I think it's a bit too small for anyone to read. This is Lisp documentation for defun. advice-add. There's some Lisp documentation. They learn about destructuring -let. This is from the dash library. Here's all that glorious, glorious documentation with examples. Sorry. I like that.
[00:09:30.300] They might make an ad-hoc mechanism to simulate arguments for special blocks, so something maybe called extract-arguments, and then, of course, to make new link types, they have to learn about org-link-set-parameters and its numerous bits and pieces. Let's close all these ones down. Of course they also need to be comfortable with loops and maps and matching and string functions. So it's a bit of a pain. It's probably not worth it. Maybe I'll just rush things quickly, or do it ad-hoc... We have things to do.
[00:10:13.680] But maybe the squad wants to have a modular and unified interface so everyone's comfortable with defun to define a function and they say, "It would be nice if we could just define simultaneously both a block and the link type." That way, we have a single interface Org mode, for these things. It would be nice if it was modular. If I defined a one kind of block and you defined another, we could compose them, then get a nice bigger block, like LEGO. That would be nice. Building blocks. This is what we have come up with, called defblock. It also has a long documentation string containing examples and things. So that way, it can try to be useful. Let's look at a solution to these friends' trilemma. So here's a way to define a block. It doesn't look that difficult, but this is how they can define a block for their top-level feedback. Let's look at the three main parts together. It's not that difficult, I hope. Just six lines, and that's including a documentation string, newlines and things.
[00:11:37.633] So in line 1, we define the block just like you define a function. We define a block. The block name is going to be called "feedback." It has an author, "who." The author has no default value. It has a color, and the color has a default value of red. So just as when you define functions, you start by define or defblock, then the name, some mandatory argument, and some optional arguments. Then the next stage is definition. Documentation. The people who use this, which are future you or future me, might want to know what this is. So let's get to document this. For Corwin, who might want to use tooltips... When Corwin writes feedback in Emacs, they'll see a nice little tooltip, and the tooltip will have this documentation string. That'll be nice. And then here's the third part. The last three lines are not so difficult. If the backend is HTML, please use this template string. Otherwise, use the other string. For each of these string markers, please put in the color, who wrote it, and then the contents of the special block or the link type. So that's pretty neat. Not so difficult. I thought that was kind of cool, then noticed it's anaphoric. This defblock gives you two new names. It gives you a name called contents, and it gives you a name called backend. So even if you're writing a defblock and you intend it to be used only for links... Like these colors, for example. These colors were defined using defblock. I used them as links right here. You don't need to worry where does the text come from in the link. If I say "red:Bob," is it Bob? Or if I put a description, is it the description? So it's whatever is available will become the value of contents. If you're really interested and you want to do some intricate stuff, defblock also gives you something called raw-contents, if you really want to touch the raw contents with all of the Org markups still there.
[00:14:12.639] Let's see how everyone can communicate amongst themselves using this new interface. So, Sacha speculates and she... How does she speculate for her Org HTML? She might just write. Hey look at that, no HTML, nice. Amin wants to have some green, and so he just says, hey here's some color green. There you go. It looks almost the same. Notice that the main argument is right here. defblock took an author, and here's the author again. And now the optional argument uses the org babel source interface You just say :, then a key, and then the argument. Quite nice. Corwin doesn't want to use blocks. It's a bit of an overkill. He can just write a link. So the main argument is now the label of the link, and the description of the link is the contents of the feedback. So that was quite nice. So it looks like everyone uses the same interface on the left and can have varying outputs. I think it looks quite nice, and I hope you do too.
[00:15:36.639] There's a few more. Maybe, as you saw in some previous ones, we had text side beside side, or we folded some regions away. We put some things in pretty boxes. We had some spoilers at the very beginning that we hid some text. We demoed some texts. Here's some Org and here's what it looks like, and most importantly, they compose. There's a a macro called thread-block. thread-block call, and it lets you thread the contents through a number of blocks, treating them as if they were functions. So, really, you can think of a block as a string-valued function. That's pretty neat, I think. Thank you for listening. I hope you've enjoyed this little happy fun time with the Emacs and friends. I'll happily answer questions right now.
[00:16:45.360] Someone says: "Why did you put optional arguments in a separate list rather than using cl-style argument lists?" So that's a very good question, and I will answer that by showing you a more involved definition of feedback. Let's look at a more involved one right here. So, for example, this one is called rremark. Please let me know if my text is not sufficiently big. Here is why we have two arguments. That takes two arguments instead of one for its argument list. You have def block, then you have the name, then you have the first argument list and the second argument list. The first argument list takes the text right after the begin. The text right after the begin is the main argument. And then the remaining key-value pairs are in the second argument list. Now the reason we have two is because in order to streamline the interface to account for both special blocks and Org link types, what we do is we say, in the first argument list, you can give a name to the first argument, give it a default value, and anything else you provide will become part of the link information. For example, this link, we decided to make its face angry red. You might want to give other features to links. So we're trying to streamline the interface for both special blocks and Org link types, and we thought this way was quite nice. That was the main reason.
[00:18:47.500] Someone asks-- if you have follow-ups, please ask-- Someone asks, "Do you intend to try to upstream this amazing work into Org?" Well, I'm glad you like it. I don't know how to upstream, but I will look into it, and any advice or guidance would be much appreciated. Lisp is awesome. Just as defun is a macro, defblock is a macro, and then source blocks are awesome. Now maybe we can have arguments in special blocks, and motivate and encourage more people to learn Lisp.
[00:19:28.799] So another person asks, "What is used to produce colorful boxes around the cursor?" I'm not quite sure if you're asking... Are you talking about my cursor right here, or are you talking about in the slide? So this cursor is some application called Streambrush, that I had to purchase. Unfortunately, I could not find a a suitable free one. The blocks... I can demonstrate some Emacs Lisp. I can open up my Emacs, if people like, and we can try some things out. Happy to do that. You're welcome.
[00:20:10.133] Someone asks a side question about org-reveal: "How do you get bespoke or multiple-column layouts without using HTML?" Excellent question. That's what we do. That's what this project is about. So it's not org-reveal, it's our fancy parallel block. So we have this thing. You say, #+begin_parallel. You say how many columns you would like. Do you want a bar or not? And then you write some text, and then you get some text, and according with the bar or not. That's how we achieve that in our slides. I'm not quite sure where this was. Somewhere here, I think. Let me try to find this for you. I can't seem to find where the parallel blocks were. Apologies. Let's move on to the next question, I suppose. I'm pretty sure they're here. Ah, there they are. So these were just instances of using the parallel block, and it makes things parallel. So that's quite nice.
[00:21:27.633] Another person asks, "How does this relate to pandoc, which is used for converting between markup formats?" So all we're doing is we're saying, hey, please write Org because Org is just fantastic, and we love it, and it's the dream, and if you would like to view things in HTML, or in org-reveal, or in PDF, that's up to the user. Made it too small now. So here is an example. Here's how parallel is implemented, just as a quick example, not too long. About half of the implementation is documentation, so, hopefully, that speaks for for how useful this feature is. We decide if there's a rule or not. We look for the column break. Here we're looking at the backend. If the backend is LaTeX, please use this incantation with multicolumns, minipages, what have you. If the backend is something else, please do this: div, style and other gibberish that we don't really want to look at. Pandoc works from Org, so it might not work directly, since our interface... The way we set it up is: when you try to export, we hook in and we do a bunch of pre-processing, so this defblock is a string-valued function. Whenever we see these #+begin_parallel when you do an export, I tell Emacs, hold up, look for those #+begin_parallels, please. Oh, you found them? Grab that text. You grabbed it. Great. Now please apply this person's function onto that text, and splice in the result. So when you export, we're performing arbitrary computations on your text. Some people might not find that comforting, to have arbitrary computations happening. In this article, there's a few where we change your text upon export. We translate it, we do other things to it.
[00:23:51.760] So someone says, "If you export to LaTeX, to PDF, does that work well with Beamer as well to create slides with columns?" I made a bunch of these changes earlier this morning, and it just says LaTeX right here. So if you want to go to beamer, I think the back end for me, beamer is called, well, beamer, so instead of a pcase, what we would do is, we would say, if it's a 'latex or it's a 'beamer, then use this. Otherwise, it's not a LaTeX, it will simply default to this one, which could be dangerous for your needs. I think it's a bad practice to put a underscore, but I did it really quickly because I just wanted to show you that it works fine in org-reveal Contributions are more than welcome. I happily would love any assistance. We have a Lisp reference cheat sheet here to learn a little bit about Lisp, if you're not comfortable, or to ask some questions. Lots of helpful people.
[00:25:06.400] So there's another question that says, "Does typing in a block mess up with syntax highlighting? Usually, you use a single color inside an example block, for example. Ah, you found my crutch. Emacs is all encompassing, and I'm not quite sure how fonts work. I learned enough to get by. Here's how links work. They're a bit complicated. This is a bit scary. I don't recommend anyone read it. Actually, let me open up an email and you can see what I see. So here's an Emacs. Let's make that a bit bigger. Let's change this slightly. Nope, that's worse. There you go. Here's some words. Here's red hello. But you're worried about preserving fontification. Let's make an emacs-lisp block. Let's say, (+ 1 2). Ah, where's the fun? Hello. Bye. Okay. Where's the coloring? If we zoom in on this #+begin_src block, you can see down here we have our our coloring when we zoom in. If we zoom out, no coloring. Zoom in, coloring. Zoom out, aah, no coloring. Let's take off these bad boys, and oh, look, my coloring's back. In a previous iteration of the system, I was able to maintain coloring. In this new iteration, I am not. I don't know how to do it. I haven't had the time to implement it. I spent a lot of time writing this 48-page documentation with some fun examples to try to help people learn. But I would appreciate any help or guidance on how to maintain the fontification. I really would like to keep those colors in.
[00:27:29.200] Amin: Musa, we have time for maybe one more question, one or two more questions, and then we have to move on to the next talk. You're more than welcome to continue taking the questions via IRC or the pad.
[00:27:42.559] Musa: Okay. Thank you. The final question we'll take is, "Should packages implement interface to one specific format, or attempt to be inclusive to all the potential output targets?" I think you should just make them as you go, and add them as you need them. We'll make Github requests for things. We can share recipes in this document, and then try to add other techniques, and then we can use these blocks as a common interface for exporting to PDF and other things. Since someone asked, here what a PDF looks like. This is the same PDF rendered. I made no effort to make it look good, but it surprisingly does look good. That was nice. That was a terrible magenta, but that is life.
[00:28:44.320] Anyhow, I hope you all enjoyed this talk. I hope you will find defblock useful to you. It is available on MELPA. In a rush to make it available for EmacsConf 2020, some MELPA guidelines may not have been adhered to. Please do not hit me. I hope everyone enjoys the rest of the EmacsConf 2020. Thank you!
Sunday, Nov 29 2020, ~ 9:33 AM - 9:53 AM EST
Sunday, Nov 29 2020, ~ 6:33 AM - 6:53 AM PST
Sunday, Nov 29 2020, ~ 2:33 PM - 2:53 PM UTC
Sunday, Nov 29 2020, ~ 3:33 PM - 3:53 PM CET
Sunday, Nov 29 2020, ~10:33 PM - 10:53 PM +08
Back to the schedule
Previous: Emacs development update
Next: Incremental Parsing with emacs-tree-sitter