Edit live Jupyter notebook cells with Emacs

Blaine Mooers (Blane Moors, he/him, Blaine-Mooers@ouhsc.edu)

In this talk, Blaine Mooers shows how to use GhostText and Atomic Chrome to edit Jupyter notebook cells and other text areas within Emacs. Afterwards, he will handle questions via BigBlueButton.

The following image shows where the talk is in the schedule for Sat 2022-12-03. Solid lines show talks with Q&A via BigBlueButton. Dashed lines show talks with Q&A via IRC or Etherpad.

Format: 18-min talk followed by live Q&A (done)
Etherpad: https://pad.emacsconf.org/2022-jupyter
Discuss on IRC: #emacsconf-gen
Status: TO_INDEX_QA

Talk

00:00.000 Introduction
00:51.520 GhostText and Atomic Chrome
01:26.920 GhostText
03:13.120 Live coding demo
05:57.040 Editing code cells
07:57.160 Python
11:11.040 Julia
11:59.600 How does GhostText work?
12:44.320 Supported web browsers
13:33.000 Atomic Chrome configuration
14:21.560 Precautions
15:21.480 Conclusions
15:48.560 Thanks

Q&A

01:00.520 Do you have a favorite color theme?
01:54.360 To your knowledge, are recent coming security changes in Chrome going to impact browser extensions?
03:07.960 Is this browser-agnostic, or do you have to use Chrome?
03:47.580 You mentioned a couple other solutions to allow emacs editing of text areas. Pointers?
05:49.520 Why not save text from Emacs?
08:03.320 Have you been talking with John Kitchin?
10:06.160 Journey
11:37.560 What was the keybinding for Linux Firefox?
12:44.960 How long have you been using Emacs?

Description

My talk will address a dilemma faced daily by many technical writers and programmers. Like many academics, I use several web-based platforms for writing prose (e.g., 750words, Overleaf) and interactive computing (Jupyter Notebook, Google Colab Notebooks). The first group lacks support for snippets. The second group has support for snippets, but this support does not include tab triggers and tab stops. The absence of tab stops can increase the number of bugs by overlooking parameter values in the snippet that need to be changed to adapt the snippet to the current problem. One solution to the absence of full-powered snippets is to apply Emacs with yasnippets to these web-based platforms.

One route to doing so is to use the atomic-chrome package for Emacs and the GhostText Extension for web browsers. These two software packages enable two-way communication via a web socket between an Emacs buffer and the text area of the web page. Edits made on the web side of the socket are immediately sent to the Emacs buffer and vice versa. The Emacs's snippets and other editing tools only work in the Emacs buffer. The connection can be closed from either side. This route has enabled me to apply snippets of LaTeX code to my daily writing in 750words. I have been able to convert 750words into a platform for writing in LaTeX; by default, it uses markdown. I have also been able to apply code snippets for Julia, Python, R, and so on in Jupyter notebook cells. In other words, I get to extend my time writing in Emacs.

In my ten-minute talk, I will describe my problems with web-based platforms and their solution with Emacs. I will describe where to find the required software and how I configured Emacs. I will present several precautions for using GhostText and describe the limitations of its application. I will provide links to collections of snippets I found handy daily usage of 750words and Jupyter.

Discussion

Notes

• https://github.com/alpha22jp/atomic-chrome atomic chrome is by alpha2jp. It is available on MELPA.
• https://ghosttext.fregante.com/ Central website for GhostText.
• https://www.youtube.com/watch?v=JWW3o104npY&t=190s Example of using GhostText with Moodle to write web papes.
• https://github.com/MooersLab/latex-emacs My configuration targeting LaTeX in Gnu Emacs.
• https://github.com/MooersLab/BerlinEmacsAugust2022 Slides from a 90-minute talk in August at the Berlin Emacs Meetup about using LaTeX in Emacs.
• https://github.com/MooersLab/DSW22ghosttext Slides to 50-minute talk in July about using GhostText with Emacs and other editors. Includes slides for friends that use Vim.
• Great talk, thank you!
• Btw, great talk Blaine!
• I've got ghosttext and emacs via atomicchrome working now: very easy and very useful! Thank you.
• Nice job Blaine!
• Thank you!
• Pretty inspring!

Questions and answers

• Q: Kind of a silly question but I'm curious... Do you have a favourite color theme?
  • A: I like several themes and use different ones in different Emacs profiles to distinguish their windows if two are more are open at once. I use the leuven theme in my default config file. It resembles the ef-light theme in the ef-theme. package by Protesilaos Stavrou. I use the ef-spring theme in my latex-emacs config; it has mint green background color.
• Q: Really interesting to discover GhostText thanks. I use ein (Emacs Ipython Notebooks https://github.com/millejoh/emacs-ipython-notebook) for running notebooks locally, I can clearly see the advantage of using GhostText with non-local notebooks/websites but is there anything that GhostText provides over EIN when interacting with locally running Jupyter Notebooks?
  • A: Good question. I have used the juptyer package in org files, but I do not have much experience with EIN. EIN has a long list of commands specific to editing Jupyter Notebooks. I expect that EIN provides the more powerful approach after making the investment in learning the new keybindings. GhostText was not designed specifically to edit Jupyter cells.
• Q: To your knowledge are recent/coming security changes in Chrome going to impact the browswer exstention?   Thanks !
  • A: I do not know the answer. If the secuirty changes inhibits GhostText, it should continue to work in the FireFox family of browsers and Safari.
• Q:Is this browser-agnostic, or do you have to use Chrome? (answered in talk 15:58)
  • A: Works on different browsers
• Q: You mentioned a couple other solutions to allow emacs editing of text areas.  Pointers?
  • Two options: (1) A: Emacs Everywhere (https://github.com/tecosaur/emacs-everywhere); however, it requires uses emacs-client.
    (2) The above-mentioned atomic-chrome package can use the browser extension for chrome called "atomic-chrome" in place of GhostText. It is called "atomic" because the extension was orginally designed to work with the text editor Atom. However, I cannot find this extension.
• Q: Why not save text from emacs?
  • A: The text in the file will get out of synch with the text in the Emacs buffer and the browser. This can lead to loss of all of the text. Maybe you can set up continuous saving of the text to the file from Emacs.
• Q: What was the key binding for Linux/Firefox?
  • A: Ctrl+Shift+h
• Q: how long have you been in Emacs?
  • A: 18 months ago I made the commitment to adopt GNU Emacs as my main text editor. I had several false starts earlier. I was too impatient to master one of the starter kits.

Transcript

[00:00:00.000] Hi, my name is Blaine Mooers. I'm an associate professor of biochemistry at the University of Oklahoma Health Sciences Center in Oklahoma City. I'm going to talk about the use of Emacs to edit live Jupyter notebook cells as well as text areas on web pages. So like a lot of technical workers, I find myself having to write prose in text areas on web pages, as well as working with code in Jupyter notebooks and Colab notebooks, and often I have wished for the full power of Emacs while doing so. Well, now that is possible. Actually, there are several solutions that have been available for some time. I'm going to talk about one solution that I'm familiar with and has worked out for me.

[00:00:55.320] So this requires the use of two software packages, GhostText and Atomic Chrome. GhostText is an extension for the web browser, whereas Atomic Chrome is a package for Emacs. You have to have both of these. So Chrome is for the editor side and GhostText handles the browser side. The GhostText extension is available in the Chrome web store. And GhostText is represented by this icon,

[00:01:29.520] which has a ghost in front of the capital letter T. It is being developed by Federico Brigante. He is a very prolific JavaScript developer. He has a web page committed to GhostText, as well as a GitHub site. So here's an example of GhostText. This is a snapshot from a session that I had while editing LaTeX on the Overleaf website. Overleaf is this web service that empowers the editing of LaTeX documents on the web. So I have clicked on this GhostText icon in the toolbar. I had already opened up Emacs, and I had the Atomic Chrome server running. So a connection was established, as indicated by this blue border around this text area. And as soon as that appeared, the text appeared in a buffer inside of Emacs. So I have overlaid the area where normally the compiled PDF would appear in an Overleaf session. So I'm using a configuration for LaTeX that I developed, which is available through the MooersLab GitHub site. I also gave a talk about how I use LaTeX in Emacs at the Berlin Emacs meetup in August. This talk was not recorded, but the slides are available on this website. So I would like to now switch to a little live coding to make this a little more interesting. So I start my day at this other website called 750Words.

[00:03:22.440] This site just takes plain text, but I like to write in LaTeX. So GhostText came to my rescue when I started using this everyday last May. So I clicked on the GhostText icon. It highlighted that area in blue. There's some boilerplate that I like to start my day with. I like to get a list of my deadlines that are coming up, as shown here, for the next several months. And then I have landed at this tab stop. And so I had issued a tab trigger which inserted this almost 50 lines of text from a snippet through Yasnippet. And then I'll change this text to whatever. And then I can hit TAB to move to the next site. I was dead tired last night, so I fell asleep at my desk, and whatever. So I just keep on going and then hit TAB again and enter my "To Be Done" items. And then what I love about Emacs is that you can hit C-c C-j to insert a new item and so forth, so you can extend the list. Initially, I just have 10 items. I'm going to have more. And on I go, using the full power of LaTeX. So I have configured Atomic Chrome so it will recognize this website as a – it will open up this website – the connection to this website with this buffer in the LaTeX major mode. To turn this off, we can close – simply just close the buffer and that will shut things down. On the browser side, you can right-click on the icon and disconnect GhostText on this page. Okay, let's go to a different situation. This is not a feature that's advertised by the developer,

[00:05:57.360] but I discovered that you can edit code cells (or any kind of cell for that matter) in a Jupyter Notebook. However, we have a challenge here. We have three text areas open – three code cells. So if we click on the GhostText icon, these three areas will show up in green and we'll be prompted to select the one that we want to activate. We want to activate the one with text. So then we can go in here and make edits, of course, and you can do this in Emacs or we can do it in the browser. It doesn't matter. You saw me editing in Emacs, but we can also make the edits in the text area of the browser and they will show up immediately in Emacs. So we could change the case of that M and that's going to – shows up over here. Okay, we can run this code. So this is R, one of the three major programming languages for data science. At least, Jupyter is supposed to be a combination of Julia, Python, and R. So we're running mcmc to get the posterior distribution and we're going to plot those out with this pyplots package, and we have these beautiful plots showing the median of the posterior distribution for four variables in – four parameters in the CARS data set, which is available – built into the R package. And then these shaded areas are the 80% interval. Okay. Oops. So now for the Python side,

[00:08:07.640] here's an example in which I'm going to actually insert a snippet of that cell and then I'm going to enter nvlig for nglview ligand, and just hit enter. Oops. Hit TAB, excuse me, and we don't need this line of code, so delete that. Yep, we want to load up this pdb file that's in this subdirectory. So the pdb file is a plain text file that contains atomic coordinates of protein crystal structure. This protein happens to be important in cancer and we have – we screened by docking 55,000 compounds on a supercomputer and then we did MD [molecular dynamics] simulations of the top 10 [actually 20] leads. Twelve of them had the compound remain bound during the period of the simulation, so those have some potential for – and require experimental validation. So we'll run this chunk of code, and this will give a view of the molecule that we can interact with by using the mouse. But I want to share this with my colleague. My colleague is not set up to use Jupyter, but instead we can write this out to a HTML file, which I have loaded up already. And so we can actually – perhaps. We click on these two arrows pointing at each other, and we can get a full screen view of this molecule and he can identify each atom in this structure. Over a thousand atoms present. We're just hovering over a specific atom. So shown in gray is the ligand that is bound. Okay. So we still have this box selected and we still have these two different – so for each of the – our selected text areas we have a separate – we have a separate buffer open. Okay. To wrap things up here, here's an example of using – with evolving Julia code. And so this Julia code in this cell is in a Emacs buffer.

[00:11:12.920] So you've got an idea now, I think. So in terms of plain text areas like in Overleaf and then these cells in Jupyter Notebooks, these are other areas that can be edited like in the text areas within Outlook Webmail and Gmail. Instead of having to point with the mouse or click with the mouse, one can also use keybindings or keyboard shortcuts. So here are the ones for three major operating systems. So how does GhostText work? Main thing is you have to open up Emacs and get this Atomic Chrome server running.

[00:11:59.680] And then with it up and going, GhostText will be able to – has to be activated and it will find the GhostText server through the localhost port 4001. Put that into the web browser. If you navigate to that port, you'll get output that looks like this if everything's working well. Otherwise, you'll get a error message and it should have a port socket – a web socket port number. It will not be the same every time. So these are the supported web browsers in addition to Chrome. These are supported

[00:12:45.360] and likewise anything in these – any browser related to these can probably use these extensions. For example, the Brave browser will use Chrome extension and the Firefox browser extension works with WaterFox. These are the supported editors. Each editor has its own extension and this GhostText was initially developed for SublimeText. So if you have SublimeText, then you can use its smooth operation as positive control when things go wrong with Emacs. This is Atomic. This is a GitHub site for Atomic Chrome. Atomic Chrome is available for installation through Melpa.

[00:13:33.160] This is my configuration for Atomic Chrome. So I have this setup so the server starts whenever I log in, and I have it set up so that default major mode is Python to deal with the Jupyter notebooks and Colab notebooks. And then I have major modes for these other websites defined below. This is a testing site so the developer has made to help with troubleshooting. He also has a protocol on his website to follow during troubleshooting. So here are some precautions. You'll find that GhostText doesn't work with Pluto. Pluto is a new computational notebook for working with Julia.

[00:14:23.480] My suggestion would be just to run IJulia in Jupyter. It also doesn't work, of course, with RStudio. Even though RStudio sort of resembles a web page, web browser, it's not. Of course, you can always run R, as you've just seen, using the IPy kernel. I will also caution you that if you use the Emacs server, you may run into issues with the server competing with the port 4001. So instead, you should probably configure the Emacs server to use a specific port. So far – although I haven't done that myself – so far, I haven't found any conflicts with the Org Roam user interface. So my conclusions are: GhostText allows you to edit prose with your favorite major mode

[00:15:24.440] in the text areas of web pages and in the cells of Jupyter notebooks. This allows you to tap into snippets and thereby save time as you have – probably have – hopefully got an idea of. I'd like to thank my friends and mentors who've helped me out during my second year in my Emacs learning spiral.

[00:15:49.360] These include my local colleagues. We meet once a month in the Oklahoma Data Science Workshop. Last July, I gave a presentation about GhostText. And then also my friends at Berlin and Austin Emacs meetups and in the UK research software engineer Emacs research Slack channel. So I don't attend these every month, but I try to make the meetings as often as I can. Then I'm supported by the following grants, which allow me to spend at least some time each day in Emacs. I'll be happy to take any questions.

Captioner: sachac

Questions or comments? Please e-mail Blaine-Mooers@ouhsc.edu

CategoryOrgMode