Back to the schedule
Previous: Budgeting, Project Monitoring and Invoicing with Org Mode
Next: Emacs with Nyxt: extend your editor with the power of a Lisp browser

Productivity Dashboards with Emacs and Kindle

Mehmet Tekman

Q&A: live Q&A and Matrix Chat (@mtekman:matrix.org)
Duration: 8:31

If you have questions and the speaker has not indicated public contact information on this page, please feel free to e-mail us at emacsconf-submit@gnu.org and we'll forward your question to the speaker.

Talk

Q&A

Description

Name pronunciation:	Meh-met Teck-man
Pronouns:	He/him
Preferred contact info:	@mtekman:matrix.org
Support:	https://liberapay.com/mtekman/donate

https://gitlab.com/mtekman/kindle-sync

Since 2008, Amazon have released a new Kindle device every year, supplanting each generation with a newer model that boasts highly promoted incremental features which greatly devalues the price of their older models. These forgotten models are sold on Ebay and other secondhand websites at highly discount prices by owners who do not see the true potential of these devices: Kindles are excellent high contrast low-refresh display rate E-Ink devices, with Wifi capability, that run embedded Linux in the background. Depending on the model, an idle Kindle can last weeks before needing a recharge. This makes them ideal as passive image devices that can be configured easily using a few shell scripts. Indeed, efforts have been made in dedicated hacker forums to expose the Linux filesystem and to enable features such as custom screensavers, SSH networking, and more. By exploiting these features, and by carefully disabling the software/bloatware that comes with the device, these Kindles have found new life as online dashboard devices which can fetch and display information from the internet at timely intervals.

Here we describe a tool to control multiple Kindle devices with a single org-mode/shell-based tool, built initially to periodically serve updated Emacs Org-Agenda views, but later expanded to produce online local weather reports and work calendar, Emacs calendars (calfw, org-gcal), daily dietary information (org-calories), Org-Mode sparse TODO trees, miscellaneous image and text content (via imagemagick), small messages, and much more.

In this talk, we show how to configure multiple Kindles with any desired custom content, following any daily/weekly schedule, all easily managed from Emacs within a single Org-Mode file.

Discussion

• Q1: I know almost nothing about this stuff so please forgive my ignorance (Actually, I did I dip a toe into some basic jail breakage a few years ago and was delighted/intimidated to find a capable community on mobilereads, as you mentioned; I was living cheaply at the time, and having KUAL and KO and kterm around improved my QOL considerably.) As for your talk, I enjoyed it very much.  I was wondering if you'd given any thought to possible real-world applications for your dashboards. Just spitballing a bit, a few possibilities came to mind, like perhaps disseminating information on a work floor or lab setting where cable runs or temperature increases from LCD monitors might be unwelcome; or perhaps doing so in more a public setting like a refugee or detention camp where power might be limited and where mounting TVs behind protective glass or restraining them with cables might be bad for morale.  Also, have you thought about putting together and/or selling "kits" so folks with limited time could acquire a basic setup as a turnkey solution (perhaps with some assembly required)? Thanks.
• Hi. Lovely idea to use an ebook reader as dashboard. Are all kindle devices supported or only older ones?
• Mehmet Tekman: I recorded this in two parts: with caffeine, and without
• As soon as i can get my hands on a kindle i will give this a try. Lovely Idea.
• Are the images only pushed or can i request for an update from the kindel itself?
  • Mehmet Tekman: images are usually only pushed, but it's done over ssh so pulling is also possible. the main idea is that the interaction is only Server → Client
• Thanks for you talk I have just finished watching it on youtube.
• I have some old kobo's rather than kindle's but thinking of nipping onto ebay to get some
  • Mehmet Tekman: I think it would work well Kobo's, since that's likely also a linux system right?
  • yes they are linux
  • Mehmet Tekman: There are only a few kindle-specific commands, but you can comment them out and adapt them for the Kobo
  • there was some work getting kde onto them
  • Mehmet Tekman: Woah you have access to an X11 session?
  • i think the developers helped, but we are talking 7 years ago for the one they helped with
    • Mehmet Tekman: it's not mainline then?
  • I don't think so I still use them both for reading books so not messed with them just in case they break
    • Mehmet Tekman: That's the beauty of the kindle, it's from such a horrible company and it's so cheap that you have no qualms if you break it :P
  • Mehmet Tekman: The kindle basically locks you out of X, which is frustrating since the Kindle Touch runs AwesomeWM. If I had money, I would definitely buy one :O
• The use concept is really useful, so thanks for pointing me in the right direction.
• as someone who easily gets distracted it will be usefull to check on what I am supposed to be doing lol
  • Mehmet Tekman: Welcome! For a more stripped down version I can really heavily recommend the kindle-dashboard from Pascal Widdershoven
  • Mehmet Tekman: And yep -- I can definitely relate!

Links:

• Main Repo : https://gitlab.com/mtekman/kindle-sync
• Mobile Read Forum (Kindle) : https://www.mobileread.com/forums/showthread.php?t=180113
• Mentioned Repos :
  • https://gitlab.com/mtekman/org-calories.el
  • https://github.com/takaxp/org-tree-slide
  • https://github.com/pascalw/kindle-dash

Outline

• 5-10 minutes:

  1-3 mins Talk about repurposing Kindles:

  • Cheap second-hand wifi device, hackable
  • Low-powered, long battery life, low refresh rate – perfect for a dashboard
    • Timely updated Org-Mode Agendas anyone?
  • Reference to inspired projects (kindle-dashboard)

  2-3 mins Generate content

  • A static text+picture image easily generated with imagemagick wrapper
  • An image of a sparse tree of org-mode TODO file
  • An image of another emacs view (e.g. Calfw, or org-calories)
  • Show post-processing for optimizing image for Kindles

  1-2 mins Configuration in a single org-mode file

  • Defining Machines
  • Defining Commands to generate content
  • Defining Schedules to run Commands on multiple Machines at specific points in the day

  1-2 mins Export and Run:

  • Show exported shell configs and generated cronjobs
  • Witness multiple Kindles producing desired content with wakeup timers

Transcript

Hi everyone! I'm Mehmet Tekman, and I'm here to talk to you about using Amazon Kindles as a productivity dashboard for your various projects. In a nutshell, you describe your machines, your commands, and your schedules in an Org-Mode file, and then you just initialize your Kindle devices. These devices are asleep most of the time, but they wake up at scheduled times to retrieve content from the centralized server. Content can be Org mode and Emacs-based, or it can be from Web content, or it can just be static images and WAV. If you, like me, struggle to keep your life under tabs, or find it very hard to separate your work life from your home life, then you, like me, likely need some kind of passive background service that reminds you of where you are and what you are supposed to be doing. Even if it's just a sign saying, "You're at home! Relax!" An Amazon Kindle is perfect for this. In a nutshell, it's a cheap black and white e-ink device that can go for weeks without needing a single charge. Every year, Amazon brings out an incrementally better model, which makes the old devices obsolete, and you can find these older models for 5 euros on second-hand websites. Plus it runs Linux, has WiFi networking, and has a dedicated forum of hackers for getting the most out of the device. Some drawbacks of this is that the device often comes with unwanted bloat: over-the-air updates, it phones home to Amazon regularly, it has a secret microphone embedded in the device, and it has a bunch of creepy seemingly interdependent background processes, where killing one kind of kills the others risking that you will break the device. But this is where the community really shines through, since the friendly (and not-so-friendly) users (and developers) from the MobileRead forums have pretty much scraped out a good portion of the harmful Amazon scripts from the device. Some of the devices even use Awesome Window Manager, meaning you can really play around with the existing system without having to create your own X11 server. This then empowers users to display whatever they want on the device. One project that really got this going was the Kindle-Dash dashboard from Pascal Widdershoven, who really refined a lot of the internal scripts to stabilize the device. However, the project then puts the onus on the device to retrieve the data from somewhere else over the internet, and so you still need to generate the content and place it on the web somewhere. Plus you need to do this and manage it for every Kindle device that you have. Kindle-Sync, however, is an entirely different beast, albeit one that builds off of the works of the aforementioned projects. It assumes that instead of just having one Kindle device around that you wish to re-purpose for productivity purposes, that you actually have multiple Kindle devices that you want to manage and configure in tandem. Everything is managed from a dedicated server (or a raspberry pi) which distributes jobs to multiple Kindles, running on different update timers. These timers are all managed from the server, and all the Kindle device has to do is: to wake up, power on the WiFi, receive some media, display the media, and receive a barebones RTC sleep request. Then it sleeps for the requested time, consuming no power, whilst displaying the desired media. That is maybe 10 seconds of awake time between each request. Cron does not actually run on the Kindle device itself, simply because it does not reliably work. All of this is handled by the server. With the server-client model, it also tries to restrict Amazon access. SSH keys are shared only from the client to the server, but not from the server to the client, so the Kindle cannot connect to the Raspberry Pi without a password. IPtables rules are also set so that the Kindle cannot phone home to Amazon, and the connections are restricted to just the LAN. So I got very curious at one point and decided to see how long a Kindle could last on a single charge in such an arrangement, so that every 15 minutes for 18 hours, I tested the device by sending a media item and recording the battery level. The Kindle doesn't seem to report the battery level very continuously, but at discrete percentages, so that you could end up with a graph that looks like this. Assuming you have half the charge, and use it once every hour - it will drop by 10% battery in 76 hours, which is roughly three days. It's hard to extrapolate with only three good summarized data points, of which the number of requests per battery level appear to diminish as shown in the table below, but the final result yields 76 requests with an average loss of 0.5% battery life per request. Which is not bad! Assuming you do a request every 2 hours from 8am to 8pm, and let it sleep at night, then that's approximately 6 requests a day, which could easily last a device for a month. The ksync script does essentially everything: from generating and fetching the media, to initializing all Kindle devices, generating the server cronjobs, log report summaries, editing the config tables, and much more. The media operations are comparatively much more complex and encompass a few media use cases such as fetching the weather (though only from Open Weather Maps) and retrieving Google Calendar views by week, month, agenda, and four day view. You can retrieve Org-Mode data from an Emacs instance on the server, which in my case I produce views for an agenda or a sparse tree of my main projects file. Finally we have gallery and wavfile, which are static resources which will never change once generated. The idea is that you feed it text and an image location, and it generates a Kindle-compatible image using imagemagick as a backend for it. In the case of the wavfile, it uses espeak on the backend. The below is summarized from the help-me text in the main ksync file, but essentially, you need to define your config in the CSV files, which we talk about in the next section; initialize all your Kindle devices, i.e. copy over SSH keys, kill all the unnecessary services, and prime them for media collection; and ensure that you have all your static media generated and fetchable; and finally you then refresh the scheduling on the server. Okay, so this is all good and well, and we now know what the server does and how to probe and inspect it - but how does the server generate much of the content? So a lot of the content will be dynamically generated, meaning it cannot be cached and is likely to change from hour to hour. The media content that is generated here are mostly PNG images and have a timestamp embedded in their filenames. The Emacs-specific content consists of a few views, namely the org-gcal views, org-agenda, and org-calories -- essentially anything that Emacs can display and that you want to capture into an image. Emacs can't (as far as I know) render graphics in a headless way, so what we do instead is run Emacs in a dummy minimal X11 session via "xvrb-run." From inside, you can take screenshots as you would in a normal desktop environment, but with the benefit that you don't actually need to invoke a desktop or interfere with an existing one. The minimal elisp shown here is all that is required to output your desired image from Emacs and configure it for the Kindle environment. On the web side of things, we don't really need to invoke a dummy X11 session because Chromium can run headless and can be controlled by the node library "puppeteer" to render dynamic content, focus on regions of the webpage, and take snapshots. The static content comprises of two types: images and audio. The content is accessed by a key, in this case Batman, and the content information is given by the "--extra" parameter which describes either or both an image and text. Okay, so now we have content, how do we schedule this content to appear on our desired machines at desired times? Everything is run via cron. So previously we saw that we only needed the tables MACHINES.csv, COMMANDS.csv, and multiple TIME_*.csv tables for the shell script to work. But Org-Mode does this far easier, since you can just have everything in the same file, and with the helper minor-mode, manage everything from a single Org-Mode document. Here I have 4 kindles and their shortnames. Yes, I even have a Kindle hanging outside my door. I have 11 defined commands which represent the views I want to see, and there are 4 timetables I use, but you can have everything on one, if you like. Rows are machine names, and columns are corresponding hours at which they run. Trust me, it's easier to configure repeating tasks just by repeating them multiple times, because at least this way, it's human readable, and the script which converts these to a cronjob collapses the repeating tasks by itself. The ksync script can be called from within the config.org file using this convenient use-package declaration. All that one needs to do is to configure the ENVIRONMENT_VARIABLES by setting them in this table where you set the repo name, the config directory, where the media shall go, and the server IP, although this can be automatically detected. The package allows you to export your tables by running C-c C-c on them, and allows you to update all the jobs related to each of your clients. You can also initialize clients using this package -- for either all of them or individual clients -- and the package comes with some convenience functions to do this automatically for all tables in the buffer. With this, I want to say a big thank you to Takaaki Ishikawa for his fantastic "org-tree-slide" presentation package. To Pascal Widdershoven and David Hamp-Gonsalves, for their fantastic kindle-dash repositories, for which some of my internal Kindle scripts are derived from. Also a big thanks to the friendly and not-so-friendly users and hackers in the MobileRead forums. And finally, a big thanks to the Emacs community and the conference organizers. Thank you! captions by Mehmet

Back to the schedule
Previous: Budgeting, Project Monitoring and Invoicing with Org Mode
Next: Emacs with Nyxt: extend your editor with the power of a Lisp browser