Back to the talks Previous by time: Re-imagining the Emacs user experience with Casual Suite Next by time: Emacs, eev, and Maxima - now! Track: General - Watch

New in hyperdrive.el: quick install, peer graph, transclusion!

Joseph Turner - https://ushin.org xmpp:discuss@conference.ushin.org (XMPP MUC for USHIN discussion), contact@ushin.org

The following image shows where the talk is in the schedule for Sat 2024-12-07. Solid lines show talks with Q&A via BigBlueButton. Dashed lines show talks with Q&A via IRC or Etherpad.

Format: 21-min talk ; Q&A: BigBlueButton conference room https://media.emacsconf.org/2024/current/bbb-hyperdrive.html Etherpad: https://pad.emacsconf.org/2024-hyperdrive
Etherpad: https://pad.emacsconf.org/2024-hyperdrive
Discuss on IRC: #emacsconf-gen
Status: Q&A open for participation

Times in different time zones:
Saturday, Dec 7 2024, ~3:25 PM - 3:45 PM EST (US/Eastern)
which is the same as:
Saturday, Dec 7 2024, ~2:25 PM - 2:45 PM CST (US/Central)
Saturday, Dec 7 2024, ~1:25 PM - 1:45 PM MST (US/Mountain)
Saturday, Dec 7 2024, ~12:25 PM - 12:45 PM PST (US/Pacific)
Saturday, Dec 7 2024, ~8:25 PM - 8:45 PM UTC
Saturday, Dec 7 2024, ~9:25 PM - 9:45 PM CET (Europe/Paris)
Saturday, Dec 7 2024, ~10:25 PM - 10:45 PM EET (Europe/Athens)
Sunday, Dec 8 2024, ~1:55 AM - 2:15 AM IST (Asia/Kolkata)
Sunday, Dec 8 2024, ~4:25 AM - 4:45 AM +08 (Asia/Singapore)
Sunday, Dec 8 2024, ~5:25 AM - 5:45 AM JST (Asia/Tokyo)
Find out how to watch and participate
Duration: 20:25 minutes

Description

hyperdrive.el is an Emacs interface to hyperdrive, a mutable, versioned, peer-to-peer shared filesystem. Among other things, hyperdrive.el has these features: Share unlimited files of unlimited size; Explore file history with built-in versioning; Stream video and audio; No signup or account creation; Free as in Freedom!

Since last year's EmacsConf talk, hyperdrive.el grew some new features, including:

  • Easily install the gateway program with M-x hyperdrive-install.
  • Visualize your network of sources as a graph or a list.
  • Transclude snippets of hyperdrive files with hyperdrive-org-transclusion.

This talk will show off these new hyperdrive.el features in action!

Feel free to join our public XMPP chat room!

Bugs can be submitted to the ushin issue tracker. Patches, comments or questions can be submitted to the ushin public inbox.

About the speaker:

I'm Joseph Turner. I enjoy fiddle, Aikido, peer-to-peer networks, Emacs, and swimming in cold water. I work with USHIN, a tiny educational US nonprofit whose mission is to promote personal, community, and global health through free and open universal shared information for everybody.

See also: EmacsConf - 2023 - talks - hyperdrive.el: Peer-to-peer filesystem in Emacs

Transcript

Hello, welcome to this talk on hyperdrive.el. I'm Joseph Turner presenting on behalf of Ushin, a tiny independent information freedom nonprofit. Hyperdrive.el is an Emacs interface to the Hyperdrive peer-to-peer file sharing system built by HolePunch. Like other peer-to-peer file sharing tools, such as BitTorrent, Hyperdrive lets you share unlimited files directly with other users without having to go through a central hub. One thing that sets HyperDrive apart is that the files you share can be modified. Since Hyperdrive has built-in versioning, you can explore the history of changes that you make to your files or others make to theirs. You can stream video and audio. There's no sign-up or account creation process required. It's all free software. This means that the community has the legal right, the tools, and the information necessary to participate in the development process, or even fork the project and go somewhere on[??]. Since last year's talk, we've added a few new features. It's now much easier to get started using hyperdrive.el, since we added the hyperdrive install command, which downloads the gateway program and installs it on your machine. We added a peer graph feature, which lets you visualize your network of sources of information. We also published a new package, hyperdrive-org-transclusion, which lets you display live updating snippets of Hyperdrive files in Org documents. Let's get started. Let's say you get an email from your friend Alice inviting you to check out hyperdrive.el. Let's click on the link here to look at the manual. Let's go down to the installation section. It says you need to have GNU Emacs version 28.1 or later. I'll assume you have it installed, but if not, you can click this link. You need to have curl installed, which is likely already the case, but if not, hyperdrive.el will let you know. Now from within Emacs you can install hyperdrive.el. Run M-x package-refresh-contents. Then, M-x package-install, type in hyperdrive, and hit return. Next, we'll install hypergateway ushin. Let's run M-x hyperdrive-install. It's prompting me to download and install the gateway, which is 51 megabytes. I'll press y. I will skip most of the download process. The download is wrapping up. Now, it's prompting me to start the gateway. I can either run M-x hyperdrive start, or I can press this button. And after a few moments, we'll see that the gateway will start. It's ready. You can also click this button, View Hyperdrive User Info Manual, to view the same manual that we were viewing in the browser. The info manual comes with hyperdrive.el and is available offline. Now that everything's installed and the gateway is running, we're back at Alice's email. Let's click on the link to her hyperdrive. Here's Alice's hyperdrive. Let's open her hello.org file. I'm being prompted to mark Alice's hyperdrive. Currently, it's unknown. I'll press ? to see more details. Safe, unsafe, unknown. I'll press e for explain. In the info manual, it says that if a hyperdrive is marked as safe, that means files in that hyperdrive will automatically have their major mode enabled based on their file extension. In this case, if I mark Alice's hyperdrive as safe and I click on this hello.org file, Emacs will automatically enable org mode in that file. Since I know Alice directly, I'll mark her hyperdrive as safe. I'll click on the file hello.org again, and now I'll press Shift-s for safe. Now I'm going to set Alice's pet name. Hyperdrive.el has different ways to name a hyperdrive. The public key is the drive's unique identifier. You can also assign yourself a public nickname, which you announce to the world. Then users on their own machines can assign each hyperdrive that they are aware of a private pet name. I'll assign Alice a pet name, which is the way that I will identify Alice just on my own machine. I'll type in my friend Alice. Then when I refresh her drive, we see that it says pet name in the top left. Now I'll do what she asks, which is to right click on her name and open her peer graph. In the context menu, I see a few different actions I can take, but I'll click on peer graph. It's prompting me to pick the max hops for sources. Default

  1. I'll choose the default. Same for blockers. I'll explain
what that means later. This is Alice's peer graph. Here we can see that Alice has two direct sources, Bob and Eve. Both Bob and Eve have assigned themselves nicknames, and so we see Bob and Eve here. Eve has assigned this other hyperdrive to be a source. But this other hyperdrive, whose public key is KB3ZR6MQ and so on--if we hover over the hyperdrive, we'll see the full public key--has not assigned itself a nickname. So we just see the truncated short public key. Because we've assigned a pet name for Alice, we see Alice's pet name show up here. Now I'll open the peer list view to show the same information in a different way. Since I'm using a large font, I'll widen this window so we can see everything going on in the peer list. We see that the root hyperdrive is pet name, my friend Alice, nickname Alice, public key, DM1, and so on. We've set the sources max hops to three, which means that we will go out at most three hops from Alice in order to find sources. One hop from Alice are Bob and Eve. One hop from Eve is KB3, which is two hops from Alice. Now we've set the max hops for sources to three, but in this case, the network is small, and KB3 has no sources. So we can only go out two hops from Alice until we run out of sources. In the future, we plan to add a search feature based on this list of sources. You type in a query, and you get back a list of results published by Eve, Bob, or KB3. It's also possible to block peers. I'll open the transient menu by pressing question mark. For now, let's turn on showing blocked peers. I'll press s x, and now it says blocked sources. Now we can see Mallory also shows up. Mallory has been included as a source by Eve, but is blocked. So in the end result, she does not show up in the list of sources. Instead, she shows up as a blocked source. But how is it that Mallory ended up being blocked? Let's open up the transient menu once more and click S, B to show blockers. Now we can see that Bob has blocked Mallory. This means that Bob is not interested in seeing search results from Mallory. Since Alice has included Bob as a blocker, that means that Alice trusts Bob to block people on her behalf. So since Bob has blocked Mallory, Mallory does not show up in Alice's peergraph as a source. Well, now I'm curious to see what it is Mallory published that drove Bob to block her. I'll right-click on Mallory, and in the context menu, open Hyperdrive, and... Wingsuit Flying Grand Canyon Point of View Not Clickbait? I gotta check this out. No! Disgusting! No wonder Bob blocked Mallory. Let's go back to Alice's paragraph. Mallory is a blocked source. Are there any peers who are blocked but who aren't sources? I'll open the transient menu and I'll press s x. Now we're looking at blocked non-sources. Darth, who's been blocked by Bob, has not been added by any of Alice's sources as a source. So Darth is a blocked non-source. I'll open the transient menu again and I'll press s x. Now we're looking at all blocked peers. Both Mallory and Darth show up. If I scroll down in the peer list, we'll see that Mallory is listed under blocked sources, and Darth is listed under blocked non-sources. This view has gotten a little bit busy. Let's say that we're only interested in how Alice relates to Mallory. I'll open the transient menu, and I will filter the graph to show paths only to Mallory. I'll press o a, type in Mallory, and hit enter. Now we see that Mallory has a bold border and KB3 and Darth are missing from the graph. Eve shows up because there's a path from Alice to Mallory that goes through Eve. Same for Bob. We can also add more than one peer in this view. I'll press OA again, and this time I'll choose Darth. Now Darth and Mallory have bold borders, and KB3 is still missing. We can remove Mallory and Darth one at a time with o r, or we can press C-u, the universal prefix argument, and then o r to remove both Mallory and Darth from the list. Now we're back to seeing everybody. This view of the graph is somewhat simplified because we're showing only the shortest paths between peers. If I press Shift-s now we're looking at all paths. Here, we can see that in addition to going from Alice to Bob as a source directly, we also go from Alice to Bob through Eve as a source. Let's go back to showing only the shortest paths. Currently, the max hops for sources and for blockers is set to 3. Let's see what happens if we set the blockers max hops to 0. After the graph reloads, we now see that Mallory shows up as a source, not as a blocked source. This is because Bob, who is one hop away from Alice, is no longer included as a blocker, because we've set the blockers max hops to 0. And since Bob is not a blocker, the fact that Bob blocks Mallory doesn't factor in. While Alice is the only one who gets to decide who her direct sources, blockers, and blocked peers are, anyone can view Alice's peer graph, tinker with the sources or blockers' max hops, and use her list of sources to do a search. Let's see what happens if we set sources' max hops to 2. Well, the graph doesn't change, because we only ever went out 2 hops anyway. Let's set source's max hops to 1. Now we only see Bob and Eve, since KB3 and Mallory are 2 hops out. Source hops 0. Now we're left with just Alice. Now I'd like to create a new hyperdrive and add Alice as a source. I'll run M-x hyperdrive new. Now it's prompting me for a new hyperdrive seed. Seed is a string of characters that's combined with your secret key, which is generated for you by the gateway program, in order to create a new public key for this new hyperdrive. I'll type in Joseph, a new drive. Now, I'll right-click on my own name, and in the context menu, I'll choose Set Relation From. From Joseph to Alice, as a source. Now, I'll right-click on my name again, and click on Peer Graph. Max hops, let's say three, and for blockers as well, Here's the peer graph from my newly created hyperdrive's perspective. Mallory is included as a source, three hops from the root. Let's open the list view. If we were to mark Alice as a blocker by clicking on this button in the blocker column next to the pet name for Alice, Then we see that Mallory shows up as a blocked source. Only now that Joseph includes Alice as a blocker does the fact that Bob blocked Mallory take effect. You can click on another peer to set them as the root hyperdrive. I'll click on Eve. Now we're looking at the peer graph from Eve's perspective. This peer graph view has a history. If I open the transient menu, you'll see here there's a back and a forward button. Back is bound to l for left. If I press l, we're looking at the graph from Joseph's perspective. If I press l again, now we're looking at Alice, as at the beginning. I'll press r to go forward, and r again, and now we're back at Eve. Now, I'd like to show you the hyperdrive-org-transclusion package we published. I'll open up the HyperDrive manual to see how to install it. Now, from here, I'll press m and type in transclusion to jump straight to the org-transclusion integration section of the manual. To summarize, we can install HyperDrive org-transclusion with M-x package-install hyperdrive-org-transclusion. Then, once this is done, we will copy the following snippet and add it either to our init.el file or, in this case, I'll just evaluate it. Now, hyperdrive-org-transclusion mode is enabled. Let's go back to the hello.org file in Alice's hyperdrive. I'll right-click on Alice, then click Open Hyperdrive, and hit Enter on hello.org. This time, I'm not prompted to mark Alice's hyperdrive as safe, since we did it already. I'll navigate to the Check Out My Peers heading. Then I'll run M-x org-store-link. Now, I'll create a new file in my hyperdrive by running M-x hyperdrive-find-file. I'll select my hyperdrive and I'll call this new file response-to-alice.org. Now, I'll paste in a response that I've written. I'll go to the bottom and I will run M-x org-insert-link and I'll insert the link that I stored earlier. As a link description, I'll change it to snippet from Alice's hyperdrive file. I'll save my hyperdrive file. Now, to show that the link works, I'll just kill this buffer showing the hello.org file, and I'll click on the link. Now in addition to linking from my file to the heading in Alice's file, I'd like to transclude this headings content in my file. So I'll go to the link in my file and run M-x org-transclusion-make-from-link and I'll save my file again. All that's been added is another link prefixed with this transclude keyword. To make the transclusion show up inline, I'll run M-x org-transclusion-add. The content of Alice's file hasn't been copied into mine. It's just being transcluded. If I run M-x org-transclusion-remove, you'll see that the underlying content is still just the transclude keyword and the link. Now some time has passed and Alice has updated her hyperdrive file. She wrote, please add me as a source. Since the transclude link that I added in my file does not specify a version, it will always transclude the latest version of Alice's hello.org file. If I run M-x org-transclusion-add once more, we'll see that the latest version of Alice's file gets transcluded into mine. Let's say that I want to transclude the previous version of Alice's hello.org file, and I don't want my transclusion to change when Alice updates her file. That's possible too. I'll go over to her file and run M-x hyperdrive-open-previous-version. Now when I widen the window a little, we can see in the mode line that this is version 57. For an explanation of what the version numbers mean, check out the last video or the info manual. For now, I'll go down to her check out my peers heading and run M-x org-store-link. Now, in my response to Alice, I'll insert the link. I'll run org transclusion make from link. Now there are two transclusions. The first transclusion will update whenever Alice updates her file, and the second transclusion is pinned to version 57. That covers Hyperdrive install, the peer graph, and hyperdrive-org-transclusion. Feel free to check out the hyperdrive.el info manual in the Ushin hyperdrive here, or on the ushin.org website. Thanks to the folks at Sopranica, we have a public XMPP group chat you can join, either using an XMPP client or anonymously from your browser. If you use Matrix, there's also a bridge. Thank you for watching, and thank you to the EmacsConf organizers and all the presenters who made this wonderful event possible.

Captioner: sachac

Questions or comments? Please e-mail contact@ushin.org

Back to the talks Previous by time: Re-imagining the Emacs user experience with Casual Suite Next by time: Emacs, eev, and Maxima - now! Track: General - Watch