New in hyperdrive.el: quick install, peer graph, transclusion!
Joseph Turner - https://ushin.org xmpp:discuss@conference.ushin.org (XMPP MUC for USHIN discussion), contact@ushin.org
The following image shows where the talk is in the schedule for Sat 2024-12-07. Solid lines show talks with Q&A via BigBlueButton. Dashed lines show talks with Q&A via IRC or Etherpad.
Format: 21-min talk ; Q&A: BigBlueButton conference room https://media.emacsconf.org/2024/current/bbb-hyperdrive.html Etherpad: https://pad.emacsconf.org/2024-hyperdrive
Etherpad: https://pad.emacsconf.org/2024-hyperdrive
Discuss on IRC: #emacsconf-gen
Status: Q&A open for participation
Saturday, Dec 7 2024, ~1:25 PM - 1:45 PM MST (US/Mountain)
Saturday, Dec 7 2024, ~12:25 PM - 12:45 PM PST (US/Pacific)
Saturday, Dec 7 2024, ~8:25 PM - 8:45 PM UTC
Saturday, Dec 7 2024, ~9:25 PM - 9:45 PM CET (Europe/Paris)
Saturday, Dec 7 2024, ~10:25 PM - 10:45 PM EET (Europe/Athens)
Sunday, Dec 8 2024, ~1:55 AM - 2:15 AM IST (Asia/Kolkata)
Sunday, Dec 8 2024, ~4:25 AM - 4:45 AM +08 (Asia/Singapore)
Sunday, Dec 8 2024, ~5:25 AM - 5:45 AM JST (Asia/Tokyo)
Description
hyperdrive.el is an Emacs interface to hyperdrive, a mutable, versioned, peer-to-peer shared filesystem. Among other things, hyperdrive.el has these features: Share unlimited files of unlimited size; Explore file history with built-in versioning; Stream video and audio; No signup or account creation; Free as in Freedom!
Since last year's EmacsConf talk, hyperdrive.el grew some new features, including:
- Easily install the gateway program with
M-x hyperdrive-install. - Visualize your network of sources as a graph or a list.
- Transclude snippets of hyperdrive files with hyperdrive-org-transclusion.
This talk will show off these new hyperdrive.el features in action!
Feel free to join our public XMPP chat room!
- xmpp:discuss@conference.ushin.org (Join anonymously from your browser)
- #_bifrost_discuss_conference.ushin.org:aria-net.org (Matrix bridge)
Bugs can be submitted to the ushin issue tracker. Patches, comments or questions can be submitted to the ushin public inbox.
About the speaker:
I'm Joseph Turner. I enjoy fiddle, Aikido, peer-to-peer networks, Emacs, and swimming in cold water. I work with USHIN, a tiny educational US nonprofit whose mission is to promote personal, community, and global health through free and open universal shared information for everybody.
See also: EmacsConf - 2023 - talks - hyperdrive.el: Peer-to-peer filesystem in Emacs
Transcript
Hello, welcome to this talk on hyperdrive.el. I'm Joseph
Turner presenting on behalf of Ushin, a tiny independent
information freedom nonprofit. Hyperdrive.el is an Emacs
interface to the Hyperdrive peer-to-peer file sharing
system built by HolePunch. Like other peer-to-peer file
sharing tools, such as BitTorrent, Hyperdrive lets you
share unlimited files directly with other users without
having to go through a central hub. One thing that sets
HyperDrive apart is that the files you share can be
modified. Since Hyperdrive has built-in versioning,
you can explore the history of changes that you make to your
files or others make to theirs. You can stream video and
audio. There's no sign-up or account creation process
required. It's all free software. This means that the
community has the legal right, the tools, and the
information necessary to participate in the development
process, or even fork the project
and go somewhere on[??]. Since
last year's talk, we've added a few new features. It's now
much easier to get started using hyperdrive.el, since we
added the hyperdrive install command, which downloads the
gateway program and installs it on your machine. We added a
peer graph feature, which lets you visualize your network
of sources of information. We also published a new package,
hyperdrive-org-transclusion, which lets you display live
updating snippets of Hyperdrive files in Org documents.
Let's get started. Let's say you get an email from your
friend Alice inviting you to check out hyperdrive.el.
Let's click on the link here to look at the manual. Let's go
down to the installation section. It says you need to have
GNU Emacs version 28.1 or later. I'll assume you have it
installed, but if not, you can click this link.
You need to have curl installed,
which is likely already the case, but if not,
hyperdrive.el will let you know.
Now from within Emacs you can install hyperdrive.el.
Run M-x package-refresh-contents.
Then, M-x package-install, type in hyperdrive,
and hit return. Next, we'll install hypergateway ushin.
Let's run M-x hyperdrive-install. It's prompting me to
download and install the gateway, which is 51 megabytes.
I'll press y.
I will skip most of the download process. The download is
wrapping up.
Now, it's prompting me to start the gateway. I can either run
M-x hyperdrive start, or I can press this button. And
after a few moments, we'll see that the gateway will start.
It's ready. You can also click this button, View Hyperdrive
User Info Manual, to view the same manual that we were
viewing in the browser. The info manual comes with
hyperdrive.el and is available offline. Now that
everything's installed and the gateway is running, we're
back at Alice's email. Let's click on the link to her
hyperdrive. Here's Alice's hyperdrive. Let's open her
hello.org file.
I'm being prompted to mark Alice's hyperdrive. Currently,
it's unknown. I'll press ? to see more details.
Safe, unsafe, unknown. I'll press e for explain. In the info
manual, it says that if a hyperdrive is marked as safe, that
means files in that hyperdrive will automatically have
their major mode enabled based on their file extension. In
this case, if I mark Alice's hyperdrive as safe and I click on
this hello.org file, Emacs will automatically enable org
mode in that file. Since I know Alice directly, I'll mark her
hyperdrive as safe. I'll click on the file hello.org again,
and now I'll press Shift-s for safe.
Now I'm going to set Alice's pet name. Hyperdrive.el has
different ways to name a hyperdrive. The public key is the
drive's unique identifier. You can also assign yourself a
public nickname, which you announce to the world. Then
users on their own machines can assign each hyperdrive that
they are aware of a private pet name. I'll assign Alice a
pet name, which is the way that I will identify Alice just on
my own machine. I'll type in my friend Alice.
Then when I refresh her drive,
we see that it says pet name in the top left.
Now I'll do what she asks,
which is to right click on her name and
open her peer graph. In the context menu, I see a few
different actions I can take, but I'll click on peer graph.
It's prompting me to pick the max hops for sources. Default
what that means later.
This is Alice's peer graph. Here we can see that Alice has two
direct sources, Bob and Eve. Both Bob and Eve have assigned
themselves nicknames, and so we see Bob and Eve here. Eve has
assigned this other hyperdrive to be a source. But this
other hyperdrive, whose public key is KB3ZR6MQ
and so on--if we hover over the hyperdrive, we'll see the full
public key--has not assigned itself a nickname. So we
just see the truncated short public key. Because we've
assigned a pet name for Alice,
we see Alice's pet name show up here.
Now I'll open the peer list view to show the same
information in a different way. Since I'm using a large
font, I'll widen this window so we can see everything going
on in the peer list. We see that the root hyperdrive is pet
name, my friend Alice, nickname Alice, public key, DM1, and
so on. We've set the sources max hops to three, which means
that we will go out at most three hops from Alice in order to
find sources. One hop from Alice are Bob and Eve. One hop
from Eve is KB3, which is two hops from Alice. Now we've set
the max hops for sources to three, but in this case, the
network is small, and KB3 has no sources. So we can only go
out two hops from Alice until we run out of sources. In the
future, we plan to add a search feature based on this list of
sources. You type in a query, and you get back a list of
results published by Eve, Bob, or KB3. It's also possible to
block peers. I'll open the transient menu by pressing
question mark. For now, let's turn on showing blocked
peers. I'll press s x, and now it says blocked sources.
Now we can see Mallory also shows up. Mallory has been
included as a source by Eve, but is blocked. So in the end
result, she does not show up in the list of sources. Instead,
she shows up as a blocked source. But how is it that Mallory
ended up being blocked? Let's open up the transient menu
once more and click S, B to show blockers. Now we can see that
Bob has blocked Mallory. This means that Bob is not
interested in seeing search results from Mallory. Since
Alice has included Bob as a blocker, that means that Alice
trusts Bob to block people on her behalf. So since Bob has
blocked Mallory, Mallory does not show up in Alice's
peergraph as a source. Well, now I'm curious to see what it is
Mallory published that drove Bob to block her. I'll
right-click on Mallory, and in the context menu, open
Hyperdrive, and... Wingsuit Flying Grand Canyon Point of
View Not Clickbait? I gotta check this out.
No! Disgusting! No wonder Bob blocked Mallory. Let's go
back to Alice's paragraph. Mallory is a blocked source.
Are there any peers who are blocked but who aren't sources?
I'll open the transient menu and I'll press s x. Now we're
looking at blocked non-sources. Darth, who's been blocked
by Bob, has not been added by any of Alice's sources as a
source. So Darth is a blocked non-source. I'll open the
transient menu again and I'll press s x. Now we're
looking at all blocked peers. Both Mallory and Darth show
up. If I scroll down in the peer list, we'll see that Mallory
is listed under blocked sources, and Darth is listed under
blocked non-sources. This view has gotten a little bit
busy. Let's say that we're only interested in how Alice
relates to Mallory. I'll open the transient menu, and I
will filter the graph to show paths only to Mallory. I'll
press o a, type in Mallory, and hit enter. Now we see that
Mallory has a bold border and KB3 and Darth are missing from
the graph. Eve shows up because there's a path from Alice to
Mallory that goes through Eve. Same for Bob. We can also add
more than one peer in this view. I'll press OA again, and
this time I'll choose Darth. Now Darth and Mallory have bold
borders, and KB3 is still missing. We can remove Mallory and
Darth one at a time with o r, or we can press C-u, the
universal prefix argument, and then o r to remove both
Mallory and Darth from the list. Now we're back to seeing
everybody. This view of the graph is somewhat simplified
because we're showing only
the shortest paths between peers.
If I press Shift-s now we're looking at all paths.
Here, we can see that in addition to going from Alice to
Bob as a source directly, we also go from Alice to Bob through
Eve as a source. Let's go back to showing only the shortest
paths. Currently, the max hops for sources and for blockers
is set to 3. Let's see what happens if we set the blockers max
hops to 0.
After the graph reloads, we now see that Mallory
shows up as a source, not as a blocked source.
This is because Bob,
who is one hop away from Alice, is no longer included as a
blocker, because we've set the blockers max hops to 0. And
since Bob is not a blocker, the fact that Bob blocks Mallory
doesn't factor in. While Alice is the only one who gets to
decide who her direct sources, blockers, and blocked peers
are, anyone can view Alice's peer graph, tinker with the
sources or blockers' max hops, and use her list of sources to
do a search. Let's see what happens if we set sources' max
hops to 2. Well, the graph doesn't change, because we only
ever went out 2 hops anyway. Let's set source's max hops to 1.
Now we only see Bob and Eve, since KB3 and Mallory are
2 hops out. Source hops 0. Now we're left with just Alice.
Now I'd like to create a new hyperdrive and add Alice as a
source. I'll run M-x hyperdrive new.
Now it's prompting me
for a new hyperdrive seed. Seed is a string of characters
that's combined with your secret key, which is generated
for you by the gateway program, in order to create a new
public key for this new hyperdrive. I'll type in Joseph, a
new drive. Now, I'll right-click on my own name, and in the
context menu, I'll choose Set Relation From. From Joseph
to Alice, as a source. Now, I'll right-click on my name again,
and click on Peer Graph. Max hops, let's say three, and for
blockers as well,
Here's the peer graph from my newly created
hyperdrive's perspective. Mallory is included as a
source, three hops from the root. Let's open the list view.
If we were to mark Alice as a blocker by clicking on this
button in the blocker column next to the pet name for Alice,
Then we see that Mallory shows up as a blocked source. Only
now that Joseph includes Alice as a blocker does the fact
that Bob blocked Mallory take effect. You can click on
another peer to set them as the root hyperdrive. I'll click
on Eve. Now we're looking at the peer graph from Eve's
perspective. This peer graph view has a history. If I open
the transient menu, you'll see here there's a back and a
forward button. Back is bound to l for left. If I press l,
we're looking at the graph from Joseph's perspective. If I
press l again, now we're looking at Alice, as at the
beginning. I'll press r to go forward, and r again, and now
we're back at Eve. Now, I'd like to show you
the hyperdrive-org-transclusion package we published.
I'll open up the HyperDrive manual
to see how to install it.
Now, from here, I'll press m and type in transclusion to jump
straight to the org-transclusion integration section of
the manual. To summarize, we can install HyperDrive
org-transclusion with M-x package-install
hyperdrive-org-transclusion.
Then, once this is done, we will copy the following snippet
and add it either to our init.el file or, in this case, I'll
just evaluate it. Now, hyperdrive-org-transclusion mode is
enabled.
Let's go back to the hello.org file in Alice's hyperdrive.
I'll right-click on Alice, then click Open Hyperdrive, and
hit Enter on hello.org. This time, I'm not prompted to mark
Alice's hyperdrive as safe, since we did it already. I'll
navigate to the Check Out My Peers heading. Then I'll run
M-x org-store-link. Now, I'll create a new file in my
hyperdrive by running M-x hyperdrive-find-file. I'll
select my hyperdrive and I'll call this new file
response-to-alice.org.
Now, I'll paste in a response that I've written.
I'll go to the bottom and I will run M-x org-insert-link
and I'll insert the link
that I stored earlier. As a link
description, I'll change it to snippet from Alice's
hyperdrive file.
I'll save my hyperdrive file.
Now, to show that the link works, I'll just kill this buffer
showing the hello.org file, and I'll click on the link.
Now in addition to linking from my file
to the heading in Alice's file,
I'd like to transclude this headings content in my file.
So I'll go to the link in my file and run
M-x org-transclusion-make-from-link
and I'll save my file again.
All that's been added is another link
prefixed with this transclude keyword.
To make the transclusion show up inline,
I'll run M-x org-transclusion-add. The content
of Alice's file hasn't been copied into mine. It's just
being transcluded. If I run M-x org-transclusion-remove,
you'll see that the underlying content is still
just the transclude keyword and the link.
Now some time has passed and Alice has updated her
hyperdrive file. She wrote, please add me as a source. Since
the transclude link that I added in my file does not specify a
version, it will always transclude the latest version of
Alice's hello.org file. If I run M-x org-transclusion-add
once more, we'll see that the latest version of Alice's
file gets transcluded into mine. Let's say that I want to
transclude the previous version of Alice's hello.org
file, and I don't want my transclusion to change when Alice
updates her file. That's possible too. I'll go over to her
file and run M-x hyperdrive-open-previous-version. Now
when I widen the window a little, we can see in the mode line
that this is version 57. For an explanation of what the
version numbers mean, check out the last video or the info
manual. For now, I'll go down to her check out my peers
heading and run M-x org-store-link. Now, in my response to
Alice, I'll insert the link. I'll run org transclusion make
from link. Now there are two transclusions. The first
transclusion will update whenever Alice updates her file,
and the second transclusion is pinned to version 57. That
covers Hyperdrive install, the peer graph, and
hyperdrive-org-transclusion.
Feel free to check out the hyperdrive.el
info manual in the Ushin hyperdrive here, or on the
ushin.org website. Thanks to the folks at Sopranica, we
have a public XMPP group chat you can join, either using an
XMPP client or anonymously from your browser. If you use
Matrix, there's also a bridge. Thank you for watching, and
thank you to the EmacsConf organizers and all the
presenters who made this wonderful event possible.
Captioner: sachac
Questions or comments? Please e-mail contact@ushin.org