Links

Overview

Welcome to the Holepunch docs! 👋
Holepunch equips developers with a powerful suite of independent components to effortlessly construct peer-to-peer applications.

Building blocks

The Holepunch Ecosystem is constructed utilizing the following structural components.
  1. 1.
    Hypercore: A distributed, secure append-only log is a useful tool for creating fast and scalable applications without a backend, as it is entirely peer-to-peer.
  2. 2.
    Hyperbee: An append-only B-tree running on a Hypercore. It provides key-value store API, with methods for inserting and getting key/value pairs, atomic batch insertions, and creation of sorted iterators.
  3. 3.
    Hyperdrive: A secure, real-time distributed file system that simplifies peer-to-peer (P2P) file sharing. It provides an efficient way to store and access data across multiple connected devices in a decentralized manner.
  4. 4.
    Autobase: An experimental module that's used to automatically rebase multiple causally-linked Hypercores into a single, linearized Hypercore for multi-user collaboration.
  5. 5.
    HyperDHT: A DHT powering Hyperswarm. Through this DHT, each server is bound to a unique key pair, with the client connecting to the server using the server's public key.
  6. 6.
    Hyperswarm: A high-level API for finding and connecting to peers who are interested in a "topic."

Helpers

Helper modules can be used together with the building blocks to create cutting-edge peer-to-peer tools and applications.
  1. 1.
    Corestore: A Hypercore factory designed to facilitate the management of sizable named Hypercore collections.
  2. 2.
    Localdrive: A file system interoperable with Hyperdrive.
  3. 3.
    MirrorDrive: Mirror a Hyperdrive or a Localdrive into another one.
  4. 4.
    SecretStream: SecretStream is used to securely create connections between two peers in Hyperswarm.
  5. 5.
    Compact Encoding: A series of binary encoding schemes for building fast and small parsers and serializers. We use this in Keet to store chat messages and in Hypercore's replication protocol.
  6. 6.
    Protomux: Multiplex multiple message oriented protocols over a stream.

Tools built using Holepunch

A CLI to create and connect to P2P E2E encrypted shells.
A swiss-knife proxy powered by HyperDHT.
A one-to-one and end-to-end encrypted internet pipe.
A CLI to run SSH over the HyperDHT.
CLI to download, seed, and mirror a Hyperdrive or a Localdrive.
These tools are extensively employed in the day-to-day development and operation of applications built on Holepunch, like Keet.io.

Applications built using Holepunch

  • Keet.io: A peer-to-peer chat and video-conferencing application with end-to-end encryption.

What's new?

If you're already familiar with Hypercore, Hyperswarm, and the rest of the modules here, you might be wondering what's changed recently. In short: a lot! Here's a brief breakdown of some of the highlights.

Better Building Blocks

We've decided to focus entirely on making our core building blocks as easy to use, as fast, and as reliable as possible. The goal is to give developers all the essential pieces needed to make killer P2P apps, without being unnecessarily opinionated and without introducing operational complexity.
We'd previously introduced the Hyperspace daemon and hyp CLI tool as core modules, but in practice found that they don't fit cleanly into this vision. We feel it's better to keep a tight focus on the fundamental building blocks so that others can build similar higher-level tools with zero friction. Given that, they are both now considered deprecated.

Hypercore v10

  • The session and snapshot methods for providing multiple views over the same underlying Hypercore, which simplifies resource management.
  • A truncate method for intentionally creating a new fork, starting at a given length. We use this method extensively in Autobase, as described below.
  • An improved fork detection in the replication protocol, to improve resilience.
  • Optional on-disk encryption for blocks (in addition to the existing transport encryption).
  • The storage layer now uses a write-ahead log to better ensure that power loss or unexpected shutdown cannot lead to data corruption.

Hyperswarm v4

  • An improved UDP holepunching algorithm that uses arbitrary DHT nodes (optionally selected by the connecting peers) to proxy necessary metadata while being maximally privacy-preserving.
  • A custom-built transport protocol, UDX, that takes advantage of the holepunching algorithm to avoid unnecessary overhead (it doesn't include handshaking since holepunching takes care of that, for example). It's blazing fast.
  • A simplified DHT API that closely resembles NodeJS's net module, but using public keys instead of IPs.

Hyperdrive

  • Uses Hyperbee internally for storing file metadata
  • Major API simplification. Instead of mirroring POSIX APIs, the new API better captures the core requirements of P2P file transfer.
  • Auxiliary tools, Localdrive and MirrorDrive, that streamline import/export flows and make it easy to mirror drives to and from the local filesystem. We use these every day when deploying Keet.

Autobase (experimental)

Hypercores are single-writer data structures, but collaboration is crucial. Autobase is an experimental module that allows you to turn many Hypercores, owned by many different people, into a single 'virtual' Hypercore. In Keet, every member of a room has their input Hypercore where they write chat messages, and Autobase merges these into the linear view you see on screen.
Since Autobase's output shares the familiar Hypercore API, you can plug it into higher-level modules like Hyperbee and Hyperdrive, getting a multi-user collaboration with little additional effort.
Autobase is still experimental and is likely to change significantly in the near future. If you're feeling adventurous, give it a shot!

Stability indexing

Throughout the documentation, indications of a module's stability are provided. Some modules are well-established and used widely, making them highly unlikely to ever change. Other modules may be new, experimental, or known to have risks associated with their use.
The following stability indices have been used:
Stability
Description
stable
Unlikely to change or be removed in the foreseeable future.
experimental
New, untested, or have known issues.
deprecated
Being removed or replaced in the future.
unstable
May change or be removed without warning.

Stability overview

Module
Stability
Hypercore
stable
Hyperbee
stable
stable
Autobase
experimental
stable
HyperDHT
stable
Any part of a module (method, event, or property) that is not documented as part of that module's public API is subject to change at any time.
Last modified 3mo ago