Links

Hyperswarm

stable
Hyperswarm allows you to find and connect to peers announcing a common 'topic' that can be anything. With Hyperswarm, you can discover and connect peers with a shared interest over a distributed network. For example, we often use Hypercore's discovery key as the swarm topic for discovering peers to replicate with.
Hyperswarm offers a simple interface to abstract away the complexities of underlying modules such as HyperDHT and @hyperswarm/secretstream. These modules can also be used independently for specialized tasks.

Installation

Install with npm:
npm install hyperswarm

API

const swarm = new Hyperswarm(opts = {})

Construct a new Hyperswarm instance.
The following table describes the properties of the optional opts object.
Property
Description
keyPair
A Noise keypair will be used to listen/connect on the DHT. Defaults to a new key pair.
seed
A unique, 32-byte, random seed that can be used to deterministically generate the key pair.
maxPeers
The maximum number of peer connections allowed.
firewall
A sync function of the form remotePublicKey => (true|false). If true, the connection will be rejected. Defaults to allowing all connections.
dht
A DHT instance. Defaults to a new instance.

Properties:

swarm.connections

A set of all active client/server connections.

swarm.peers

A Map containing all connected peers, of the form: (Noise public key hex string) -> PeerInfo object
See the PeerInfo API for more details.

swarm.dht

A HyperDHT instance. Useful if you want lower-level control over Hyperswarm's networking.

Events

swarm.on('connection', (socket, peerInfo) => {})

Emitted whenever the swarm connects to a new peer.
socket is an end-to-end (Noise) encrypted Duplex stream
peerInfo is a PeerInfo instance

Methods

const discovery = swarm.join(topic, opts = {})

Returns a PeerDiscovery object.
Start discovering and connecting to peers sharing a common topic. As new peers are connected, they will be emitted from the swarm as connection events.
topic must be a 32-byte Buffer and use a publicly sharable id, typically a Hypercore discoveryKey which we can then link to (join will leak the topic to DHT nodes).
opts can include:
Property
Description
Type
Default
server
Accept server connections for this topic by announcing yourself to the DHT
Boolean
true
client
Actively search for and connect to discovered servers
Boolean
true
Calling swarm.join() makes this core directly discoverable. To ensure that this core remains discoverable, Hyperswarm handles the periodic refresh of the join. For maximum efficiency, fewer joins should be called; if sharing a single Hypercore that links to other Hypercores, only join a topic for the first one.

Clients and Servers

In Hyperswarm, there are two ways for peers to join the swarm: client mode and server mode. If you've previously used Hyperswarm v2, these were called 'lookup' and 'announce', but we now think 'client' and 'server' are more descriptive.
When you join a topic as a server, the swarm will start accepting incoming connections from clients (peers that have joined the same topic in client mode). Server mode will announce your keypair to the DHT so that other peers can discover your server. When server connections are emitted, they are not associated with a specific topic -- the server only knows it received an incoming connection.
When you join a topic as a client, the swarm will do a query to discover available servers, and will eagerly connect to them. As with server mode, these connections will be emitted as connection events, but in client mode, they will be associated with the topic (info.topics will be set in the connection event).

Methods

await swarm.leave(topic)

Stop discovering peers for the given topic.
topic must be a 32-byte Buffer
If a topic was previously joined in server mode, leave will stop announcing the topic on the DHT.
If a topic was previously joined in client mode, leave will stop searching for servers announcing the topic.
leave will not close any existing connections.

swarm.joinPeer(noisePublicKey)

Establish a direct connection to a known peer.
noisePublicKey must be a 32-byte Buffer
As with the standard join method, joinPeer will ensure that peer connections are reestablished in the event of failures.

swarm.leavePeer(noisePublicKey)

Stop attempting direct connections to a known peer.
noisePublicKey must be a 32-byte Buffer
If a direct connection is already established, that connection will not be destroyed by leavePeer.

const discovery = swarm.status(topic)

Get the PeerDiscovery object associated with the topic, if it exists.

await swarm.listen()

Explicitly start listening for incoming connections. This will be called internally after the first join, so it rarely needs to be called manually.

await swarm.flush()

Wait for any pending DHT announcements, and for the swarm to connect to any pending peers (peers that have been discovered, but are still in the queue awaiting processing).
Once a flush() has completed, the swarm will have connected to every peer it can discover from the current set of topics it's managing.
flush() is not topic-specific, so it will wait for every pending DHT operation and connection to be processed -- it's quite heavyweight, so it could take a while. In most cases, it's not necessary, as connections are emitted by swarm.on('connection') immediately after they're opened.

Peer Discovery

swarm.join returns a PeerDiscovery instance which allows you to both control discovery behavior, and respond to lifecycle changes during discovery.

Methods

await discovery.flushed()

Wait until the topic has been fully announced to the DHT. This method is only relevant in server mode. When flushed() has completed, the server will be available to the network.

await discovery.refresh({ client, server })

Update the PeerDiscovery configuration, optionally toggling client and server modes. This will also trigger an immediate re-announce of the topic when the PeerDiscovery is in server mode.

await discovery.destroy()

Stop discovering peers for the given topic.
If a topic was previously joined in server mode, leave will stop announcing the topic on the DHT.
If a topic was previously joined in client mode, leave will stop searching for servers announcing the topic.

PeerInfo

swarm.on('connection', ...) emits a PeerInfo instance whenever a new connection is established.
There is a one-to-one relationship between connections and PeerInfo objects -- if a single peer announces multiple topics, those topics will be multiplexed over a single connection.

Properties:

peerInfo.publicKey

The peer's Noise public key.

peerInfo.topics

An Array of topics that this Peer is associated with -- topics will only be updated when the Peer is in client mode.

peerInfo.prioritized

If true, the swarm will rapidly attempt to reconnect to this peer.

Methods:

peerInfo.ban()

Ban the peer. This will prevent any future reconnection attempts, but it will not close any existing connections.

HyperDHT

The DHT powering Hyperswarm and built on top of dht-rpc. The HyperDHT uses a series of holepunching techniques to make sure connectivity works on most networks and is mainly used to facilitate finding and connecting to peers using end-to-end encrypted Noise streams.
In the HyperDHT, peers are identified by a public key, not by an IP address. If you know someone's public key, you can connect to them regardless of where they're located, even if they move between different networks.

Installation

Install with npm:
npm install hyperdht

API

const node = new DHT([options])

Create a new DHT node.
options include:
Property
Description
Type
Default
bootstrap
overwrite the default bootstrap servers, just need to be an array of any known DHT node(s)
Array
['node1.hyperdht.org:49737', 'node2.hyperdht.org:49737', 'node3.hyperdht.org:49737']
keyPair
optionally pass the public key and secret key as a key pair to use for server.listen and connect
Object
null
See dht-rpc for more options as HyperDHT inherits from that.
The default bootstrap servers are publicly served on behalf of the commons. To run a fully isolated DHT, start one or more DHT nodes with an empty bootstrap array (new DHT({bootstrap:[]})) and then use the addresses of those nodes as the bootstrap option in all other DHT nodes. You'll need at least one persistent node for the network to be completely operational.

Methods

keyPair = DHT.keyPair([seed])

Generates the required key pair for DHT operations.
Returns an object with {publicKey, secretKey}. publicKey holds a public key buffer, secretKey holds a private key buffer.
Any options passed are forwarded to dht-rpc.

node = DHT.bootstrapper(port, host, [options])

To run your own Hyperswarm network use this method to easily create a bootstrap node.

await node.destroy([options])

Fully destroy this DHT node.
This will also unannounce any running servers. If you want to force close the node without waiting for the servers to unannounce pass { force: true }.

Creating P2P Servers

const server = node.createServer([options], [onconnection])

Create a new server for accepting incoming encrypted P2P connections.
options include:
{
firewall (remotePublicKey, remoteHandshakePayload) {
// validate if you want a connection from remotePublicKey
// if you do return false, else return true
// remoteHandshakePayload contains their ip and some more info
return true
}
}
You can run servers on normal home computers, as the DHT will UDP holepunch connections for you.

Events

server.on('connection', socket)

Emitted when a new encrypted connection has passed the firewall check.
socket is a NoiseSecretStream instance.
You can check who you are connected to using socket.remotePublicKey and socket.handshakeHash contains a unique hash representing this crypto session (same on both sides).

server.on('listening')

Emitted when the server is fully listening on a keyPair.

server.on('close')

Emitted when the server is fully closed.

await server.listen(keyPair)

Make the server listen on a keyPair. To connect to this server use keyPair.publicKey as the connect address.

Methods

server.refresh()

Refresh the server, causing it to reannounce its address. This is automatically called on network changes.

server.address()

Returns an object containing the address of the server:
{
host, // external IP of the server,
port, // external port of the server if predictable,
publicKey // public key of the server
}
You can also get this info from node.remoteAddress() minus the public key.

await server.close()

Stop listening.

Connecting to P2P Servers

const socket = node.connect(remotePublicKey, [options])

Connect to a remote server. Similar to createServer this performs UDP hole punching for P2P connectivity.
const node = new DHT()
const remotePublicKey = Buffer.from('public key of remote peer', 'hex')
const encryptedSocket = node.connect(remotePublicKey)
options include:
Property
Description
Type
Default
nodes
optional array of close dht nodes to speed up connecting
Array
[]
keyPair
optional key pair to use when connection
Object
node.defaultKeyPair

Properties

socket.remotePublicKey

The public key of the remote peer.

socket.publicKey

The public key of the connection.

Events

socket.on('open')

Emitted when the encrypted connection has been fully established with the server.
encryptedSocket.on('open', function () {
console.log('Connected to server')
})

Additional Peer Discovery

const stream = node.lookup(topic, [options])

Look for peers in the DHT on the given topic. The topic should be a 32-byte buffer (normally a hash of something).
The returned stream looks like this
{
// Who sent the response?
from: { id, host, port },
// What address they responded to (i.e. your address)
to: { host, port },
// List of peers announcing under this topic
peers: [ { publicKey, nodes: [{ host, port }, ...] } ]
}
To connect to the peers, you should also call connect afterward with those public keys.
If you pass any options they are forwarded to dht-rpc.

Methods

const stream = node.announce(topic, keyPair, [relayAddresses], [options])

Announce that you are listening on a key pair to the DHT under a specific topic. An announce does a parallel lookup so the stream returned looks like the lookup stream.
Any passed options are forwarded to dht-rpc.
When announcing you'll send a signed proof to peers that you own the key pair and wish to announce under the specific topic. Optionally you can provide up to 3 nodes, indicating which DHT nodes can relay messages to you - this speeds up connects later on for other users.
Creating a server using dht.createServer automatically announces itself periodically on the key pair it is listening on. When announcing the server under a specific topic, you can access the nodes it is close to using server.nodes.

await node.unannounce(topic, keyPair, [options])

Unannounce a key pair.
Any passed options are forwarded to dht-rpc.

Mutable/Immutable Records

Methods

const { hash, closestNodes } = await node.immutablePut(value, [options])

Store an immutable value in the DHT. When successful, the hash of the value is returned.
Any passed options are forwarded to dht-rpc.

const { value, from } = await node.immutableGet(hash, [options])

Fetch an immutable value from the DHT. When successful, it returns the value corresponding to the hash.
Any passed options are forwarded to dht-rpc.

await { publicKey, closestNodes, seq, signature } = node.mutablePut(keyPair, value, [options])

Store a mutable value in the DHT.
Any passed options are forwarded to dht-rpc.

await { value, from, seq, signature } = node.mutableGet(publicKey, [options])

Fetch a mutable value from the DHT.
options include:
  • seq - OPTIONAL, default 0, a number that will only return values with corresponding seq values that are greater than or equal to the supplied seq option.
  • latest - OPTIONAL - default false, a boolean indicating whether the query should try to find the highest seq before returning, or just the first verified value larger than options.seq it sees.
Any passed options are forwarded to dht-rpc.

SecretStream

SecretStream is used to securely create connections between two peers in Hyperswarm. It is powered by Noise and libsodium's SecretStream. SecretStream can be used as a standalone module to provide encrypted communication between two parties.
The SecretStream instance is a Duplex stream that supports usability as a normal stream for standard read/write operations. Furthermore, its payloads are encrypted with libsodium's SecretStream for secure transmission.

Installation

Install with npm:
npm install @hyperswarm/secret-stream

API

const s = new SecretStream(isInitiator, [rawStream], [options])

Make a new stream.
isInitiator is a boolean indicating whether you are the client or the server.
rawStream can be set to an underlying transport stream you want to run the noise stream over.
options include:
Property
Description
Type
pattern
Accept server connections for this topic by announcing yourself to the DHT
String
remotePublicKey
PublicKey of the other party
String
keyPair
Combination of PublicKey and SecretKey
{ publicKey, secretKey }
handshake
To use a handshake performed elsewhere, pass it here
{ tx, rx, handshakeHash, publicKey, remotePublicKey }
The SecretStream returned is a Duplex stream that you use as a normal stream, to write/read data from, except its payloads are encrypted using the libsodium secretstream.
By default, the above process uses ed25519 for the handshakes.
To load the key pair asynchronously, the secret stream also supports passing in a promise instead of the keypair that later resolves to { publicKey, secretKey }. The stream lifecycle will wait for the resolution and auto-destroy the stream if the promise gives an error.

Properties

s.publicKey

Get the local public key.

s.remotePublicKey

Get the remote's public key. Populated after open is emitted.

s.handshakeHash

Get the unique hash of this handshake. Populated after open is emitted.

Events

s.on('connect', onConnectHandler)

Emitted when the handshake is fully done. It is safe to write to the stream immediately though, as data is buffered internally before the handshake has been completed.

Methods

s.start(rawStream, [options])

Start a SecretStream from a rawStream asynchronously.
const s = new SecretStream({
autoStart: false // call start manually
})
// ... do async stuff or destroy the stream
s.start(rawStream, {
... options from above
})

s.setTimeout(ms)

Set the stream timeout. If no data is received within a ms window, the stream is auto-destroyed.

s.setKeepAlive(ms)

Send a heartbeat (empty message) every time the socket is idle for ms milliseconds.

const keyPair = SecretStream.keyPair([seed])

Generate an ed25519 key pair.