GitHubTwitterDiscord
Search…
⌃K
Links

Hyperdrive

Hyperdrive is a secure, real-time distributed file system designed for easy P2P file sharing. We use it extensively inside Holepunch; apps like Keet are distributed to users as Hyperdrives, as is the Holepunch platform itself.

Installation

Install with npm:
npm install [email protected]

API

const drive = new Hyperdrive(corestore, [key])

Creates a new Hyperdrive instance.
corestore must be an instance of Corestore.
key should be a Hypercore public key. If you do not set this, Hyperdrive will use the core at { name: 'db' } in the passed Corestore instance.

Properties:

const hypercore = drive.core

The underlying Hypercore backing the drive.

const buffer = drive.key

The public key of the Hypercore backing the drive.

const buffer = drive.discoveryKey

The hash of the public key of the Hypercore backing the drive can be used to seed the drive using Hyperswarm.

const buffer = drive.contentKey

The public key of the Hyperblobs instance holding blobs associated with entries in the drive.

const integer = drive.version

The version (offset in the underlying Hypercore) of the drive.

Methods:

await drive.ready()

Wait for the drive to open fully. In general, you do NOT need to wait for ready unless checking a synchronous property on drive since internals await this themselves.

const buffer = await drive.get(path)

Returns the blob at path in the drive. Internally, Hyperdrive contains a metadata index of entries that 'point' to offsets in a Hyperblobs instance. Blobs themselves are accessible via drive.get(path), whereas entries are accessible via drive.entry(path). If no blob exists at path, returns null.

const stream = drive.createReadStream(path, options)

Returns a stream that can be used to read out the blob stored in the drive at path.
options are the same as the options to Hyperblobs().createReadStream(path, options).
const Hyperdrive = require('hyperdrive')
const Corestore = require('corestore')
const fs = require('fs')
const corestore = new Corestore('storage')
​
const drive = new Hyperdrive(corestore)
​
const readStream = fs.createReadStream('./my-data.txt')
const ws = drive.createWriteStream('/blob.txt')
​
// write data from my-data.txt to blob.txt of the drive
readStream.pipe(ws)
​
ws.on('close', function () {
const rs = drive.createReadStream('/blob.txt')
rs.pipe(process.stdout) // prints Hello, world!
})

const entry = await drive.entry(path)

Returns the entry at path in the drive. An entry holds metadata about a path, currently:
{
seq: Number,
key: String,
value: {
executable: Boolean, // whether the blob at the path is an executable
linkname: null // if entry is not symlink, otherwise a string to the entry this links to
blob: { // a Hyperblob id that can be used to fetch the blob associated with this entry
blockOffset: Number,
blockLength: Number,
byteOffset: Number,
byteLength: Number
},
metadata: null
}
}
Creates an entry in drive at path that points to the entry at linkname.
If a blob entry currently exists at path , then drive.symlink(path, linkname) will overwrite the entry and drive.get(path) will return null, while drive.entry(path) will return the entry with symlink information.

const hyperblobs = await drive.getBlobs()

Returns the hyperblobs instance storing the blobs indexed by drive entries.
const fs = require('fs')
const Corestore = require('corestore')
const Hyperdrive = require('hyperdrive')
const b4a = require('b4a')
​
const drive = new Hyperdrive(new Corestore('storage'))
​
await drive.put('./drive-data.txt', fs.readFileSync('./my-data.txt'))
const bufFromGet = await drive.get('./drive-data.txt')
​
const { value: entry } = await drive.entry('./drive-data.txt')
const blobs = await drive.getBlobs()
const bufFromEntry = blobs.get(entry.blob)
​
console.log(b4a.compare(bufFromGet, bufFromEntry)) // prints 0

const stream = await drive.entries([options])

Returns a read stream of entries in the drive.
options are the same as the options to Hyperbee().createReadStream().
for await (const entry of drive.entries()) {
console.log(entry)
}

await drive.put(path, blob, [options])

Sets the blob in the drive at path.
path should be a utf8 string. blob should be a Buffer.
options includes:
Property
Description
Type
Default
executable
whether the blob is executable or not
Boolean
true

const ws = drive.createWriteStream(path, [options])

Stream a blob into the drive at path.
options include
Property
Description
Type
Default
executable
whether the blob is executable or not
Boolean
true
const Hyperdrive = require('hyperdrive')
const Corestore = require('corestore')
​
const corestore = new Corestore('storage')
​
const drive = new Hyperdrive(corestore, /* optionalKey */)
​
// to initialize the keys and other properties of the drive
await drive.ready()
​
const ws = drive.createWriteStream('/blob.txt')
​
ws.write('Hello, ')
ws.write('world!')
ws.end()

await drive.del(path)

Removes the entry at path from the drive. If a blob corresponding to the entry at path exists, it is not currently deleted.

const Hyperdrive = drive.checkout(version)

Checks out a read-only snapshot of a Hyperdrive at a particular version.
const fs = require('fs')
const Corestore = require('corestore')
const Hyperdrive = require('hyperdrive')
​
const drive = new Hyperdrive(new Corestore('storage'))
​
await drive.put('/fst-file.txt', fs.readFileSync('fst-file.txt'))
​
const version = drive.version
​
await drive.put('/snd-file.txt', fs.readFileSync('snd-file.txt'))
​
const snapshot = drive.checkout(version)
​
console.log(await drive.get('/snd-file.txt')) // prints Buffer
console.log(await snapshot.get('/snd-file.txt')) // prints null
console.log(Buffer.compare(await drive.get('/fst-file.txt')
​
await snapshot.get('/fst-file.txt'))) // prints 0

const stream = drive.diff(version, folder, [options])

Efficiently create a stream of shallow changes to folder between version and drive.version.
Each entry is sorted by key and looks like this:
{
left: <the entry in the folder at drive.version for some path>,
right: <the entry in the folder at drive.checkout(version) for some path>
}
If an entry exists in drive.version of the folder but not in version, then left is set, right will be null, and vice versa.
await drive.put('/fst-file.txt', fs.readFileSync('fst-file.txt'))
​
const version = drive.version
​
await drive.put('/snd-file.txt', fs.readFileSync('snd-file.txt'))
​
const snapshot = drive.checkout(version)
​
console.log(await drive.get('/snd-file.txt')) // prints Buffer
console.log(await snapshot.get('/snd-file.txt')) // prints null
console.log(b4a.compare(await drive.get('/fst-file.txt'), await snapshot.get('/fst-file.txt'))) // prints 0
​
const stream = drive.diff(version,'/')
for await (const { left, right } of stream) console.log(left, right)

await drive.downloadDiff(version, folder, [options])

Downloads all the blobs in folder corresponding to entries in drive.checkout(version) that are not in drive.version. In other words, it downloads all the blobs added to folder up to version of the drive.

const stream = drive.list(folder, [options])

Returns a stream of all entries in the drive at paths prefixed with folder.
options include:
Property
Description
Type
Default
recursive
whether to descend into all subfolders or not
Boolean
true

await drive.download(folder, [options])

Downloads the blobs corresponding to all entries in the drive at paths prefixed with folder. Options are the same as those for drive.list(folder, [options]).

const stream = drive.readdir(folder)

Returns a stream of all subpaths of entries in the drive stored at paths prefixed by folder.
await drive.put('/parent/child', Buffer.from('child'))
await drive.put('/parent/sibling', Buffer.from('sibling'))
for await (const path of drive.readdir('/parent')) console.log(path) // prints 'child', then prints 'sibling'

await drive.close()

Close the drive and its underlying Hypercore-backed data structures.

const batch = drive.batch()

Atomically mutates the drive, and has the same interface as Hyperdrive.

await batch.flush()

Atomically commit a batch of mutations to the underlying drive.

await downloadRange(dbRanges, blobRanges)

Downloads the entries and blobs stored in the ranges dbRanges and blobRanges.
Building blocks - Previous
Hyperbee
Next - Building blocks
Autobase
Last modified 2mo ago