GitHubTwitterDiscord
Search…
⌃K
Links

Hyperbee

​stable​
Hyperbee is an append-only B-tree based on Hypercore. It provides a key/value-store API, with methods for inserting and getting key/value pairs, atomic batch insertions, and creating sorted iterators. It uses a single Hypercore for storage, using a technique called embedded indexing. It provides features like cache warmup extension, efficient diffing, version control, sorted iteration, and sparse downloading.
As with the Hypercore, a Hyperbee can only have a single writer on a single machine; the creator of the Hyperdrive is the only person who can modify to it, because they're the only one with the private key. That said, the writer can replicate to many readers, in a manner similar to BitTorrent.

Installation

Install with npm:
npm install hyperbee

API

const db = new Hyperbee(core, [options])

Make a new Hyperbee instance. core should be a Hypercore.
options include:
Property
Description
Type
Default
valueEncoding
Encoding type for the values. Takes values of 'json', 'utf-8', or 'binary'.
String
'binary'
keyEncoding
Encoding type for the keys. Takes values of 'ascii', 'utf-8', or 'binary'.
String
'binary'
Currently read/diff streams sort based on the encoded value of the keys.

Properties

db.version

Current version. An incrementing number, starting at 1, reflects all put/del operations that have been performed on the Hyperbee.

Methods

await db.put(key, [value], [options])

Insert a new key. Value can be optional.
options include:
Property
Description
Type
Default
cas
Function that controls whether the put succeeds.
Function
(prev, next) => { return true }
If you are inserting a series of data atomically, or you just have a batch of inserts/deletions available using a batch can be much faster than simply using a series of puts/dels on the db.

const batch = db.batch()

Make a new batch.

await batch.put(key, [value], [options])

Insert a key into a batch.
options include:
Property
Description
Type
Default
cas
Function that controls whether the put succeeds.
Function
(prev, next) => { return true }
const Hyperbee = require('hyperbee')
const Hypercore = require('hypercore')
​
// Create a Hypercore instance for the Hyperbee
// value encoding can be set directly in the Hyperbee constructor
const core = new Hypercore('./hyperbee-storage')
​
const db = new Hyperbee(core, {
keyEncoding: 'utf-8',
valueEncoding: 'binary'
})
const batch = db.batch()
​
await batch.put('key1', 'value1')
await batch.put('key2', 'value2')
await batch.put('key3', 'value3')
​
// execute the batch
await batch.flush()

const { seq, key, value } = await batch.get(key)

and Get a key, and value out of a batch.

await batch.del(key, [options])

Delete a key into the batch.
options include:
Property
Description
Type
Default
cas
Function that controls whether the put succeeds.
Function
(prev, next) => { return true }

await batch.flush()

Commit the batch to the database.

batch.destroy()

Destroy a batch and releases any locks it has acquired on the db. Call this if you want to abort a batch without flushing it.

const stream = db.createReadStream([options])

Make a read stream. All entries in the stream are similar to the ones returned from .get and the sort order is based on the binary value of the keys.
options include:
Property
Description
Type
Default
gt
only return keys > than this value
Integer
null
gte
only return keys >= than this
Integer
null
lt
only return keys < than this
Integer
null
lte
only return keys <= than this
Integer
null
reverse
determine order of the keys
Boolean
false
limit
max number of entries you want
Integer
-1
const Hyperbee = require('hyperbee')
const Hypercore = require('hypercore')
​
// Create a Hypercore instance for the Hyperbee
const core = new Hypercore('./hyperbee-storage', {
valueEncoding: 'utf-8' // The blocks will be UTF-8 strings.
})
​
const db = new Hyperbee(core, {
keyEncoding: 'utf-8',
valueEncoding: 'binary'
})
await db.put('key', 'value')
​
for await (const { key, value } of db.createReadStream()) {
console.log(`${key} -> ${value}`)
}

const { seq, key, value } = await db.peek([options])

Similar to doing a read stream and returning the first value, but a bit faster than that.

const stream = db.createHistoryStream([options])

Create a stream of all entries ever inserted or deleted from the db. Each entry has an additional type property indicating if it was a put or del operation.
options include:
Property
Description
Type
Default
live
determine whether the stream will wait for new data and never end or not
Boolean
false
reverse
determine the order in which data is received
Boolean
false
gt
start after this index
Integer
null
gte
start with this seq (inclusive)
Integer
null
lt
stop before this index
Integer
null
lte
stop after this index
Integer
null
limit
max number of entries you want
Integer
-1
If any of the gte, gt, lte, lt arguments are < 0 then they'll implicitly be added with the version before starting so doing { gte: -1 } makes a stream starting at the last index.

const stream = db.createDiffStream(otherVersion, [options])

Efficiently create a stream of the shallow changes between two versions of the db. Each entry is sorted by key and looks like this:
{
left: <the entry in the db>,
right: <the entry in the other version>
}
If an entry exists in db but not in the other version, then left is set and right will be null, and vice versa.
If the entries are causally equal (i.e. they have the same seq), they are not returned, only the diff.
Accepts the same options as the read stream except for reverse.
const db = new Hyperbee(new Hypercore(ram), {
keyEncoding: 'utf-8',
valueEncoding: 'utf-8'
})
await db.ready()
const keys = 'abcdefghijkl'
for (const char of keys) {
await db.put(char, char)
console.log(`Version after inserting ${char}: ${db.version}`)
}
​
// The createDiffStream method allows us to observe differences between versions of the Hyperbee.
// Let's see what's changed between the latest version, and version 9.
console.log(chalk.green('\nDiff between the latest version, and version 9:\n'))
for await (const { left, right } of db.createDiffStream(9)) {
// Since we've only inserted values, `right` will always be null.
console.log(`left -> ${left.key}, right -> ${right}`)
}

const watcher = db.watch([range], [onchange])

The watcher API is still under active development and subject to breaking changes. Therefore, it is strongly suggested to avoid using it in production environments.
watcher listens to the changes that are in the optional range. The range options include all the properties of createReadStream options except for reverse.
Methods:
watcher.destroy()
This closes the watcher.
Events:
watcher.on('change', (newVersion, oldVersion) => {})
Emitted on any feed change.
watcher.on('error', onerror)
Throws any critical and unexpected errors and the watcher gracefully auto closes on errors.
watcher.on('close', onclose)
Emitted when watcher is closed.

const dbCheckout = db.checkout(version, opts = {})

Get a read-only db checkout of a previous version.
options include:
Property
Description
Type
Default
valueEncoding
Encoding type for the values. Takes values of 'json', 'utf-8', or 'binary'.
String
defaults to the parents
keyEncoding
Encoding type for the keys. Takes values of 'ascii', 'utf-8', or 'binary'.
String
defaults to the parents

const dbCheckout = db.snapshot()

Shorthand for getting a check out for the current version.

const sub = db.sub('sub-prefix', opts = {})

Create a sub-database where all entries will be prefixed by a given value.
This makes it easy to create namespaces within a single Hyperbee.
options include:
Property
Description
Type
Default
sep
A namespace separator
Buffer
Buffer.alloc(1)
valueEncoding
Encoding type for the values. Takes values of 'json', 'utf-8', or 'binary'.
String
defaults to the parents
keyEncoding
Encoding type for the keys. Takes values of 'ascii', 'utf-8', or 'binary'.
String
defaults to the parents
For example:
const db = new Hyperbee(new Hypercore(ram), {
keyEncoding: 'utf-8',
valueEncoding: 'utf-8'
})
​
// A sub-database will append a prefix to every key it inserts.
// This prefix ensure that the sub acts as a separate 'namespace' inside the parent db.
const sub1 = db.sub('sub1')
const sub2 = db.sub('sub2')
​
await sub1.put('a', 'b')
await sub2.put('c', 'd')
​
for await (const { key, value } of sub1.createReadStream()) {
// 'a' -> 'b'
console.log(key, value)
}
​
for await (const { key, value } of sub2.createReadStream()) {
// 'c' -> 'd'
console.log(key, value)
}
​
// You can see the sub prefixes by iterating over the parent database.
for await (const { key, value } of db.createReadStream()) {
// 'sub1\0a' -> 'b'
// 'sub2\0c' -> 'd'
console.log(key,value)
}

await db.ready()

Ensures that the internal state is loaded. Call this once before checking the version if you haven't called any of the other APIs.

await Hyperbee.isHyperbee(core, opts?)

Returns true if the core contains a hyperbee, false otherwise.
An error is thrown if the first block cannot be loaded. This can only happen in wait: false or timeout: someTimeout configurations (see the documentation for hypercore.get). The default behaviour is to wait until the first block is available, thereafter returning either true or false.

Compare And Swap (CAS)

You have the option to pass a cas function as an option to put that controls whether the put succeeds. Given bee.put(key, value, { cas }), cas is passed the current node (i.e. { seq, key, value }) in bee at key and the next tentative node. Then put succeeds only if cas returns true and fails otherwise.
const cas = (prev, next) => prev.value !== next.value
const db = new Hyperbee(core, { keyEncoding: 'utf8', valueEncoding: 'utf8' })
await db.put('key', 'value')
console.log(await db.get('key')) // { seq: 1, key: 'key', value: 'value' }
await db.put('key', 'value', { cas })
console.log(await db.get('key')) // { seq: 1, key: 'key', value: 'value' }
await db.put('key', 'value*', { cas })
console.log(await db.get('key')) // { seq: 2, key: 'key', value: 'value*' }

const { seq, key, value } = await db.get(key)

Get a key, value. If the key does not exist, null is returned. seq is the Hypercore version at which this key was inserted.

await db.del(key, [options])

Delete a key
options include:
Property
Description
Type
Default
cas
Function that controls whether the del succeeds.
Function
(prev, next) => { return true }
You can pass a cas function as an option to del that controls whether the del succeeds. Given bee.del(key, { cas }), cas is passed the current node (i.e. { seq, key, value }) in bee at key, and del succeeds only if cas returns true and fails otherwise.
const cas = (prev) => prev.value === 'value*'
const db = new Hyperbee(core, { keyEncoding: 'utf8', valueEncoding: 'utf8' })
await db.put('key', 'value')
console.log(await db.get('key')) // { seq: 1, key: 'key', value: 'value' }
await db.del('key', { cas })
console.log(await db.get('key')) // { seq: 1, key: 'key', value: 'value' }
await db.put('key', 'value*')
console.log(await db.get('key')) // { seq: 2, key: 'key', value: 'value*' }
await db.del('key', { cas })
console.log(await db.get('key')) // null
Although some of its internal components are still being developed and/or improved, the API and feature set is largely stable for use.
Building blocks - Previous
Corestore
Next - Building blocks
Hyperdrive
Last modified 3d ago