Hyperbee
Hyperbee is an append-only B-tree based on Hypercore. It provides a key/value-store API, with methods for inserting and getting key/value pairs, atomic batch insertions, and creating sorted iterators. It uses a single Hypercore for storage, using a technique called embedded indexing. It provides features like cache warmup extension, efficient diffing, version control, sorted iteration, and sparse downloading.
As with the Hypercore, a Hyperbee can only have a single writer on a single machine; the creator of the Hyperdrive is the only person who can modify to it, because they're the only one with the private key. That said, the writer can replicate to many readers, in a manner similar to BitTorrent.
- Basic:
- Properties:
- Methods:
- Methods:
npm install hyperbee
options
include:Property | Description | Type | Default |
---|---|---|---|
valueEncoding | Encoding type for the values. Takes values of 'json', 'utf-8', or 'binary'. | String | 'binary' |
keyEncoding | Encoding type for the keys. Takes values of 'ascii', 'utf-8', or 'binary'. | String | 'binary' |
Currently read/diff streams sort based on the encoded value of the keys.
Current version. An incrementing number, starting at 1, reflects all put/del operations that have been performed on the Hyperbee.
Insert a new key. Value can be optional.
options
include:Property | Description | Type | Default |
---|---|---|---|
cas | Function that controls whether the put succeeds. | Function | (prev, next) => { return true } |
If you are inserting a series of data atomically, or you just have a batch of inserts/deletions available using a batch can be much faster than simply using a series of puts/dels on the db.
Make a new batch.
Insert a key into a batch.
options
include:Property | Description | Type | Default |
---|---|---|---|
cas | Function that controls whether the put succeeds. | Function | (prev, next) => { return true } |
const Hyperbee = require('hyperbee')
const Hypercore = require('hypercore')
// Create a Hypercore instance for the Hyperbee
// value encoding can be set directly in the Hyperbee constructor
const core = new Hypercore('./hyperbee-storage')
const db = new Hyperbee(core, {
keyEncoding: 'utf-8',
valueEncoding: 'binary'
})
const batch = db.batch()
await batch.put('key1', 'value1')
await batch.put('key2', 'value2')
await batch.put('key3', 'value3')
// execute the batch
await batch.flush()
and Get a key, and value out of a batch.
Delete a key into the batch.
options
include:Property | Description | Type | Default |
---|---|---|---|
cas | Function that controls whether the put succeeds. | Function | (prev, next) => { return true } |
Commit the batch to the database.
Destroy a batch and releases any locks it has acquired on the db. Call this if you want to abort a batch without flushing it.
Make a read stream. All entries in the stream are similar to the ones returned from .get and the sort order is based on the binary value of the keys.
options
include:Property | Description | Type | Default |
---|---|---|---|
gt | only return keys > than this value | Integer | null |
gte | only return keys >= than this | Integer | null |
lt | only return keys < than this | Integer | null |
lte | only return keys <= than this | Integer | null |
reverse | determine order of the keys | Boolean | false |
limit | max number of entries you want | Integer | -1 |
const Hyperbee = require('hyperbee')
const Hypercore = require('hypercore')
// Create a Hypercore instance for the Hyperbee
const core = new Hypercore('./hyperbee-storage', {
valueEncoding: 'utf-8' // The blocks will be UTF-8 strings.
})
const db = new Hyperbee(core, {
keyEncoding: 'utf-8',
valueEncoding: 'binary'
})
await db.put('key', 'value')
for await (const { key, value } of db.createReadStream()) {
console.log(`${key} -> ${value}`)
}
Similar to doing a read stream and returning the first value, but a bit faster than that.
Create a stream of all entries ever inserted or deleted from the db. Each entry has an additional
type
property indicating if it was a put
or del
operation.options
include:Property | Description | Type | Default |
---|---|---|---|
live | determine whether the stream will wait for new data and never end or not | Boolean | false |
reverse | determine the order in which data is received | Boolean | false |
gt | start after this index | Integer | null |
gte | start with this seq (inclusive) | Integer | null |
lt | stop before this index | Integer | null |
lte | stop after this index | Integer | null |
limit | max number of entries you want | Integer | -1 |
If any of the gte, gt, lte, lt arguments are
< 0
then they'll implicitly be added with the version before starting so doing { gte: -1 }
makes a stream starting at the last index.Efficiently create a stream of the shallow changes between two versions of the db. Each entry is sorted by key and looks like this:
{
left: <the entry in the db>,
right: <the entry in the other version>
}
If an entry exists in db but not in the other version, then
left
is set and right
will be null, and vice versa.If the entries are causally equal (i.e. they have the same seq), they are not returned, only the diff.
Accepts the same options as the read stream except for reverse.
const db = new Hyperbee(new Hypercore(ram), {
keyEncoding: 'utf-8',
valueEncoding: 'utf-8'
})
await db.ready()
const keys = 'abcdefghijkl'
for (const char of keys) {
await db.put(char, char)
console.log(`Version after inserting ${char}: ${db.version}`)
}
// The createDiffStream method allows us to observe differences between versions of the Hyperbee.
// Let's see what's changed between the latest version, and version 9.
console.log(chalk.green('\nDiff between the latest version, and version 9:\n'))
for await (const { left, right } of db.createDiffStream(9)) {
// Since we've only inserted values, `right` will always be null.
console.log(`left -> ${left.key}, right -> ${right}`)
}
The watcher API is still under active development and subject to breaking changes. Therefore, it is strongly suggested to avoid using it in production environments.
watcher
listens to the changes that are in the optional range
. The range
options include all the properties of createReadStream
options except for reverse.Methods:
watcher.destroy()
This closes the watcher.
Events:
watcher.on('change', (newVersion, oldVersion) => {})
Emitted on any feed change.
watcher.on('error', onerror)
Throws any critical and unexpected errors and the watcher gracefully auto closes on errors.
watcher.on('close', onclose)
Emitted when watcher is closed.
Get a read-only db checkout of a previous version.
options
include:Property | Description | Type | Default |
---|---|---|---|
valueEncoding | Encoding type for the values. Takes values of 'json', 'utf-8', or 'binary'. | String | defaults to the parents |
keyEncoding | Encoding type for the keys. Takes values of 'ascii', 'utf-8', or 'binary'. | String | defaults to the parents |
Shorthand for getting a check out for the current version.
Create a sub-database where all entries will be prefixed by a given value.
This makes it easy to create namespaces within a single Hyperbee.
options
include:Property | Description | Type | Default |
---|---|---|---|
sep | A namespace separator | Buffer | Buffer.alloc(1) |
valueEncoding | Encoding type for the values. Takes values of 'json', 'utf-8', or 'binary'. | String | defaults to the parents |
keyEncoding | Encoding type for the keys. Takes values of 'ascii', 'utf-8', or 'binary'. | String | defaults to the parents |
For example:
const db = new Hyperbee(new Hypercore(ram), {
keyEncoding: 'utf-8',
valueEncoding: 'utf-8'
})
// A sub-database will append a prefix to every key it inserts.
// This prefix ensure that the sub acts as a separate 'namespace' inside the parent db.
const sub1 = db.sub('sub1')
const sub2 = db.sub('sub2')
await sub1.put('a', 'b')
await sub2.put('c', 'd')
for await (const { key, value } of sub1.createReadStream()) {
// 'a' -> 'b'
console.log(key, value)
}
for await (const { key, value } of sub2.createReadStream()) {
// 'c' -> 'd'
console.log(key, value)
}
// You can see the sub prefixes by iterating over the parent database.
for await (const { key, value } of db.createReadStream()) {
// 'sub1\0a' -> 'b'
// 'sub2\0c' -> 'd'
console.log(key,value)
}
Ensures that the internal state is loaded. Call this once before checking the version if you haven't called any of the other APIs.
Returns
true
if the core contains a hyperbee, false
otherwise.An error is thrown if the first block cannot be loaded. This can only happen in
wait: false
or timeout: someTimeout
configurations (see the documentation for hypercore.get). The default behaviour is to wait until the first block is available, thereafter returning either true
or false
.You have the option to pass a
cas
function as an option to put
that controls whether the put
succeeds. Given bee.put(key, value, { cas })
, cas
is passed the current node (i.e. { seq, key, value }
) in bee
at key
and the next tentative node. Then put
succeeds only if cas
returns true
and fails otherwise.const cas = (prev, next) => prev.value !== next.value
const db = new Hyperbee(core, { keyEncoding: 'utf8', valueEncoding: 'utf8' })
await db.put('key', 'value')
console.log(await db.get('key')) // { seq: 1, key: 'key', value: 'value' }
await db.put('key', 'value', { cas })
console.log(await db.get('key')) // { seq: 1, key: 'key', value: 'value' }
await db.put('key', 'value*', { cas })
console.log(await db.get('key')) // { seq: 2, key: 'key', value: 'value*' }
Get a key, value. If the key does not exist,
null
is returned. seq
is the Hypercore version at which this key was inserted.Delete a key
options
include:Property | Description | Type | Default |
---|---|---|---|
cas | Function that controls whether the del succeeds. | Function | (prev, next) => { return true } |
You can pass a
cas
function as an option to del
that controls whether the del
succeeds. Given bee.del(key, { cas })
, cas
is passed the current node (i.e. { seq, key, value }
) in bee
at key
, and del
succeeds only if cas
returns true
and fails otherwise.const cas = (prev) => prev.value === 'value*'
const db = new Hyperbee(core, { keyEncoding: 'utf8', valueEncoding: 'utf8' })
await db.put('key', 'value')
console.log(await db.get('key')) // { seq: 1, key: 'key', value: 'value' }
await db.del('key', { cas })
console.log(await db.get('key')) // { seq: 1, key: 'key', value: 'value' }
await db.put('key', 'value*')
console.log(await db.get('key')) // { seq: 2, key: 'key', value: 'value*' }
await db.del('key', { cas })
console.log(await db.get('key')) // null
Although some of its internal components are still being developed and/or improved, the API and feature set is largely stable for use.
Last modified 3d ago