Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions docs/developers/replication/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,23 @@ replication:

When using controlled flow replication, you will typically have different route configurations for each node to every other node. In that case, typically you do want to ensure that you are _not_ replicating the `system` database, since the `system` database containes the node configurations, and replicating the `system` database will cause all nodes to be replicated and have identical route configurations.

The `replicates` property also allows you to specify routes with more granularity by specifying `sendsTo` and/or `receivesFrom` properties, which each can have an array with node and database names.

```yaml
replication:
databases:
- data
routes:
- host: node-two
replicates:
sendsTo:
- target: node-three
database: data
receivesFrom:
- source: node-four
database: system
```

#### Explicit Subscriptions

By default, Harper automatically handles connections and subscriptions between nodes, ensuring data consistency across your cluster. It even uses data routing to manage node failures. However, you can manage these connections manually by explicitly subscribing to nodes. This should _not_ be used for production replication and should be avoided and exists only for testing, debugging, and legacy migration. This will likely be removed in V5. If you choose to manage subscriptions manually, Harper will no longer handle data consistency for you. This means there’s no guarantee that all nodes will have consistent data if subscriptions don’t fully replicate in all directions. If a node goes down, it’s possible that some data wasn’t replicated before the failure. If you want single direction replication, you can use controlled replication flow described above.
Expand Down
53 changes: 53 additions & 0 deletions release-notes/v5-lincoln/5.0.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
---
title: 5.0.0
---

# 5.0.0

## Open Source and Pro Editions

Harper v5.0 is available in two editions: Open Source and Pro. The Open Source edition is free and open source under the Apache 2.0 license, while the Pro edition includes replication, certificate management, and licensing functionality (the source code is available under the Elastic 2.0 License).
The open source edition can be installed with:
`npm i -g harper`
And the pro edition can be installed with:
`npm i -g @harperfast/harper-pro`

## RocksDB

Harper 5.0 now uses RocksDB as its default underlying storage engine. RocksDB provides a significantly more robust and reliable
storage engine with consistent performance characteristics. RocksDB is well-maintained, has powerful background compaction
capabilities, and a wide array of tuning options and features.

Harper also introduces its own native transaction log as a write-ahead log (WAL) for RocksDB, which drives ACID compliance
in RocksDB, as well as powers real-time delivery of data. This is a highly optimized transaction log designed for high throughput
of messaging and data. This is all powered by our new [open source rocksdb-js library](https://github.com/harperfast/rocksdb-js).

RocksDB will enable robust transactions in Harper, with the complete ability to read and query after writes (and get data from those writes) within a transaction.

Harper will continue to support the existing LMDB storage engine for v5.0, and will continue to load databases created with LMDB.

Switching a database from LMDB to RocksDB requires a database migration. This can be done using replication by creating
new nodes and replicating the data from the old nodes to the new nodes.

### Current Limitations of RocksDB

Currently, there are a number of optimizations with querying, caching, contention monitoring, and write batching that
have not yet been implemented in v5.0, but are planned for a future release. LMDB exhibits better performance for data
that is cached in-memory.
Retrieval of past events is not guaranteed to return every event when concurrent events take place on different nodes. This is often used by non-clean MQTT sessions. However, the latest message is always guaranteed to be delivered, sequences of messages from the same node are guaranteed to be delivered in order, ensuring the correctness of most applications and message retain consistency.
Retrieval of past events for subscriptions will not support a `count` option.
Published messages do not support streamed blobs.

## Resource API Updates

Harper v5.0 has upgraded the resource API with several important changes:

- Harper v5.0 is specifically encouraging the use of static REST methods, and providing functionality to easily use these methods.
- The `target` (`RequestTarget` type) will now be parsed prior to calling static REST methods, for access to any query information in the URL.
- The current request (`Request` type) will be available in any function through asynchronous context tracking, using the `getContext()` function available from the `harper` module.
- The `get` method will always return a frozen enumber record. The return value does not include all the methods from the Resource API (like `wasLoadedFromSource`, `getContext`, etc.). It only includes methods `getUpdatedTime` and `getExpiresAt`.
- When using the LMDB storage engine, Harper will no longer attempt to cache resource instances that can make were used to make a record stored in a write visible in a subsequent read.

## Application Context Separation

Harper now runs each application its own separate JavaScript "context", which has its own global object, top level variables, and module imports. This provides isolation of applications and access to application-specific configuration data and functionality. These contexts will limit access to certain node modules like `fs` and `child_process`. This functionality can be controlled with configuration options.
7 changes: 7 additions & 0 deletions release-notes/v5-lincoln/lincoln.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
title: Harper Lincoln (Version 5)
---

# Harper Lincoln (Version 5)

In honor of [Lincoln](/img/dogs/lincoln.png).
Binary file added static/img/dogs/lincoln.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading