Move indices between Elasticsearch and OpenSearch.
Kravex is a command-line tool that streams documents from one cluster to another through the bulk API. It watches how the destination is coping and adjusts batch size as it goes. Elasticsearch 5–8 and OpenSearch 1–3, in either direction.
git clone https://github.com/kravex-net/kravex
cd kravex
cargo run --release -p kvx-cli -- kvx.toml- Reads from / writes to
- Elasticsearch, OpenSearch, NDJSON files
- Written in
- Rust
- Runs as
- One CLI binary and a TOML file
- License
- BSL 1.1, then Apache 2.0 in 2029
How it works
You describe a source and a destination in a TOML file and run the binary with the file as its only argument. OpenSearch goes through the same Elasticsearch backend, because both speak the same bulk API. Moving from Elasticsearch to OpenSearch is a matter of changing a URL.
# kvx.toml
[source_config.Elasticsearch]
url = "http://localhost:9200"
index = "employees"
[sink_config.Elasticsearch]
url = "http://localhost:9201" # an OpenSearch cluster works here too
index = "employees"cargo run --release -p kvx-cli -- kvx.tomlWhat it does while it runs
- Adapts batch size to the destination. It backs off when the cluster returns 429s and ramps back up when it recovers. Pick a fixed size, a throughput-seeking mode, or a latency or CPU-targeting controller in the
[governor]section. - Doesn't trust a 200. The bulk API returns 200 even when individual documents are rejected. Kravex reads the response body, and retries the documents that failed.
- Retries with backoff. Retry count and backoff are configurable under
[drainer]. - Authenticates with basic auth or an API key.
What it doesn't do yet
- It copies documents only. Create the destination index, with its mappings and settings, before you start. Kravex doesn't migrate mappings, settings or templates.
- There is no checkpoint file, so an interrupted run can't pick up where it stopped.
- Copying to an older version works only if the older cluster supports the index's mappings and field types, and the index keeps its full
_source. Newer-only field types and disabled_sourcecan't be copied backwards. - It migrates one index per run.Orchestration, with per-index cutover and rollback, is planned and not released.
- You build it from source, which needs a Rust toolchain.
- The version range above comes from the project README. The repo's own Docker setup exercises Elasticsearch 8.15 and OpenSearch 2.19.
Numbers
One dataset, one laptop. This is the Rally geonames corpus: 11,396,503 documents, about 3.5 GB of JSON. Elasticsearch and OpenSearch ran in Docker Desktop on an M1 Pro, with the destination limited to 2 cores and 4 GiB of memory.
| Destination | Documents | Time | Docs / min | Peak memory |
|---|---|---|---|---|
| Elasticsearch 8.15 | 11,396,503 | 8 min | 1.43 M | 135 MB |
| OpenSearch 2.19 | 11,396,503 | 9.5 min | 1.19 M | 135 MB |
Fixed-size batches. Document counts matched the source. The destination's two cores were the limit, so a larger cluster should go faster. Don't read these as a promise for your data, because document size, mappings and hardware change everything. The harness is in the repo's demo/ directory, and you can run it yourself.
Alternatives
You have options for moving index data, and several are good at what they were built for. This is where each one tends to stop short for a large index-to-index move. Claims about other tools link to their sources, and these tools change, so check before you decide.
| Snapshot and restore |
Built into both engines, but the data goes through an intermediate store: source cluster, then a snapshot repository such as S3, then destination cluster. The snapshot has to be finished before it can be restored, and every master and data node in both clusters must be able to reach the repository, so you end up sharing it or copying it between environments. Kravex streams instead. It writes to the destination as soon as it has read from the source, which can shorten the total time and removes the intermediate copy. Restore only moves forward, and only by one major version. Every index in the snapshot must have been created in the target cluster's major version or the one before it. You can't restore into an older cluster, not even an older minor release. OpenSearch also can't restore indexes created by Elasticsearch 7.11 or later. Elastic docs · OpenSearch docs · Compatibility reference |
| Elasticdump | Reports of about 1 GB/hour in a community thread, and single-index reads go through the scroll API. Thread · Issue #11 |
| Logstash | Elastic's write-up of moving a billion log lines saw throughput fall from about 6,000 to 700 docs/sec as pagination got deeper. Blog post |
| Reindex API |
Runs inside the destination cluster. For a remote source, the destination must be able to reach it, and the source host must be on the destination's allowlist. Docs That gets difficult on managed services. Some providers restrict outbound connections from the cluster, or don't give you control of the allowlist. If the source sits on a private network, such as an OpenSearch domain in a VPC, its endpoint has to be exposed to the destination through NAT, peering or a public address, and the destination has to be allowed in. Kravex runs on any machine that can reach both clusters, so neither cluster needs a network path to the other. |
| AWS Migration Assistant | Runs on an AWS stack (CloudFormation, ECS, MSK, S3) and targets OpenSearch only. Aimed at live capture of up to about 4 TB/day. Docs |
| Airbyte | General-purpose data movement. Elasticsearch is a destination connector, and pricing is based on volume. Connector · Pricing |
Rollback is the bigger difference
A snapshot restore is a one-way move. Neither Elasticsearch nor OpenSearch can restore a snapshot into an older version, so once the new cluster takes writes, its snapshots can't go back to the old one. If your rollback plan is the snapshot you took before cutover, rolling back loses every write made since then.
Kravex copies documents through the bulk API, so the target writes its own index files and the source's Lucene version doesn't matter. It can also copy from a newer cluster to an older one, as long as the older cluster supports the mappings and field types in use and the full documents can be read from _source. Orchestrating that fallback is the next piece of work, described below.
Coming: per-index orchestration not released
Today Kravex copies one index per run, and you handle the cutover yourself. Orchestration is planned to take that over, one index at a time, with a way back for each.
- Read-only source. The source index goes read-only while its migration runs.
- Per-index routing. You update your application to point each index at the cluster that currently holds it. Different indices can live on different clusters during the move.
- Rollback. If the new cluster misbehaves, you migrate the index back, even after it has received writes on the new side.
- Backward compatibility. Because an index can move backwards, nothing is stranded on the newer version, provided the older cluster supports the index's mappings and field types.
This section describes planned work. None of it is in the current CLI.
Licensing
Kravex is source-available under the Business Source License 1.1, not an OSI-approved open-source license. On 2029-03-15 this version converts to Apache 2.0.
Use it for your own migrations
Free, in production, with no limits. That covers internal tooling, CI pipelines and side projects.
Sell something built on it
You need a commercial license if you offer a competing migration product or service to others. The same applies if Kravex is a material part of paid consulting, professional services or managed migrations.
Enterprise features
The ee/ directory is under a separate license. It's reserved, with nothing shipped in it yet. If you need something specific there, tell us.
Licensing questions: daniel@kravex.net